Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFE] add a way to run in a new systemd scope automatically #428

Closed
keszybz opened this issue May 27, 2016 · 44 comments
Closed

[RFE] add a way to run in a new systemd scope automatically #428

keszybz opened this issue May 27, 2016 · 44 comments

Comments

@keszybz
Copy link

keszybz commented May 27, 2016

With systemd 230 we switched to a default in which user processes started as part of a login session are terminated when the session exists (KillUserProcesses=yes). See https://github.com/systemd/systemd/blob/master/NEWS#L3, the commit that made the change: systemd/systemd@97e5530cf20, and bug reports which gave the impetus for the change: https://bugs.freedesktop.org/show_bug.cgi?id=94508, systemd/systemd#2900.

Unfortunately this means starting tmux in the usual way is not effective, because it will be killed upon logout. There are a few option to avoid that, the best being:

systemd-run --scope --user tmux

This starts tmux as a scope unit under the systemd --user instance. It would be great if tmux could do this automatically. Probably the best way to do this would be to make the dbus call to org.freedesktop.systemd1.Manager.StartTransientUnit directly from tmux. See https://github.com/systemd/systemd/blob/master/src/run/run.c#L907 for how systemd-run does it, and https://www.freedesktop.org/wiki/Software/systemd/dbus/ for the description of the API.

@nicm
Copy link
Member

nicm commented May 27, 2016

tmux is not going to use dbus. It sounds like this should be done as part of daemon(3).

@keszybz
Copy link
Author

keszybz commented May 27, 2016

daemon(3) is an old BSD interface, I'm afraid it's infeasible to change it.

I think tmux is the the right place to add such functionality, because it's one of the mechanisms that people use to starts long-running processes from a user session. daemon(3) is called from daemons, which certainly should not start their own scope.

In case it wasn't clear, this dbus call should be compile time and run time optional, i.e. it should probably be only compiled in on Linux systems, and should be disableable with a switch (assuming that the default would be yes).

@nicm
Copy link
Member

nicm commented May 27, 2016

I am not suggesting you change the daemon(3) API, and I don't see why you would need to. tmux wants to leave itself running in the background - daemon() does that, in a way that is vaguely portable and works similarly on all platforms. If you want to change how processes daemonize, why don't you change how daemon() is implemented instead of adding a new API?

@keszybz
Copy link
Author

keszybz commented May 27, 2016

Because I don't want to change how processes daemonize in general, just a few specific processes: tmux, screen, x2go was also mentioned.

There has been a request to add a library function to wrap the dbus call. So far I haven't considered that, but indeed it would make things easier. Would that work for you?

@nicm
Copy link
Member

nicm commented May 27, 2016

Well, tmux uses daemon() so if it needs to change so will everything else that uses daemon() (or does fork/setsid itself) and expects to stay running in the background.

An alternative libc function would be simpler, then we could just use it instead of daemon() on Linux.

@rfrancoise
Copy link

FWIW, speaking as the Debian maintainer I'm okay with adding a dependency on libsystemd to the package, but the code needs to be upstream in tmux. There's a precedent with the utempter support.

@nicm
Copy link
Member

nicm commented May 27, 2016

Well, I wouldn't say utempter is a precedent. We have it because there is no way to achieve the same functionality without making tmux setgid. I don't think we should use it as an excuse to add other dependencies.

My concern is that we have a little function, daemon(), that does a simple little procedure to make a daemon that has worked basically unchanged across multiple platforms for maybe, what, 30 years? Now to do the same thing we need to add 150 lines of new, Linux-only code AND a library dependency.

Sure, maybe the fork/setsid procedure needs to change for whatever reason. But why not make daemon() do all the new stuff for us? That's what it's there for, to hide these details.

If that won't fly, then a similar new API would be a sensible addition.

@mathstuf
Copy link
Contributor

mathstuf commented May 27, 2016

I think there is, alternatively, a way to start a new session through PAM so that systemd knows to leave tmux alone. I had a thread on the ML a few years ago about it, but PAM calls were deemed as not suitable to be in touch itself (similar to the D-Bus calls requested here). I also don't know if the PAM dance is still a valid way to do this though.

@mathstuf
Copy link
Contributor

@nicm
Copy link
Member

nicm commented May 27, 2016

The PAM API is certainly more compact.

Anyway, if we need to do this, the way to do it is to add osdep_daemon to all the osdep*.c files (like we have osdep_event_init) and call it instead of daemon (for most platforms it can just forward directly to daemon() itself).

@nicm
Copy link
Member

nicm commented May 27, 2016

OS X could probably also use it too. It seems they did fiddle with daemon(3) but caused a different class of problems, that needs some other code to fix.

@joshtriplett
Copy link

This is not something that all daemons should do; most daemons should not run in a new scope, but as part of the existing user scope. For instance, gpg-daemon and ssh-agent should not do this.

This is only something that programs like tmux, screen, a VNC or remote desktop daemon, or some other program that represents a distinct "interactive user session" from the program that launches it should do. (And they should only do so when starting the background service, not in the client process that connects to an existing server.)

I do think PAM sessions are the correct mechanism to use here. tmux should start a new user session using PAM, which will do the right thing not only with logind but also with any other PAM session modules that the admin might configure.

@rain-1
Copy link

rain-1 commented May 28, 2016

@keszybv
why don't you fix systemd instead of forcing other programs to add systemd specific code?

@mathstuf
Copy link
Contributor

How would systemd know that tmux is supposed to be special? And tmux and screen are different than your normal daemons (since they are essentially nested login sessions).

@lmachucab
Copy link

FWIW, speaking as the Debian maintainer I'm okay with adding a dependency on libsystemd to the package,

Why? Why should something like tmux depend on systemd? Should stuff like wget too?

Adding a Suggests, yeah, maybe. A hard Depends, no reason to.

@nicm
Copy link
Member

nicm commented May 28, 2016

I have thought about it further and I don't think I will accept a
dependency on systemd or on PAM, such large dependencies are excessive for
this. If a simpler API is provided I will reconsider at that point.
On 28 May 2016 8:44 p.m., "Luis Machuca Bezzaza" notifications@github.com
wrote:

FWIW, speaking as the Debian maintainer I'm okay with adding a dependency
on libsystemd to the package,

Why? Why should something like tmux depend on systemd? Should stuff like
wget too?

Adding a Suggests, yeah, maybe. A hard Depends, no reason to.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#428 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AASkc9jsZd6rdQLU_PbL2apXDyr2Vyzyks5qGJsmgaJpZM4IomDa
.

@mathstuf
Copy link
Contributor

Would you mind me opening a PR with my patches here so that folks interested can easily find it to at least try out?

@shadowcat-mst
Copy link

Or somebody could go find the actual problem @keszybz saw here - systemd/systemd#3005 - which is:

In particular, for my gnome session, if I log out, without KillUserProcesses=yes I get some processes which are obviously mistakes. Even if I log in again, I'm much better starting those again cleanly.

fix that, and stop trying to make systemd break the world because somebody's gnome session doesn't currently exit cleanly.

@joshtriplett
Copy link

joshtriplett commented May 28, 2016

On Sat, May 28, 2016 at 01:37:50PM -0700, Nicholas Marriott wrote:

I have thought about it further and I don't think I will accept a
dependency on systemd or on PAM, such large dependencies are excessive for
this. If a simpler API is provided I will reconsider at that point.

I certainly wouldn't suggest adding systemd-specific code; a systemd
transient unit/scope is the wrong solution for this anyway, as this isn't a
daemon, it's a user session.

However, PAM is supposed to be the correct way to start a user session.
Integrating with it helps with more than just systemd. For instance,
depending on user configuration, starting a new PAM session would avoid
the problem of tmux using an ssh-agent or keyring that doesn't last as
long as the tmux session (because PAM would associate the agent with
tmux).

If you're concerned about the overhead of PAM support, would you
consider providing optional integration with PAM, such as via
--enable-pam-session? If someone supplied a patch for optional PAM
session support (off by default if that help), would you consider
merging it?

@nicm nicm closed this as completed May 28, 2016
@tsypa
Copy link

tsypa commented May 29, 2016

loginctl enable-linger

@natermer
Copy link

natermer commented May 29, 2016

How would systemd know that tmux is supposed to be special?

There are probably a couple decent ways that systemd can be told these are 'special'. First way is to tell systemd these are special via a service file. Tell people that want to use tmux/screen/etc to start them then as 'systemctl --user start tmux'. Second way is by wrapping the tmux/etc commands in a simple shell script that runs 'systemd-run' in a way to indicate not to kill this process.

I am hoping that there is a 'lingering' option that can be set per-user service file or 'systemd-run' to allow this to happen. Being able to set a 'linger' per-service rather then for all user's programs via 'loginctl' seems like a better solution because then it wouldn't be a 'all or nothing' choice.

I feel this is better because it won't require software authors to code special PAM support just to deal with systemd + Linux. It can be used by distributions to maintain expected behavior for these classes of programs. And, finally, it will make life easier for users/developers that wish to make new programs of these types and have things 'just work' by providing a suitable service file in their source code.

@ap
Copy link

ap commented May 29, 2016

So then we get to the next iteration where the “tell systemd I want to stay running in the background” step has become part of everyone’s dæmonization recipe, and now you’re back to square 1 where a user exits their session but a bunch of unwanted processes stick around.

Result: a long period of making everything in world depend on systemd, followed by nothing gained for all the pain.

@kentfredric
Copy link

IMHO, This sounds more like there needs to be a user_daemon() interface so processes can voluntarily state "yes, kill me when the user logs out please". ( And have autoconf just fall-back to using daemon() instead on systems that don't support user_daemon() ).

Changing semantics of existing functionality however seems unwise.

(And then maybe, people who have a foot-to-gun-ratio greater than 2 can tell systemd to prefer one instead of the other, instead of covering earth with a foot assassination squad)

I half expected such a mechanism to already exist, but my reading of daemon(3) and setsid(2) indicates there isn't.

That seems to be why we're where we are now, because everyone who wants "exit on user vanishes" has to manually implement painful mechanics ... so they opt not to and just reach for a full daemon.

@hashworks
Copy link

Honestly, I don't see why processes should be able to decide that they should run forever. I'm completely fine with starting specific processes with systemd user scope. This is just something new for people who use systemd what they need to learn.

@eekee
Copy link

eekee commented May 29, 2016

I haven't had time to participate in open source for a few years. Now I'm coming back to it, I'm a bit shocked by this. Persistent screen and tmux sessions were common enough many years before systemd was released. If systemd won't make it easy for users to set up and manage these sessions, that's systemd's problem, not tmux's.

lilydjwg added a commit to lilydjwg/dotzsh that referenced this issue May 29, 2016
@lilydjwg
Copy link

I've met this kind of problem already: I was using ssh.socket on Debian and every time I logged out, I found my tmux session disappeared....After some debugging I finally learned to set ServiceKillMode=process for ssh@.service.d.

@joshtriplett
Copy link

@lilydjwg That's a different problem. You need to have SSH configured to use PAM (typically the default), and have pam_systemd installed. That will then start a new user session for SSH, rather than leaving the logged-in user processes as part of the SSH socket unit's scope.

@ThomasAdam
Copy link
Contributor

Any chance you can shift this discussion elsewhere? It's not anything to do
with tmux.
On 29 May 2016 1:57 p.m., "Josh Triplett" notifications@github.com wrote:

@lilydjwg https://github.com/lilydjwg That's a different problem. You
need to have SSH configured to use PAM (typically the default), and have
pam_systemd installed. That will then start a new user session for SSH,
rather than leaving the logged-in user processes as part of the SSH socket
unit's scope.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#428 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAGLod76iR13OU2O7CkNIrOaDrHbrBFrks5qGY0ogaJpZM4IomDa
.

@joshtriplett
Copy link

@ThomasAdam @nicm
At this point, I would suggest closing this issue to comments; there isn't any new discussion happening related to the actual problem reported, and judging by the number of unproductive comments and the 👎s and 👍s on comments, a link to the issue most likely wound up somewhere that has little to do with actually solving the problem. The proposal to run in a new systemd scope isn't the right solution to the problem regardless, so this issue as originally reported should remain closed. Someone interested in working on PAM support may want to file a separate pull request with patches implementing it.

@wisq
Copy link

wisq commented May 29, 2016

Yeah, it's making the rounds on Twitter, as a "wtf systemd" sort of thing. Closing comments might be best.

@shadowcat-mst
Copy link

Hey, to anybody who say this because I tweeted it in a fit of aggravation - please use the thumbs up or whatever thing to add 1 to one of the numbers underneath my comment rather than making another comment that doesn't need to be here on an already-closed-as-reject-issue. (@wisq - more part 3 of 3 of a wtf systemd tweet stream of mine, I suspect, but that doesn't help, of course)

@nicm
Copy link
Member

nicm commented May 4, 2020

Was a simple API ever added to allow applications to do this? I don't see anything obvious in the libsystemd source... something like systemd_im_a_daemon_dont_kill_me(getpid())?

On OS X we now do this which is rather inscrutable (if at least compact), if someone has a similar block of code for Linux we can do that too:

        if (bootstrap_get_root(bootstrap_port, &root) == KERN_SUCCESS &&
            bootstrap_look_up_per_user(root, NULL, uid, &s) == KERN_SUCCESS &&
            task_set_bootstrap_port(mach_task_self(), s) == KERN_SUCCESS &&
            mach_port_deallocate(mach_task_self(), bootstrap_port) == KERN_SUCCESS)
                bootstrap_port = s;

Probably best to email me or open a new issue if you have something.

@nicm
Copy link
Member

nicm commented May 4, 2020

OK someone pointed me to some other code that does this a bit more concisely (thanks), so I have worked it out and it not that much more than the voodoo we already have for OS X.

However I only have CentOS 7, Debian and Ubuntu VMs... CentOS 7 seems to be too old, both Debian and Ubuntu have a new systemd but both must have turned off or worked around this behaviour already because tmux stays around when I log out.

On Debian the code doesn't seem to work at all (sd_bus_call gives me either ENOENT or EPERM), but possibly that is a local problem since it doesn't have X installed.

Ubuntu the code runs but I haven't used systemd before and I can't figure out where to see this has worked (systemctl list-units? ps?)... still, tmux runs.

Anyway, if anyone still has this problem, please try applying this against master and configuring with --enable-systemd, you will need the systemd headers and libraries: tmux-systemd.diff.txt

@tmux tmux unlocked this conversation May 4, 2020
@lilydjwg
Copy link

lilydjwg commented May 5, 2020

@nicm Your patch works. The tmux is indeed running inside a scope:

$ systemctl --user list-units | grep scope
  init.scope                           loaded active running   System and Service Manager                                              
  tmux-142.scope                       loaded active running   tmux-142.scope                                                          

And when KillUserProcesses=yes in logind.conf, it keeps running after the current session is closed (when I disconnect the ssh connection). (My system has KillUserProcesses=no by default, and has KillMode=process in sshd.service, so tmux just works well in this situation without this change; however I don't know about other types of sessions.)

However it will still be killed by systemd some time (10s by default) after the user logs out completely if linger for the user isn't enabled (default). An interesting thing is that, by default on my system, when tmux isn't put into a scope and survives a ssh logout because of KillMode=process, systemd won't think the user is idle and should be killed because of lingering.

My system is Arch Linux with systemd 245.5-2 and I was testing with lxc.

BTW this shouldn't need X installed since nothing about X involved (however I do have it installed with the test above).

@nicm
Copy link
Member

nicm commented May 5, 2020

Thanks for testing.

However it will still be killed by systemd some time (10s by default) after the user logs out completely if linger for the user isn't enabled (default)

I don't follow - so even with this change tmux will be killed after logout by default, or is that not the default? I don't mind it being killed if the user explicitly configured it.

BTW this shouldn't need X installed since nothing about X involved

It must be a problem in Debian I guess then.

@lilydjwg
Copy link

lilydjwg commented May 5, 2020

so even with this change tmux will be killed after logout by default, or is that not the default?

Yes, all processes created by the user's systemd instance (basically any processes not created by other users / services) will be killed after the user is completely logged out by default. Users who want to run daemons need to run loginctl enable-linger once (and with this change those daemons (enabled systemd services of the user) will start at boot).

There are chances that tmux without this change survives a ssh logout due to the KillMode setting in the sshd service. So this may be a breaking change for those users....

@nicm
Copy link
Member

nicm commented May 5, 2020

Thanks. Does the same happen with tmux started with systemd-run? I don't see anything different it is doing but perhaps I am missing it...

@lilydjwg
Copy link

lilydjwg commented May 5, 2020

Does the same happen with tmux started with systemd-run?

Yes, they are the same, just with a different scope name.

@nicm
Copy link
Member

nicm commented May 5, 2020

OK thanks. So in summary:

  • This change doesn't fix the problem, but it does mean that users only have to run one systemctl command instead of changing configuration files.

  • However, that systemctl command may come with side effects (daemons started at boot) so they probably won't want to run it anyway.

  • Additionally, it might break existing fixes that packagers or users have put in place.

Does this sound accurate?

Unless there is also a way for tmux to turn off this linger option itself it sounds like this change doesn't bring much benefit, especially since I expect most users or packagers have already solved this is some way.

I'll have a look and see if there is a way to do it later on.

@lilydjwg
Copy link

lilydjwg commented May 5, 2020

The "daemons started at boot" shouldn't be a big problem since users don't have any daemons by default so there will be only the user's systemd instance running. (It does keep user-activated services running, e.g. D-Bus.) On a server users probably want to enable linger so they can run their services.

Yes I don't think this patch brings much benefit given the current situation and users can use systemd-run to achieve the same result if they want. At least for users login with sshd.service, since KillMode=process should be there. (However with sshd.socket + sshd@.service it's not there. I heard there is a DoS issue with this scheme some days ago.)

Personally I use systemd-run to start tmux and my linger state is enabled to run my services on both my desktop and servers.

@nicm
Copy link
Member

nicm commented May 5, 2020

Yes I think you're right, although it is a little unfortunate - my concern at the moment is less people like you who understand what they are doing and more with new users for whom it would be better if tmux could just work out of the box without having to fiddle with additional configuration. But it does seem like many system maintainers or tmux packagers have already made the necessary changes, so perhaps those users are already served well enough.

I'll leave this for the moment unless anything else comes up.

@joshtriplett
Copy link

joshtriplett commented May 11, 2020 via email

@nicm
Copy link
Member

nicm commented May 11, 2020

It's technically possible for a program to set the linger option,
though I think that might be surprising behavior to do by default.

I am not going to have tmux change options that affect other applications as well, that is a receipe for disaster.

It would also be possible for tmux to detect the current state of the
linger option, and print a warning message for the user when they
detach, suggesting that they enable lingering.

I think this is neither one thing nor the other.

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

No branches or pull requests