[poettering]

You know, because we knew this would be controversial we made sure it was both a compile-time option and a runtime option. Yes the upstream default of both defaults to on, but that's just upstream. We made it very easy and supported for downstream distros to switch between opt-out and opt-in of this option for their users. We have encouraged distributions to leave it on, but we were fully aware that for compatibility reasons this is something downstreams likely wanted to turn off, and most compat-minded distros did, as we expected.

Now I am used to taking blame for apparently everything that every went wrong on Linux, but you might as well blame your downstream distros for this as much you want to blame us upstream about this, as it's up to them to pick the right compile-time options matching their userbase and requirements in compatibility, and if they didn't do that to your liking, then maybe you should complain to them first.

(And yes, I still consider it a weakness of UNIX that "logout" doesn't really mean "logout", but just "maybe, please, if you'd be so kind, i'd like to exit, but not quite". I mean, that's not how you build a secure system. We fixed that really, fully knowing it would depart from UNIX tradition, but that's why we made it both compile-time and runtime configurable)

(Also, nobody has to "incorporate" systemd's library to avoid the automatic clean-up. In fact, there's no library we provide that could do that. What was requested though is to either run things as child of systemd --user or just register a separate PAM session, neither of which requires any systemd-specific library.)

Lennart

[inferiorhuman]

> Now I am used to taking blame for apparently everything that every went wrong on Linux, but you might as well blame your downstream distros for this as much you want to blame us upstream about this, as it's up to them to pick the right compile-time options matching their userbase and requirements in compatibility, and if they didn't do that to your liking, then maybe you should complain to them first.

It's up to you as a systemd developer to pick sane defaults. Claiming that it's okay to introduce opt-out breaking changes upstream and then abdicate responsibility is a quite bit like walking around while waving your hands and arms around and then blaming whoever you hit for walking into you.

[poettering]

Well. What is a distro for then if not for picking the most highlevel of defaults suitable for them?

[inferiorhuman]

> Well. What is a distro for then if not for picking the most highlevel of defaults suitable for them?

IOW the distros maintainers made a mistake by picking systemd? Agreed.

[sfilargi]

You are right. Distros failed us completely by choosing systemd.

[izacus]

Killing software that might be running after a valid login session is a sane default.

[inferiorhuman]

And that's what SIGHUP is for. The process will exit by default. If that's not the desired behavior a handler can be registered. Killing things that are explicitly designed to run after logout is a piss poor default.

[poettering]

We send SIGHUP btw. The kernel's own sending of SIGHUP is bound to the TTY concept btw, which is specific to TTY logins only, not graphical ones.

That said the question is not so much about who sends what, but more about whether a secure system should allow user code to escape lifecycle management or whether logging out means logging out and giving up all resources.

[tyingq]

I get what you're saying. However, I'd probably apply the kernel rule of "when maintaining the kernel, do not do something which breaks user programs/applications". Yes, this isn't the kernel, but it's comparable in being a core function that heavily affects userland stuff.

[pas]

Sometimes the ole way o' logg out is just insecure. And there is no way to conjure up a new backward compatible and secure way. cgroups work well, especially because they are not opt-in. That means programs daemonizing either has to set themselves up as a system service or start a new logind scope (or PAM session, etc. which translates to escaping the cgroup, which requires user approval to remain secure).

[inferiorhuman]

> more about whether a secure system should allow user code to escape lifecycle management

Please stop trotting that tired old line out. It is simply untrue. Systemd does the exact opposite of providing increased security. If nothing else the greatly increased surface area of systemd makes for a less secure system.

The pwnie articulates a number of other ways in which your code and your behavior are actively reducing the security of Linux.

[onlydeadheroes]

I know right, I run openvpn as user nobody and I keep thinking that nobody user better stay logged in!

[v_lisivka]

If you created a problem, it's your duty to provide a workaround or a solution to the problem. Why not provide systemd specific version of `nohup` for such cases and encourage users to use it instead of old and insecure version?

[tyingq]

This. There's a reason the defacto way to keep running post logout was named "nohup". This wasn't some deep dark unknown secret behaviour that was broken.

[zbentley]

It was called that because connected pty devices could hang up. Whether hanging up due to intentional logout or actually hanging up the modem was, and is, left as an exercise to the user. Unless we try to disambiguate it via login/pty manager programs, that is.

[v_lisivka]

Because 1) maintainer can be overloaded, so (s)he will stick to defaults, 2) maintainer needs a logical reason to change default setting to something else, which is not obvious in most cases. Maintainer is not a QA team.

[zpallin]

Look, it's everyone's responsibility, this doesn't just fall on Systemd. While it's clear that Systemd made some difficult changes to how user processes operate, it still performed the due diligence of providing the original behavior as configurations. They should reconfigure their tools. If they're not doing that, then it's not necessarily Systemd's fault that things don't work for sysadmins trying to use their tools.

[youdontknowtho]

Wait a minute. Why isn't it the distro's responsibility to choose the most compatible defaults?

[bjourne]

Isn't it more efficient if 1 upstream picks the sane defaults rather than N distros? The situation was exactly the same when PulseAudio was introduced in Ubuntu. Audio broke for a huge amount of users and according to upstream it was because they had configured it wrongly...

IMO, it is part and parcel of designing great software that you pick as universally agreeable defaults as possible.

[darkpuma]

It's the responsibility of both to pick sane defaults. When the software developer picks insane defaults they are being antisocial, those distro packagers are people too and developers who pick insane defaults are causing unnecessary grief for packagers.

[inferiorhuman]

If you smell shit while walking down the street, maybe someone dropped a deuce on the sidewalk. If you smell shit everywhere you go, maybe it's you, maybe you shat your pants. When you violate the principle of least astonishment you're creating a huge stink.

That you can configure systemd to behave in a less obnoxious manner is well beside the point. Systemd should be unobtrusive and predictable without any extra action on the part of the distribution folks or end users.

That the suggestion is to simply read the code or documentation is the height of arrogance considering how sloppy and insecure the systemd code is (parse error equals root privileges? come on…).

[SahAssar]

Your argument assumes that systemd is simply meant to be a in-place compatible drop-in for what it replaces, which I don't think is something anyone would/should expect. If systemd was meant to behave the exact same way as systems it is replacing then there wouldn't be much point of it. For those cases it sometimes will break things, and will sometimes have settings to follow previous behavior.

[inferiorhuman]

There's plenty of room within the POSIX specs to address service management without requiring kernel integration, breaking userland tools, etc. When your init replacement manages to interfere with the kernel you've done something very, very wrong.

[SahAssar]

Not sure if I missed something here but how has it interfered with the kernel? AFAIK it has broken some userland tools (which is bad in itself in most cases), but actually breaking kernelspace is not something I've heard of.

[pas]

Some distros focus on user convenience some on security. Different defaults are required.

And sometimes security requires breaking compatibility.

[jimrandomh]

There's a bug here, which impacts end users: a variety of programs which are clearly intended to persist in the background (nohup, tmux, etc) are failing to persist. This is a real bug. We care about it. I won't be satisfied until it appears that the bug is on track to be fixed, and a lot of other people won't either.

The options for fixing the bug are:

* nohup, tmux, emacs, etc all take dependencies on systemd and use the new systemd daemonization procedure. This is not a viable path because the maintainers of those utilities have refused (see https://github.com/tmux/tmux/issues/428), and because there are too many of them.

* Each distro separately works around the problem by maintaining forks of nohup, tmux, etc. This is not a viable solution because it's way too many forks; people will be finding broken distro+utility pairs forever.

* Each distro separately works around the problem by putting loginctl enable-linger in /etc/profile and KillUserProcesses=no. This would effectively be overruling a systemd's decision. Some distros won't know they need to do this, and the github systemd repo becomes a trap.

* Or: systemd backs down and changes the defaults so that the old daemonization APIs work again.

If you have a fifth option, we'd all love to hear it. But the status quo is that there's a user-facing bug, and the bug is still there. Rather than make the case for it not being a bug, you're currently making the case for it being someone else's bug, but the "someone else" doesn't actually have the power to fix it. You are the only one with the power to fix this bug.

[inferiorhuman]

> If you have a fifth option, we'd all love to hear it.

Replace systemd with something else.

[NullPrefix]

There's literally nothing wrong with OpenRC

[pas]

Devuan

[apple4ever]

I don't understand the issue. systemd offers the option to override the default. Its literally a config. If its such a big deal, why don't the distros just override it? Its a one time change.

[inferiorhuman]

> And yes, I still consider it a weakness of UNIX that "logout" doesn't really mean "logout", but just "maybe, please, if you'd be so kind, i'd like to exit, but not quite". I mean, that's not how you build a secure system.

As an aside this is the height of arrogance to suggest that the systemd is somehow a more secure alternative. Lest this be considered an empty ad hominem attack, let me quote the pwnie you won in 2017[1]:

> Where you are dereferencing null pointers, or writing out

> of bounds, or not supporting fully qualified domain names,

> or giving root privileges to any user whose name begins with

> a number, there's no chance that the CVE number will

> referenced in either the change log or the commit message.

> But CVEs aren't really our currency any more, and only the

> lamest of vendors gets a Pwnie!

1: https://pwnies.com/archive/2017/winners/#lamestvendor

[arpa]

> giving root privileges to any user whose name begins with > a number

https://github.com/systemd/systemd/issues/6237

oh my god, what a spectacular issue. And, seriously, the Poetterings' response is basically "not my job" and "not a bug". And this person develops something that sits at the core of a modern linux system...

[inferiorhuman]

> oh my god, what a spectacular issue. And, seriously, the Poetterings' response is basically "not my job" and "not a bug". And this person develops something that sits at the core of a modern linux system...

All the while Lennart claims that he's making Linux more secure. FFS.

Edit: I forgot about this

https://igurublog.wordpress.com/2014/04/03/tso-and-linus-and...

> He (Theodore Ts’o) goes on to describe how he previously had to neuter policykit’s security (rendering his system very vulnerable) just to get his system working, and how he has found systemd "very difficult sometimes to figure out".

And:

> As for Kay Sievers, maybe he should rename himself to Kay Sewers, because that’s exactly what he smells of. He told to IETF internet area director and previously DHCP working group co-chair “Tod Lemon” to lmgtfy when he asked about a systemd related git repository.

This gem sums it up perfectly though:

> Yet just two days ago, we see Linus Torvalds (the creator of Linux and maintainer of the Linux kernel), launching into a tirade against – yes, you guessed it – systemd developers because of their atrocious response to a bug in systemd that is crashing the kernel and preventing it from being debugged. Linus is so upset with systemd developer Kay Sievers (gee, where I have heard that name before – oh, that’s right, he’s the moron who refused to fix udev problems) that Linus is threatening to refuse any further contributions from this Red Hat developer, not just because of this bug, but because of a pattern of this behavior – a problem for Kay because Red Hat is also foaming at the mouth to have their kernel-based, no doubt bug- and security-flaw-ridden D-Bus implementation included in our kernels. Other developers were so peeved that they suggested simply triggering a kernel panic and halting the system when systemd is so much as detected in use.

[the_why_of_y]

Only the root user can put such an invalid unit file into a directory where systemd will read it - what is the security impact exactly?

[benchaney]

The security impact is that if you allow a user to choose their own username, and you use a standard POSIX specified way of verifying that the username is valid, and at any point in time you run a service as that user, an attacker can gain root privileges.

[inferiorhuman]

Or if you have a package that generates a service user that starts with a digit. Then you'll be running an arbitrary service as root in which case any vulnerabilities become that much more serious. Or have things regressed so much with systemd that the standard is now verify each and every thing you have the init system do?

The other problem is, of course, the utter lack of understanding Lennart demonstrates by being so dismissive and the increased potential for systemd to be hiding future security vulns.

[youdontknowtho]

You know it's open source and that you could actually get involved? If you submit a pull request and it doesn't get merged you can take your concerns to the the larger group.

As to the stuff mentioned in the pwnie. Those sound like great contributions that would be appreciated.

You could also take your concerns to the distro development group. If that doesn't work you could also customize your distro with a custom build of systemd.

If you still don't get satisfaction you can stop using it.

If you dislike how they do thing you have options. Or, you could just be mean on a forum...

[sametmax]

For what it's worth, systemd makes my life easier.

When I switch distro, it's almost always systemd, and not the system du jour, so I know how it works. Creating service files is a google query away, and makes common use cases a breathe, while advanced features that were hard to bash script yourself into, are now just a few options to type.

I understand that many people may have problems with systemd for their particular situation, but that's not my experience.

As a dumb user with a few laptops and servers that needs an occassional daemon, I'm glad systemd won. I know you get a lot of heat since it came out, so thank you for working on it.

[nine_k]

Sure, systemd solves a number of real problems. This is good.

What is not as good: (1) systemd takes over or duplicates functionality not related directly to its primary purpose, and (2) is not solid enough to trust it in a number of cases, while (3) the developers' attitude does not give a lot of hope that the situation will materially improve.

(Of course, I run a distro without systemd.)

[gerbilly]

> I still consider it a weakness of UNIX that "logout" doesn't really mean "logout"

Ok, but UNIX and it's behaviour has evolved over forty years, and users have a certain set of expectations about it.

Also, it should be noted, systems like UNIX are cultural artifacts. The way they are is the result of forty years of back and forth debate and negotiation and eventually compromise.

I can't speak for all of them, but I think that people that are bothered by systemd are upset that all of history has been brushed aside to make place for the preferences of just a few influential developers.

Whether a feature like logout is "logical" or not, is besides the point. Operating system design isn't just about logic, it's about serving users.

[pas]

Yes, indeed, it's not about logic, as those same users cheer Linux instead of sticking with BSD, and then complain about not being UNIX enough.

[toomim]

That was the point of OP's article. That it's hard to change.

[sfilargi]

Completely agree. The problem is not upstream, but downstream. Distros should have done better job and chosen a better default system manager and not systemd.

You build your software the way you want and like. If others don’t like that it breaks POSIX they should stop using it instead of complaining. Or fork it.

[majewsky]

> What was requested though is to either run things as child of systemd --user or just register a separate PAM session

When you run your screen or tmux below `systemd --user`, you still would have to `loginctl enable-linger`, no? I remember having to do that when I set up a PulseAudio server on a headless machine where I don't maintain an active session.

[jjolla]

> still consider it a weakness of UNIX that "logout" doesn't really mean "logout" ... I mean, that's not how you build a secure system

so, unix has been running for 20+ years laden with this security flaw? strange that nobody has been screaming out to plug it all this time.

this feels like you have a bee in your bonnet that it is not a very 'pure' logout by some interpretation of what a "logout" should be. imho, "logout" should mean what it has always meant in the past.

[tasuki]

Lennart, thanks for the information. Mind explaining why you chose to kill user processes on logout as the default?

[poettering]

I think my comment above explained that already.

[belorn]

I think tasuki is asking you to elaborate a bit further on what kind of security issues you have solved by not using SIGHUP signal. I would personally also like to hear more in-depth details, preferable with some examples of security vulnerabilities that was caused because of that POSIX design choice.

[poettering]

Well, this boils down to: in a modern operating system, is it good design that an unprivileged user who logs in once can consume arbitrary runtime resources uncontrolled, unbounded forever, even after logout just because they decided to mask SIGHUP? I think not, I think the system should default to behaviour where unprivileged processes are clearly lifecycle bound, and when the user's sessions end they end comprehensively. I mean, other OSes don't really allow this unprivileged either, for good reasons: the lifecycle of the unpriv user's processes should be controlled by privileged code, and clearly be defined by the act of logging in and logging out in its lifetime.

It's entirely OK if the admin then opts out specific users or even all users from this behaviour, i.e. if a privileged players decides to liberalize unbounded, unlifecycled resource consumption for unprivileged players. But a default where unprivileged code can just stick around uncontrolled and consume as much as it wants forever is just a strange choice security wise.

i.e. I think the fact that SIGHUP masking is unrestricted, i.e. is not subject to privilege checks is the problem really. Something is unpriv by default that should be priv by default. And that's pretty much what this option in systemd provides you with.

[mfritsche]

> Well, this boils down to: in a modern operating system, is it good design that an unprivileged user who logs in once can consume arbitrary runtime resources uncontrolled, unbounded forever, even after logout just because they decided to mask SIGHUP?

This was well known and accounted for where necessary. You considered everyone else to be wrong about the issue and went ahead and fixed it according to your opinion. Don't be surprised that a considerable portion of "everyone" doesn't agree with you.

[pas]

> This was well known and accounted for where necessary.

Could you please explain that in a bit more detail?

[dooglius]

> is it good design that an unprivileged user who logs in once can consume arbitrary runtime resources uncontrolled, unbounded forever

A unprivileged user can still do this by setting up an intermediary box that keeps a persistent ssh session open. Incidentally, this is exactly what I plan to do if I ever need to ssh into a server with KillUserProcesses=yes.

> other OSes don't really allow this unprivileged either

On Windows, if I remote desktop from a laptop into a desktop, and start a web server, then shut down the laptop, the server stays running. On iOS if I start drafting an email, and reboot my phone, I don't lose my work. On ChromeOS, my tabs will stick around after a system crash. The world is moving toward processes being _more_ persistent, not less.

[arwineap]

Windows has a different concept for services and processes. All of your processes are killed when you logout

[pas]

If you already have a middle box, then great, but usually malware (eg a nasty Chrome extension) likes to stick around to snoop on user activity. (Preferably on all user activity, forever.)

[jimrandomh]

> Well, this boils down to: in a modern operating system, is it good design that an unprivileged user who logs in once can consume arbitrary runtime resources uncontrolled, unbounded forever, even after logout just because they decided to mask SIGHUP? I think not, I think the system should default to behaviour where unprivileged processes are clearly lifecycle bound

If there were some way to design this so that nohup would give a permission denied error on start and tmux would give one on detach, rather than die on logout when it's too late to display a warning, that would be a lot better. There may not be a feasible way to do this, but it would solve a key part of this problem, which is that people don't find out about this behavior until something has already gone wrong, and don't find out that systemd is responsible for the behavior until after they've gotten frustrated enough to be mad about it.

[belorn]

From a hosting perspective I understand the issue being addressed but I don't see specific problems being solved. For example, if I lock out a compromised account by locking the unix user they can still be currently logged in with running processes which I then need to manually address and kill, and they can also have cron jobs which restarts them. Services like Apache (with mpm_itk) will still change user-id to those locked users. There is no general system-wide method to declare that a user and all its connected aspects should stop being available, and therefore a compromised account must currently be handled rather individually.

What I see most companies in this industry do is use per-user virtual machines to address the issue, which completely bypasses the question about logged in and logged out. It would be interesting if the intention in current development is to give us administrators more options here and allow for cleaner handling of compromised accounts.

[mruts]

But the point is that Linux and Unix isn't a modern operating system. It's ancient, and built upon decades and decades of work by hundreds of thousands of developers. You can't just decide to break norms handed down through the decades.

And I don't think anyone really had a problem with the old default of letting things run. Worse is better, after all. Pursuit of a perfect system will just make things too complicated, too brittle, and too obtuse.

I like Unix because it doesn't try to solve every problem. It's a libertarian operating system, if you will. Sometimes this causes problems, sure, but if the system is simple and liberal, you can always fix them without much effort.

[m4r35n357]

They show that you do not understand what "log out" is for in Unix.

[yellowapple]

I feel like the kernel policy of "don't break userspace" would be a valuable one for y'all to adopt.

[loudtieblahblah]

>You know, because we knew this would be controversial we made sure it was both a compile-time option and a runtime option.

This is standard from you. You knock the glass on the floor and blame the maid service for not cleaning up after you.

It's everyone's faults but yours.

>And yes, I still consider it a weakness of UNIX that "logout" doesn't really mean "logout", but just "maybe, please, if you'd be so kind, i'd like to exit, but not quite".

Oh how hyperbolic. Nuances and caveats in terminology is not a weakness.

I don't see why you're splitting hairs over this but can't be bothered to care about your UID numbering bug.

Or he fact systemd-resolv is responsible for DNS leaking on VPNs.

But yes, tell me more about how a functionality that enables terminal multiplexes is a "weakness"

>Now I am used to taking blame for apparently everything that every went wrong on Linux,

It's because of your smarmy, arrogance.

You break POSIX compliance, which has a real world effect in multiple areas and you accept bug reports with the humility of Donald Trump being interviewed by MSNBC.

Then when you retreat into your safe space, you play victim to the situation you created.

You talk of Linux culture toxicity, smearing the likes of Linus Torvalds, while essentially being the metaphorical sibling putting your finger in people's face repeating "I'm not touching you" over and over. Then you acted attacked when someone claps back.

You're a cry bully hiding behind a vaneer of professionalism acceptable for Red Hat's HR department which enables you to mark one more bug as "wontfix"; your attitude, your arrogance, your conceits that things not broken in fact, are so you can provide solutions no one asked for and no one benefits from.

[ownagefool]

To be fair, at least poettering presented an argument and is responsible for software that helps a whole bunch of us get things done.

You're just kind of yelling, and it diminishes any point you may have made.

[zeveb]

After awhile, anyone who deals with Lennart just starts yelling, because he is impossible to reason with. He's very intelligent, and absolutely convinced that his is the One True Correct Right Way. It doesn't matter than hundreds or thousands of voices oppose him; I don't think it would matter if every single human being on earth opposed him.

What makes it worse is that he's often not completely wrong. Linux did need something like PulseAudio, something like Avahi and something like systemd. But his reach exceeds his grasp (which probably applies to us all, as I've found on my own projects), which leads to the well-known problems of PulseAudio & systemd.

I don't actually want him to quit the Linux world. But I wish he would scale back his ambitions just a tad, and consider that maybe — just maybe — other people have some good points, and valid concerns.

And also Windows/DOS are not terribly good design exemplars.

[SahAssar]

I get what you are saying, but

> It doesn't matter than hundreds or thousands of voices oppose him; I don't think it would matter if every single human being on earth opposed him.

makes it seem like everyone that uses systemd hates it or sees the same flaws as you or the other people yelling.

I and many others started admin'ing during or slightly before the systemd transition (ubuntu14->16 and rhel6->7) and have found it a much easier path to running services in a sane way than before. It was certainly possible before it, but with systemd I can do it a lot better and easier than I would have been able with previous inits.

For every person saying that systemd made things worse I expect there to be 10 silent sysadmins that appreciate what it did. I have no evidence of that, but that is my experience.

[loudtieblahblah]

It does a lot more than manages services.

It breaks screen and tmux functionality, leaks DNS when connected to a VPN, it riddled with "wontfix" security vulnerabilities stemming from a refusal to be POSIX compliant.

Systemd replaced udev for crying out loud.

[SahAssar]

That might be true and still not contradict what I said. A lot of the systemd critics still seem to not see what it actually did for most people using it. You're free to hate it and some of that is certainly justified, but don't assume that the contrary opinion is based on uneducated or misguided opinions.

Most of what I see/use of systemd I like. Some of it I don't, and some of it is a dumpsterfire. I think I could say the same or worse for any ambitious software project.

As for the security issues I certainly place those in the dumpsterfire category and I'd like for the systemd team to handle them better.

[apple4ever]

I agree. I love systemd as compared to the other ways (though I think launchd is pretty nice too).

[Asooka]

So you made the default the worst possible option, because... why exactly? And now that the problem is apparent, you haven't changed the default because...? I don't know what goes through your and the rest of the systemd's team's heads, but good software engineering it is not.

[vesak]

You've done great work as a whole, as you probably know. Try not let the lowlifes get to you.