[setq]

Yes. I've burned hours on systemd related problems that just didn't exist before. From DBus message delivery failures, debugging things that haven't broken for 20 years (which has incidentally got MUCH harder) to the whole shit crock that is NTP and locale management. That and the whole desktop buggery that has been going on for the best part of 8 years now has really put me off. To be honest you haven't really experienced systemd's fuckery until you have a hosed box. It's like negotiating a boot with a pigeon. Any other platform is ZERO hassle and that includes windows, OSX, freebsd, openbsd etc.

And you can't criticise it anywhere (watch the downvotes).

The problem is it lacks the mysterious as in Zen and the Art of Motorcycle Maintenance class "quality". Merely glitter around a turd.

However we're stuck with it so it's suck it up or move on.

On the subject of moving on, you could run FreeBSD to run your servers. Much lower resource overheads, native ZFS, simple service management, some documentation worth more than toilet paper and the best thing of all, more sleep at night. Now there are binary updates and packages it's pretty easy. No more make world of any of that stuff.

[icebraining]

And you can't criticise it anywhere (watch the downvotes).

Nonsense. HN has plenty of comments criticizing systemd, some of them in this very thread. They just have to be substantial, not merely empty rants or technically wrong claims.

[setq]

I apologise. This is the first time I've criticised systemd on HN. I had a massive rant on another site a couple of years ago when CentOS 7 dropped about the catalogue of failures I had been through with systemd, completely backed up with evidence and bug reports and it was hammered by downvotes instantly.

[smhenderson]

Any other platform is ZERO hassle and that includes windows, OSX, freebsd, openbsd etc.

I'd add Slackware to that list, at least for now. I've switched some servers to OpenBSD and a few others to Slackware, plus I've switched to Slackware for a laptop I use.

I'm cautiously optimistic that Patrick will keep systemd out of Slackware for the foreseeable future. It is after all one of the most BSD like Linux distros available.

[setq]

Interesting. I haven't used slackware for many years, so far back in fact that it had to come on CDs because I had a dial up. Will take a look in that direction this evening.

[smhenderson]

I hadn't used it since the mid-nineties but it was the first distro I ever used and, really, my first exposure to any Unix like system. My first copy came on a CD in a book I bought at a software trade show.

I joined a company that used Debian for everything so I ended up switching to it for many years. When systemd came along I decided to look around again; thus my foray into OpenBSD and my return to Slackware.

If you do give it a look you may be surprised at how little some things have changed. Packages are kept pretty current but the installer and the basic tools are surprisingly similar to the way they were when it was first released in the early nineties.

[setq]

Excellent. Downloading now :)

I ran NetBSD and OpenBSD on old sun kit in the late 1990s and early 2000's followed by Debian and then CentOS. FreeBSD got a look in persistently on the side. I'd rather like a step back to the sensibility of times past when you chucked something out and it just did its job until something blew up :)

[smhenderson]

I'd rather like a step back to the sensibility of times past when you chucked something out and it just did its job until something blew up

Then I think you'll love Slackware!

[rollcat]

Add Void Linux to the list, it uses runit.

[petre]

I'll probably switch to Void Linux when the next Ubuntu LTS comes out. I'm running Ubuntu 16.04 on my laptop right now.

[digi_owl]

Didn't Docker switch to using Void as basis for their container images?

[cyphar]

No, alpine. But running systemd in Docker has never /really/ worked (except if you're running on RHEL because they have a bunch of patches to make it work) due to how much shit systemd does that makes it hard to run in a container. Even systemd-nspawn (their container runtime which runs systemd inside the container) doesn't work in a lot of cases.

LXC is the only runtime I'm aware of that actually runs systemd inside containers well, but they had to do a lot of unholy shit to play nicely with systemd.

runc has had countless issues with systemd thinking that it owns the system and it messing with container cgroups.

And don't get me started on the fact that cgroupv2 is specifically designed to only work if you have a global management process for cgroups (can you guess what management process that is?).

[digi_owl]

And surprise surprise, the current cgroups maintainer is a Red Hat employee.

I'm not saying it is planned, but i wonder if there is a echo chamber effect going on...

[cyphar]

> And surprise surprise, the current cgroups maintainer is a Red Hat employee.

Tejun used to work at RedHat, he's at Facebook now and I believe he was working at Google as well. However, he also does contribute to systemd development (recently he got a patch merged that broke every container runtime because they switched to a "hybrid" cgroupv2 setup in v232 which caused countless issues).

[setq]

He who controls the spice....

[JdeBP]

Yes, you should not get started on that, because it is a falsehood and not actually a fact.

* https://jdebp.eu/FGA/systemd-documentation-errata.html#Contr...

* https://news.ycombinator.com/item?id=11845867

[cyphar]

There's no need to lecture me, I am very familiar with cgroups, having contributed to their implementation and also maintain runc which is a container runtime (that obviously uses cgroups quite heavily). I've also discussed these issues with other container runtime developers such as the LXC folks and kernel developers.

So let's talk about the API. First of all, cgroupv2 requires a single hierarchy. This means that if systemd is using cgroups for managing services, you cannot use cgroups for anything else because systemd will get confused if you create any new hierarchies. You may argue this is a bug in systemd, but I would argue it's because you can't have named cgroup hierarchies in v2 (like you could in v1, which is what systemd uses on v1).

But ignoring that "slight" issue, how about we talk about the no-internal node constraints and how subtree control works. First of all, in order to use a cgroup controller you must have all of your ancestor cgroups have that controller activated. So if systemd decides to not use a controller, then you can't use it either (without messing with things that systemd thinks it owns). But ignoring that, let's say you want to create a new cgroup under inside your user session (we've already established systemd won't like that, but let's assume that systemd plays along). You can't just create a new subcgroup (you won't be able to use the controllers), you have to create two and then move all of the other processes into one and then the process you wanted to control into the other. While this may sound okay, you have to realise that as a container runtime you now have to mess with processes that you have no control over or idea what they do. Not to mention that there's no way to atomically move all processes into a cgroup (so there'll be race conditions in trying to set this up).

The "delegation" model of cgroupv2 is effectively based around the systemd delegation model, where the higher level has to semantically grant you the right to manage your own resources. What kind of resource management system requires you to request the right to manage your own resources? prlimit(2) doesn't do that. cgroupv1 somewhat had this issue as well, but there is another cgroupv2 limitation added that actually means that even if you have write access to a child cgroup you still need to have write access in your current cgroup in order to move it into the child. Write access to cgroups.proc is actually a privilege in cgroups, so giving users access to this won't always be desirable, but it also further bakes in the management process design.

I've talked to Tejun on the mailing lists, and it's very clear that he prioritises the model of having a higher level process managing cgroups. In discussions about making unprivileged subtree delegation (something that is necessary for rootless containers to use cgroups) he made it clear that he isn't interested in the feature because it will cause systemd issues because it manages all cgroups on a system.

There's actually even more stuff you have to do to manage cgroups if you're not systemd by the way. I've talked to some LXC folks and we collated a list of 12 of different cases and things you need to deal with in order to use cgroupv2 effectively (and all of them break rootless containers, as well as making container runtimes very "noisy neighbours" as a result). cgroupv1 (despite its downsides) had none of these issues.

The only current user of cgroupv2 is systemd, and they've had several instances where they broke every container runtime because they flipped the cgroupv2 switch early.

Yes this was a rant, but I'm really tired of people defending this. cgroupv2 did make some good decisions, but then followed up by making some truly awful ones.

[throwawaymanbot]

> However we're stuck with it so it's suck it up or move on.

This here is the hubris and arrogance of the entire thing.

I take heart that in the linux community, there is enough technical skill to come up with a viable, more in spirit of linux alternative. And I believe that I shall see the day where it is implemented. Lenny P and the gang should be less arrogant, and better at improving their product with this in mind. No one needs red hats "buy in" for this.

[digi_owl]

Thing is that none of them have the backing of Red Hat's finances. Thus they can'ẗ keep up with the churn that systemd and other RH backed projects produce.

[setq]

This is why we need standards and APIs which are immutable. It doesn't matter what the implementation is then.

The whole of systemd is a comedic fuck you to POSIX and the mentality of a system which is standardised.

[rleigh]

This is exactly why I've moved (and am in the continuing process of moving) more and more stuff over to BSD. Standards matter. Compliance with POSIX and other common standards and conventions are what made the success of Linux possible, and I find it disturbing that the new generation of RedHat developers are keen to tell us that it's old hat and is no longer important. But they are forgetting much of the history and the reasons for it.

Linux didn't get to where it is by throwing its weight around and dictating to others (at least, not much). Its success came from interoperability with everything, making it indispensable glue, and its adherence to standards was an integral part of that. As mainstream Linux becomes more insular and controlled by a single vendor, it becomes correspondingly less useful and less desirable. I don't want to be locked in to a RedHat world any more than I wanted to be tied to Solaris or HP-UX. I get occasional PRs for my software demanding that I add systemd-specific functionality, with the assumption that it's the only thing that matters, and people get annoyed when I refuse to compromise the portability of my already-standards-compliant software with Linux- and systemd-specific hacks.

[pjmlp]

"POSIX Has Become Outdated", ;login: issue: Fall 2016, Vol. 41, No. 3

http://www.cs.columbia.edu/~vatlidak/resources/POSIXmagazine...

https://www.usenix.org/publications/login/fall2016/atlidakis

[setq]

One doesn't win a game of chess by moving all the pieces at the same time.

[JdeBP]

That's a trifle unfair. A lot of this area was intentionally not addressed, for various reasons best laid out in contemporary writings on the subject, by the POSIX standardization effort.

[throwawaymanbot]

I hate to point out that LOTS of Linux Distros did not and do not have Red Hats finances. People have been contributing to it for nearly 20 years just fine.

Not sure how you could even think that people need Red Hats finances to create something in linux that actually functions, and is worthwhile.

[digi_owl]

True for initial implementation of individual components. But RH employ developers that can sit full time and churn the interfaces (not necessarily nefariously) between the components such that only their components stay in sync.

[throwawaymanbot]

What systemD has turned in to, is FAR out of scope, for what RH probably gave encouragement for. This will boomerang back some day.