Hacker News new | ask | show | jobs
by FeepingCreature 3446 days ago
I'd probably respect SystemD a lot more if it measured itself against a modern init system like Gentoo's OpenRC instead of pretending it's invented dependency handling.

https://wiki.gentoo.org/wiki/Comparison_of_init_systems

> OpenRC provides a number of features touted as innovative by recent init systems like systemd ...

6 comments

I'm on Gentoo and remember the switch to OpenRC. I was nervous. But all the old stuff worked! And my custom init scripts? Changed only one or two lines to make them work and a few more edits to make them fully OpenRC like. Backwards compatible FTW!

And, now with OpenRC it still "just works". Fast, predictable boot with logs in normal places that I can easily debug, manage and modify.

Systemd isn't an init system, it's a service (a.k.a daemon) management daemon. Its primary purpose is to restart and diagnose failing daemons cleanly.

Systemd won for one simple reason: it's the only tool that accomplishes this task without bugs. We've been running daemontools for almost a decade in production, and it's a nightmare of bugs. Very glad to be finally switching to systemd.

> Its primary purpose is to restart and diagnose failing daemons cleanly.

If this is true, and speaking as a systemd user for close to five years now, it universally sucks at its primary purpose.

Specifically, whenever a service fails, I've lost count of the number of times systemd has barked out useless errors with 200 lines that boil down to "service has entered failed state". Whenever a systemd service fails, odds are better than even I have to spend two hours debugging why by enabling internal logging in that service, running in debug mode etc.

Like when I tried to switch to networkd, and had the wrong password for a wifi I was connecting to, networkd never told me this in any way I could find. Had to go back to the old solution (after an hour of pulling my hair out) before I realised the password was wrong.

Networkd does not handle WLAN authentication. This is the job of wpa_supplicant, which is the defacto standard on linux in every setup until maybe iwd from Intel takes over.
Yeah, but I'm sure you agree networkd should propagate errors from wpa_supplicant such that they reach the user, instead of piping them to /dev/null (not literally, but you get my point)?
systemd-networkd doesn't know about wpa_supplicant. You would start wpa_supplicant@wlan0.service and see wpa_supplicant errors there.

networkd only springs into action when wpa_supplicant succeeds in establishing the layer 2 connection and the interface becomes UP. I like the wpa_supplicant+networkd combo precisely because of this decoupling between network layers. One day, I'll get off my lazy ass and replace NetworkManager by wpa_supplicant+networkd on my notebook.

networkd doesn't know about wpa_supplicant, just as it doesn't know about openvpn, vpnc, ...

If you want a network manager that does know about those and might give more helpful error messages if they fail, use for example NetworkManager.

Then perhaps networkd should be dropped, because:

repeat after me:

everything eventually fails.

How can you tell when a programmer has graduated from "completely new at this" to "has some valuable experience"? That point comes when they stop assuming success.

Check for error and do something useful with the returned value.

Write tests yourself.

Fail gracefully.

Log status, so you know what was happening just before it failed.

Set reasonable timeouts on external processes.

Systemd is written from the perspective of a laptop user who will hand over the whole thing to a support tech when things go wrong. This is antithetical to the spirit of UNIX, which is not "write programs with one purpose that chain together well".

The spirit of UNIX is this: At any time, a user on the system may decide to become a developer or a sysadmin. The tools and information they need should be available.

Indeed, I'm not using networkd anymore. And it was just an example, the phenomenon exists all over systemd.

So basically, you're saying that systemd et al integrate with everything on my system, except for when it's useful?

And "not knowing about" a service is enough of an excuse to hide any errors of those services it's configured to run?
systemd works somewhat ok for me, because I decided to stick to a very limited subset of the functionality it attempts to provide.

I've also used runit (which follows the daemontools model) as a service manager and I've never had an issue with that. I may just have been lucky though.

For me systemd fails because of some bugs I have repeatedly experienced: - systemd stops reaping zombies for some reason; the OS's PID table becomes full and it's impossible to create new processse -> need to hard reset the machine - systemd overtakes halt/reboot commands which is ok when it works I guess -- but for some reason I sometimes get "operation timed out" (likely because of some bug in systemd or dbus). At this point I have to hard reset the machine to get back to a usable OS.

Imagine if #2 happens to you on a remote machine. In my case, I had to call someone and ask them to reset the machine.

#1 is unthinkable for me, because it's the second thing init is supposed to do (the first one being bringing up the various services). I've never had this happen to me on older Linux or other Unices because let's face it, it's not that hard to do. At that point I honestly thought "how can I expect systemd to provide all the features it boasts when it can't do the easy things well ?".

I've also had daemons that systemd lost track of, apparently because of a wrong setting in the .service file. Now that's not a systemd bug, but it was very difficult to debug because I couldn't "trace" the process starting. On the other hand, with daemontools/runit it's quite simple: manually execute the ./run script and see where it fails. With classical init, run the /etc/init.d/service with sh -x and you see exactly where it fails.

if you need to hard reset remotely, and you've still got a shell:

# echo b > /proc/sysrq-trigger

I've used them, too, but while they always seem like a hassle, I haven't encountered any bugs. What are the bugs that you've found?
> Its primary purpose is to restart and diagnose failing daemons cleanly...it's the only tool that accomplishes this task without bugs.

I've been running runit in production for many years, and it does just this, flawlessly.

Systemd is definitely an init system.
As well as logging daemon, dbus daemon, session manager, device node manager and many other things.
That session manager, logind, is a particular mess.

Here you have a daemon that ties into PAM that tries to second guess the kernel regarding what constitutes a session.

Effectively systemd is becoming something akin to Android. It may be using the Linux kernel, but it is not the GNU/Linux we have grown familiar with over the years.

I don't think the kernel has a session concept that corresponds to the session concept of PAM or systemd. (Yes, PAM also has sessions.)

It has some sort of sessions for processes, but that is just something sharing the same name, not the same concept.

systemd session handling is also particularly joyful to debug.
What bugs are you seeing with daemontools? I've been running it in production for 15 years now (shortly after djb released it) and it's been rock solid the entire time.
OpenRC is still more difficult to write init scripts for than systemd; and offers less flexibility in scheduling inits. For example, I can add a 'before' clause to a custom systemd service unit, and I don't need to modify the subsequent units to depend on my custom unit. This is especially useful when you want a custom forking or oneshot service to reliably run before a standard service shipped with a package in your distro.

In a similar vein, systemd targets are considerably more useful than sysv/OpenRC runlevels. A typical sysadmin can read a couple paragraphs of manpage and set up a new target (analogous to runlevel) in systemd; and it can be used for debugging or recovery in the same fashion that runlevels are in other init systems.

On top of this, there are powerful system services developed in tandem with systemd, in the same repository, which offer well-integrated standard alternatives to things which were superficially different on ever distro only four years ago.

Less flexibility how? Please be specific. Before clause exists in OpenRC and is typically rarely useful.

The things it has are incomplete and too integrated, making them hard to replace with working alternatives that do what you want.

OpenRC's the only init system I've actually liked. It's one of the things I really miss from when I used Gentoo, and I don't know why more distros didn't adopt it years ago.
probably it is not related to technical, SystemD has Redhat in the back initially I think.
You should check out SMF, if you have a couple of hours to play with Solaris.. it handles things that systemd does (socket activation, process management/watching etc;) but in an orthogonal way that makes sense.

I like openrc, I also like runit (for different reasons) but SMF is the gold standard for me and I recommend more people check it out. Even if it has XML elements :(

OpenRC was definitely my favorite init system before systemd existed. Definitely beat out the defaults of distributions who were using the debian/redhat style sysvinit.
TrueOS has just switched from Mewburn rc to OpenRC. They've had some problems to overcome.
Pretty much this. It is designed and developed by people that seems to think that RH's take on rc.d was the be all, end all of scripted init.
(Or Solaris SMF)