[thasmin]

People who aren't extremely familiar with how the Linux init system works and whose job doesn't include keeping the servers stable don't see why the neckbeards are up in arms about systemd, but there's good reason. Many peoples' jobs depends on making sure the servers are working, and knowing how the servers work is a big part of their job (and sanity). In its current incarnation, systemd changes the fundamentals of how servers work without much increase in features - large risk and little reward.

[zanny]

I've had huge productivity gains with unit files for systemd over trying to write spaghetti shell code for old sysv. The syntax and features are well documented and writing them is extremely simple. I also don't believe any sysv implementation had crash recovery or socket activation of daemons, both of which are huge feature wins.

[nitrogen]

I also don't believe any sysv implementation had crash recovery or socket activation of daemons, both of which are huge feature wins.

That's because there were other components handling those tasks, like inetd and /etc/inittab. I do like having Upstart handle respawning for me, though.

[icebraining]

Inetd only did TCP socket activation, not of unix sockets, though.

[thwarted]

Inetd only did TCP socket activation, not of unix sockets, though.

False.

http://manpages.ubuntu.com/manpages/hardy/man8/inetd.8.html

    The service name entry is the name of a valid service in the file
    /etc/services. … For UNIX domain sockets this field specifies the
    path name of the socket.


    The protocol must be a valid protocol as given in /etc/protocols.
    Examples might be “unix”, “tcp” or “udp”. … A protocol of “unix”
    is used to specify a socket in the UNIX domain.

xinetd does not appear to support this feature.

[icebraining]

Oops, you're right, I misread an article about socket activation.

[rodgerd]

> People who aren't extremely familiar with how the Linux init system works and whose job doesn't include keeping the servers stable don't see why the neckbeards are up in arms about systemd

Hi, I'm a sysadmin who's fed up with neckbeards (most of who apparently don't know much and refuse to learn) claiming to speak for all sysadmins on this topic.

> large risk and little reward.

It's four years old, and claiming "large risk and little reward" is like listening to someone claim that moving from sendmail to postfix would be a disaster.

[vacri]

The only sysadmin I know with an actual neckbeard (over a foot long) is a 20-year unix/linux admin, and he greatly favours systemd.

Perhaps if you're tired of neckbeards speaking for all sysadmins, you should return the favour and not declare what all neckbeards are saying. A lot of old, experienced admins are for systemd. It's not the young go-getters who are at the top level of distros making the foundational architectural decisions, after all.

[johnny22]

I deal with sysadmin stuff for quite some time(since 2001), and I just won't use distributions without systemd.

The benefits far outweigh the risks (imo obviously)

[zobzu]

Meh I don't mind either way. There's nice stuff in systemd but nothing thats so critical i wouldnt use a sysv based system (there's pretty good ones).

What generally annoy me are things like supervisor and other things people use to "auto restart" services but these aren't exactly integrating nicely and put stuff all over the filesystem/etc.. I like that systemd includes that and does it mostly properly.

[lucian1900]

Servers is precisely where I want systemd!

There are some things I've wanted reliable and consistent mechanisms for so long: starting/restarting/inspecting services, isolation/resource limiting, socket activation, log collection.

[toolz]

Server admins have so much new technology to learn and play with to stay relevant. If they feel like learning systemd is a chore, they might be in the wrong business.

[dredmorbius]

All the more reason to be judicious in what new technologies are introduced.

One of the huge benefits of the Unix/Linux, CLI, and Free Software traditions is that they tend to be very strongly preserving of established knowledge. Changes are incremental, usually additive, a reliance on scripting means that interfaces are unlikely to change, and new tools are very frequently drop-in replacements for old.

As specific examples:

I first learned editing under BSD vi in the mid 1980s. In the time since I've learned and used on various PCs (and a few other systems): WordPerfect, WordStar, MacWrite, AmiPro, several iterations of MS Word, the EDT and EVE editors under VAX, the TSO-ISPF editor, and a few others under Unix: emacs, ae, nano, nedit, Abiword, Lyx, and various iterations of what's now LibreOffice. Most of that skill-acquisition is now dead to me -- the tools simply aren't available or aren't useful.

I'm no longer using vi, but vim (adopted in the mid 1990s as I switched to Linux), but the basic muscle-memory is the same. And its an editor I can utilize across a huge number of systems (though I do admit to finding traditional vi / nvi painful).

Similarly, the bash shell is an iteration on the basic Bourne and Korn shells.

ssh is a drop-in replacement for rsh, to the extent that /usr/bin/rsh is typically a symlink to ssh. While the dynamic is slightly different from telnet, it's still pretty similar with a few exceptions.

The rare occasions in which a utility changes its commandline options you'll virtually always hear about it. The fact that it's so painful (and tends to break decades-old scripts) means its generally avoided. Authors who make a point of doing this tend to find that people avoid their tools.

A bigger point is that forgetting stuff is often much harder (and more important) than learning stuff. And when you're invalidating long-established patterns, that's really painful.

There's also the fact that we manage technology by managing complexity, and most of us in the field work at the limits of our ability to manage the complexity we're faced with: the basic OS, shells and interpreters, hardware, vendors, hosting providers, management tools, employers, clients, customers, co-workers, engineering and development teams, services, abuse and security concerns. It's a really complex and dynamic field.

Linux has done quite well (with a few notable exceptions) of maintaining a balance between capabilities provided and complexity imposed. One problem is that as systems become more complex, the additional benefits of yet more complexity are lower, and the costs are higher (this is a very general rule, not just specific to Linux, operating systems, or computers).

The question of how to introduce radical change is a key one. I've seen a number of failed attempts to drastically revise existing systems in place -- this almost always fails. Linux itself wasn't introduced in this way -- it emerged as an alternative to both "traditional" proprietary Unices, to Big Iron (mainframes, VAX), and Microsoft's then-new WinNT. Linux ended up dominating virtually all of these categories, but it did so by incrementally beating out the competitors through replacement.

An interesting space where a lot of this comes to a head specifically is in the graphical user interface field. I've noted several times that Apple, notable for a great deal of success in this area, has been exceptionally conservative in its GUI development. It's effectively had two GUIs, the initial Mac System interface, and Aqua. Each has had a roughly 15 year lifespan, and yes, there was incremental improvement over the span of both, but the essential base remained the same.

Since the early 1990s, I've watched Unix/Linux go from twm to fvwm, Motif/mwm, VUE/CUE (a "corporate" standard based on Motif plus a desktop), Enlightenment, GNOME, and KDE, and now alternatives such as xfce4 and ... oh, that funky graphics thing Suse's got, as the "primary" desktops. GNOME and KDE themselves have gone through about three major revisions. And there are a number of other "lesser" more minimal desktops as well -- I use one of these, WindowMaker, which is actually based in a late 1980s ancestor of the Aqua interface now used by Apple.

Microsoft's experienced some similar recent tribulations. As has pretty much every online site ever that's done a site redesign.

As jwz has observed: changes to GUIs just don't offer that much win. They're highly disruptive, they're possible because the interfaces generally aren't scripted (other than via automated QA testing systems, but that's another story), but more importantly: the productivity benefits granted users really aren't that significant, especially regards the cost.

Worse: changing an existing interface leaves users in a no-recourse situation, especially in the case of SAAS. For Linux and systemd, the options are slightly more open in that (for now) it's possible to disable or block systemd from installing in at least some cases. But over the long run, it may be that the only options are voice and exit, as opposed to loyalty (a reference to the book and concept of Voice, Exit, and Loyalty, which I recommend looking up).

So yes: those of us with numerous decades of experience in the field often do have an extremely jaundiced view toward radical change. And with very good reason.

But your comment is really unwarranted.

[bcantrill]

Wow, great comment -- and one that all who endeavor to innovate in systems should take to heart. As my former colleague Bart Smaalders was fond of saying, "the hardest software to upgrade is the software in our brains"; when inventing new abstraction, it must be done so sparingly and (as much as reasonable) by leveraging extant notions. This isn't merely to allow a technology to be readily understood (though that too, certainly); it also requires thinking in terms of reinvention versus reuse. This thinking enforces a kind of humility: you must learn about the systems that have come before, if only to understand which of their abstractions can be reused. I think it is a perceived lack of this kind of humility in systemd that has been so alienating for those who have a long history with Unix: it's not as if other approaches are being rejected so much as they are not being considered at all.

[microtonal]

I think it is a perceived lack of this kind of humility in systemd that has been so alienating for those who have a long history with Unix: it's not as if other approaches are being rejected so much as they are not being considered at all.

I really have the feeling that people are using double standards here, especially when suggesting Solaris or Solaris-derived systems. Since systemd is implementing pretty much what has been in Solaris (SMF) and OS X (launchd) for a while now:

https://docs.oracle.com/cd/E23824_01/html/821-1451/dzhid.htm...

https://developer.apple.com/library/mac/documentation/Darwin...

Also, it is of somewhat questionable ethics that members of the Solaris community submit such troll posts (as others have pointed out, there is not much substance there). It reeks of wanting to destroy Linux' image for your own (Illumos, SmartOS) gain.

[cthalupa]

This is a rather disingenuous response.

It assumes that this is a troll post - which I don't think is fair. The author has concerns that are legitimate to them, and outright dismissal as a troll, whether or not you agree with them, is petty and judgmental.

Second, you are somehow conflating dislike of systemd with love of sysv init. The cognitive dissonance here only makes sense to me if you believe that systemd is perfectly fine, and think that the only reason people dislike it is because it's different.

However, if someone is recommending a solution that utilizes SMF, is it such a stretch to think that it might not be because they are in love with sysv init, and instead might think that the implementation of systemd is lacking?

I personally like the underlying idea of SystemD - because I like SMF. I do not like the implementation of systemd, and also have reservations about the people helming the project.

[lil_cain]

SMF is a pita - far, far worse than the process management stuff in systemd.

SMF does not seem to want to own every bit of my Linux machine, however.

[toolz]

I want to know how this even became an argument in the first place.

It's not that I don't like systemd, it's that [insert affiliated party] is way too cocky

It blows my mind to see people regress so far back into arguments that this because an issue of emotions in a technical debate.

[dredmorbius]

It's not a matter of emotions.

It's a matter of having observed similar behavior in other projects which went similarly off the rails.

Poettering's own track record with Pulseaudio comes to mind. There's also the GNOME project, which I identified as actively intelligent-user-hostile around 2004. It's been somewhat gratifying to see that particular perception bear out with time.

There are other projects which have shown similar levels of arrogance, though mostly with more limited and self-contained damage.

And being prickly or hard to deal with has shades. Neither Linus nor Theo de Raat are pussycats, but both focus very much on technical issues and are generally highly responsive to specific technical complaints. Sure, they make mistakes and bad calls occasionally, but on balance they've tended to get things right.

The attitudes expressed by Poettering and Sievers in particular aren't simply cocky, but contemptuous. And they're getting called on it. Including by Linus.

I could give a shit about personalities themselves, I really could. For the most part I really don't care how socially awkward someone is if they're good at their job. And if they don't start going out of their way to do harm to me or others. Personality disputes in discussions bore the piss out of me.

But I'm also not blind to technical failings with roots in personality traits. And those are what I'm seeing in the systemd crowd and leadership.

[deirdres]

LOL. Exactly. When you don't have any good response based on the merits of the argument, attack the presentation. This tactic is very familiar to those of us who call out sexism or racism: "You should be nicer when asking for your problems to be taken seriously."

But, ultimately, what happens in tech is as much about people and personalities as it is about actual technical merit. To delude ourselves otherwise is dangerous. When someone claims to be arguing from technical merit, look very closely at their history and probable motivations. There's always more there.

[dredmorbius]

Thanks.

[wazoox]

Exactly that. You know what's insanely great? When you watch this presentation video of 1978 at AT&T where Ken Thomson explains Unix and type some commands on his VT-52 and you think: all of this is still current knowledge, and all of his explanations still hold true. Just like celestial mechanics or Pythagorean theorems. We are heirs of this ancient wisdom and this is friggin good, this is culture.

[XorNot]

And systemd is changing precisely none of that.

Nothing about systemd removes the basic unix command line. Because he's most definitely not explaining the init system, which wouldn't have been the same from year to year then, or even similar decade to decade.

[dredmorbius]

Systemd does touch numerous parts of Unix as it existed in 1978: logging, authentication, and devices come to mind. But much of what it's interacting with came along afterward: networking, far more services than existed at the time, a much more complex security scope, and more.

But that's still a good 25-30 years of work, experience, practices, and smoothing out the rough edges that will be shot down the drains.

Systemd also fundamentally changes the control locus of key features within Linux and how applications, the kernel, and OS as a whole are constructed and constrained. Putting all of that under the control of a small group with highly evident disdain for any "outside" concerns (in quotes as these are of the larger Linux community, and the concerns are most decidedly inside that group), contempt, and plays-poorly-with-others attitudes.

I'm not impressed.

Nor with your comment, FWIW.

[XorNot]

Authentication is done with PAM and Kerberos these days - Kerberos is late 1980s, PAM came along in the 90s. Unix evolves and had continued to do so since its inception. udev certainly changed how we do devices.

The rest of your comment is fear mongering which could be applied to any group of core devs on any OSS project in existence. After all who controls Debian and security defaults? Do YOU trust them?

[mercurial]

It's an interesting take, but that's not really how software works. Look at Plan9. It isn't POSIX compliant, but it does a lot of things much better than traditional Unix (or nowadays Linux, for that matter). Traditional Unix is not the philosopher's stone. There are plenty of good things about it, but it also comes with a number of dubious design decisions or what is now irrelevant cruft (why are we leaving with code replicating the behaviour of obsolete hardware in our terminal emulators?). It's not so much the actual implementation that is important, it is the "good parts" of its philosophy that we need to keep.

[dredmorbius]

Plan9 is actually a really good example to bring up, for any number of reasons. I have to admit that I've never used it, though I've read bits about it. There are definitely some ideas in there that I'd like to play with and experience.

The most important elements to consider about Plan9 are these:

1. Plan9 wasn't Unix (nor was it Linux). It was its own OS, it was absolutely informed by Unix, and tried to learn from mistakes practiced in Unix. Because it wasn't Unix it provided for an independent test bed in which these ideas could be explored without disrupting a large established installed base and user community. And that is a key benefit of branched development. All of these I consider positives of Plan9.

2. It was hampered by an overbearing corporate control and licensing model. It was an ugly stepchild of AT&T's, under a proprietary license. The fact that it was under development kept it from being widely deployed (among other factors), the fact that it had a restricted license meant that other possible collaborators couldn't get involved.

When Linux emerged in the early 1990s, it had a lot of problems -- it was far from the best or most obvious Unix alternative out there (look up ESR's PC Unix guides from that era). But in a world of large proprietary Unicies priced far out of the hobbyist's range, a handful of small PC ports of varying quality, and BSD which was embroiled in its lawsuit with AT&T (speaking of Plan9), Linux was unencumbered, free, and (pretty quickly) available under the GPL. That gave it the critical mass to develop. As with Plan9, it was its own OS, providing a testbed environment for development, but also allowing stable cuts to be made for use in specific deployments as it reached sufficient states of readiness.

Which is to say: the community and development dynamics mattered a lot.

I'm seeing a far more troubled path for Systemd in this regard.

Also of note: in the Debian init system debate, a specific concern raised against upstart, one of the init alternatives, was its own requirement of a developer license grant to Canonical, which was seen as a strong demerit against upstart. As with Plan9, exercising too much proprietary control may well have cost Canonical critical votes in the Debian decision.

[mercurial]

It seems that folks outside of Red Hat do contribute to systemd, if that's your concern. What I could imagine is that some projects under the systemd umbrella will live an independent life, once things stabilize a bit.

I must admit the ever-growing scope of systemd is starting to concern me somewhat (though I've been running it with satisfaction more or less since it became available in Debian experimental).

[pjmlp]

What can I say, some people in 2014 like to live in 1978.

It was fun for a while, but I grew out of it.

[wazoox]

You completely missed the point. It's not about living as in 1978, it's about _not throwing away_ accumulated knowledge. What use is my CP/M knowledge nowadays? None at all. What use is the knowledge I could have of '78 Bourne shell, pipes, signals, vi, ed, awk, grep, man? Not only useful, but still of daily use.

[pjmlp]

Cars from last century still take me from A to B. It doesn't mean I want to drive one.

[toolz]

I don't accept that being irritated at 'radical change' (I'm not sure exactly how this is so radical, it's an init system, not a kernel.) because you're losing domain knowledge is a good reason. That would imply radical change is necessarily bad and should be avoided everywhere. Carriages to cars, ice houses to refrigeration, bow and arrow to black powder. Every single one of these radical changes required people to learn something vastly different. With your reasoning we should've waited longer for some intermediate technology to smooth the learning curve. People were getting really good at taking care of horses, now they need auto mechanics? Building an ice house was becoming a science, how do I fix leaking refrigerant? My aim and dexterity with a box is second to none, but now we're using guns? Now your first thought might be that all of those things are different than server admin, but are they really? I would suggest the only thing different for server admins is that it's entirely less radical of a change than all of these other technologies.

As for my comment being unwarranted, sysadmin'ing requires learning new tech. If there is an improvement on a tech such that it has mass adoption, learn the tech. It's your job. If you don't like it, change jobs. I'm not saying you should shut up and put up. However, we're far past that stage of valued input and people are still complaining. The decisions have pretty much been made that are going to be made concerning systemd adoption. Yet here I am, reading yet again how systemd was the wrong choice, even though rigorous debate was had and core teams decided it was the best decision. Even though this was the biggest drama piece since that blogger blasted linus for being rude. Here we are with 'radical change' in systemd.

[east2west]

I understand your strong opinion and since I am no sysadmin I have no technical problem with arguments. But I am not sure I totally agree with your characterization of slow pace of change by Apple or the wonderful state of Unix/Linux. Aqua was quite a break from the previous GUI and Apple changed the whole stack at one point from computer architecture to OS to graphic library. I don't know a more radical change than that for a software company. As for Linux graphic environment, I can only say that replacing X-win with Wayland is not evolutionary and it cannot come soon enough. Anyway, hopefully things will quiet down for a while and we can compare and contrast alternatives in the real world.

[dredmorbius]

Also, to be clear, I'm not accusing Apple of failing to innovate elsewhere in its product chain. It clearly has. Since 1999: the iBook, MacBook Pro, Air, and a few iterations of the iMac, just in form factors. There's been a lot of under-the-hood stuff going on as well.

But where the user interacts with the system, things have been remarkably stable. Even the relatively minor changes which have been presented have been covered with the usual Apple levels of obsession -- skewmorphic vs. flat designs, etc., ad nauseum.

Again the point being: screw with how things are visually and how users interact with the system, you're going to create huge usability costs with little to show for.

[dredmorbius]

I'm not saying that the System 9 (I think -- I'm not fully up on my MacOS nomenclature) to Aqua break wasn't big. It was.

BUT IT WAS THE FIRST SUCH BREAK IN 15 YEARS OF THE GUI, AND IT'S BEEN THE ONLY MAJOR BREAK IN THE PAST 15 YEARS.

I'm also not saying that Aqua hasn't changed at all. It has, with the most notable addition that I'm aware of being virtual desktops (something NextSTEP had in the 1980s). But other than some minor cosmetic changes, and largely invisible-to-the-user under-the-hood updates, the visible UI has NOT changed appreciably.

Contrast that with the disruption that's prevailed in the Microsoft Windows and Linux spaces from 1999 to present. We've gone from the Win98 UI to the candy-cane XP styling, and Metro in Windows, and at least three generations each of KDE and GNOME on Linux, plus a few other desktops which have waxed and waned in popularity.

I've continued to use WindowMaker, and after 17 years, it is, hands down, the one GUI metaphor I've had the longest experience with of any. It's been exceptionally stable, with very few changes. Even minor ones are quite jarring to me, which is somewhat odd to reflect on.

X11 and/or replacements is a whole 'nother discussion, but I'll simply note that the network transparency of X has been hugely underappreciated by many who've sought to upend it (I don't know what the status of Wayland is in this regard).

[jude-]

> If they feel like learning systemd is a chore, they might be in the wrong business.

I think IT managers would prefer it if they didn't have to spend time and money re-training their sysadmins or hiring/firing them to ensure their staff has the skills to use the $NEW_SHINY from $CORPORATE_VENDOR. Skill transference is a boon for customers (see also: "Stop breaking the UI!").