Hacker News new | ask | show | jobs
by barosl 970 days ago
The cool thing about this project is that as it uses systemd's socket activation, it requires no server processes at all. There is no waste of resources when Cockpit is not being used. Accessing a page is literally the same as invoking a command-line tool (and quitting it). No more, no less. What a beautiful design.
9 comments

To be fair, we've had this since BSD4.3 (1986) through inetd - which worked slightly differently, but same overall idea. Once popular, it fell out of fashion because... Well, there isn't really any reason for it.

A good server process is idle when nothing is happening, and should be using miniscule real memory that should be easy to swap out. If the server in question uses significant memory for your use-case, you also don't want it starting on demand and triggering sporadic memory pressure.

It does make it easier to avoid blocking on service start in early boot though, which is a common cause of poor boot performance.

There's good reasons for it though!

One is boot performance. Another is zero cost for a rarely used tool, which may be particularly important on a VPS or a small computer like a Raspberry Pi where you don't want to add costs for something that may only rarely be needed.

I think a nice benefit for an administrative tool is the ability to update it, and reload the updated version. You don't need the tool to have its own "re-exec myself" code that's rarely used, and that could fail at an inconvenient time.

The reason why inetd didn't stick is because it's a pain to use -- it's separated from SysV init, so it needs to be very intentionally set up. Plus there was the inetd/xinetd disagreement.

Tying in init, inetd and monit into a single system that can do all those things IMO made things much nicer.

> Another is zero cost for a rarely used tool.

Zero cost is only true for unused services. For rarely used services, it's a rarely occuring full cost that might come by surprise at a bad time.

> I think a nice benefit for an administrative tool is the ability to update it, and reload the updated version.

This is only a benefit if the systemd socket unit is co figured to operate in inetd-mode (Accept=yes), where systemd spawns a new process for every accepted connection, which is quite inefficient resource-wise.

"Normal" systemd socket activation just starts the service and hands over the socket. The service runs indefinitely afterwards as if it was a normal service, and needs to be manually restarted or hot-reloaded after upgrade or configuration change.

> The reason why inetd didn't stick is because it's a pain to use -- it's separated from SysV init, so it needs to be very intentionally set up.

Being separated has a lot of benefits - easy nesting, easy reuse in minimal containers, etc. The integrated model works best for monolithic servers.

Around the time I was first learning Linux, I recall reading that there were two ways to run a service:

1. Start the daemon on boot and have it running all the time, like some undereducated neanderthal.

2. Configure your system to run a daemon which monitors a port/socket and starts up only when there is traffic, like a civilized person.

I believe which one of these to use is highly dependent on your resources, usage, and deployment model. For services that are fast and cheap to start but are rarely used, #1 makes more sense. If you have a server or VM which only does one thing (very much the norm, these days), then running just keeping that service running all the time is easier and better for performance.

Actually I think what killed inetd is, partially, http. At the time, http was connectionless. Open socket, send packet, read response, close. Out of the box inetd would support that, for sure, but it would be constantly forking new http processes to do it.

FTP, SMTP were all stateful, so living under inetd worked OK. One process per overall session rather than individual messages within a session.

Obviously, inetd could have been hammered on to basically consume the pre-forking model then dominant in something like Apache, caching server processes, etc.

But it wasn't. Then databases became the other dominant server process, and they didn't run behind inetd either.

Apache + CGI was the "inetd" of the web age.

I ended up reading more about this and looks like SSHD in Ubuntu 22.10 and later also uses systemd socket activation. So there should be no sshd process(es) started until someone SSHs in!

https://discourse.ubuntu.com/t/sshd-now-uses-socket-based-ac...

This is messed up, totally messed up:

"On upgrades from Ubuntu 22.04 LTS, users who had configured Port settings or a ListenAddress setting in /etc/ssh/sshd_config will find these settings migrated to /etc/systemd/system/ssh.socket.d/addresses.conf."

It's like Canonical is doing 1960's quality acid.

At least the garbage can be disabled:

"it is still possible to revert to the previous non-socket-activated behavior"

With having to remove snapd then mark it to not be installed and in the next Ubuntu having to fix ssh back to the current behavior, it might be easier to migrate my servers back to Debian, or look for a solid non-systemd OS.

What exactly is "garbage" about this? It's so tiring how systemd opponents insist on name-calling instead of substantiated criticism.

There is no reason every single application should manage network socket acquisition on its own - I'm not very fond of the times everyone and their mother wrote whacky shell scripts to start and stop their services, either. But somehow those seem to be the "good old times" you guys miss.

I don't think its a systemD thing. This sounds more like an issue with changing a server's behaviour without asking.
Distribution upgrades have never been an unobtrusive thing. Despite this, everything will continue working exactly as configured before the upgrade, which applies new configuration recommendations by the vendor. What is wrong with that?
What's wrong is the config file moved. If a sysadmin is used to a config file being somewhere they know, and then it disappears that can be extremely frustrating. Especially on a production system.
Certainly for SSH I find this a bad idea. If you need to ssh into a troubled machine then it might very well be it cannot be started.
I don't necessarily think it's an outright bad idea, but it's certainly a departure from how sshd is traditionally run, and without awareness of this kind of change, this kind of "magic" runtime change could lead you to not expecting sshd to be unavailable in this kind of a scenario, and increase time to resolution during an incident.

If your systems are more pets than cattle, then I think I too would prefer an always-running ssh daemon. If your workflow is only to ssh into machines during bootstrap, however, then having sshd run only during initial bootstrap and then shut itself off does seem like a nice way to free up a small amount of resources without stopping or disabling the daemon post-bootstrap.

If it's so troubled that a process won't start, it's probably time to reach for the IPMI console. Even if ssh is still running, if the system is that broken, is bash going to start, or what tools you might need?
I've rescued pretty messed up systems before. In one case, a service went haywire and created tens of millions of <1kb files, eating up all the available inodes in the filesystem. The volume was only ~45% full, but you couldn't make any new files. If that happened here, it's unlikely the ssh process could start correctly since it creates locks, logs, and pid files. Those are mostly in /run, so it might be okay, but it does make me a bit antsy.
TBH - for any non-server class machine on my network, I'm fine with that.

SSH should probably be running 24/7 on any server(to keep those resources allocated for maintenance access), but if it's my workstation with a monitor - then it's a non-issue.

Ew
I should really spend more time learning systemd. The more I look into it, the more cool and useful features I discover.
If you have anything at all to do with OS administration, management, or software packaging, it's worth it.

If I could offer a little advice: The systemd man pages are useful as a reference, but are terrible to learn from. Part of this is because there are parts of systemd that everyone uses, and there are parts that almost nobody uses and it's hard to guess which these are at first. Also, the man pages are dry and long and quite often fail to describe things in a way that would make any sense whatsoever to someone who isn't already intimately familiar with systemd.

Most of my systemd learning came from random blog articles and of course the excellent Arch wiki.

Also, it's 99% "not different than doing it via command line", and also comes with a little js terminal gui, uses native users + passwords, has some lightweight monitoring history, lets you browse a bunch of configuration that you usually would have to remember byzantine systemd command lines for... it's awesome for what it is!

I'm happy to run it (aka: have it installed) on all my little raspberry pi's, because sometimes I'm not at a terminal when I want to scope them out, and/or if I'm at "just a web browser", being able to "natively ssh into them" via a web server (and then run `curl ...etc...` from a "real" command prompt) is super helpful!

Just want to clarify: there's still a server process running to serve the Cockpit web app's static HTML/JS assets, right?

Do you essentially mean that systemd socket activation is used basically only if/when the Cockpit web app end-user/client sends a REST/GQL/etc/? request for logs, for example?

I thought the cool thing was all the rookies who install this thing in a way that it's publicly accessible. How many stories have I heard about people who accidentally configure phpMyAdmin to be publicly accessible... Now you might not JUST leak your whole customer DB!
Interesting, I always thought socket activation meant defer launching a process until somebody tries to access it through the network, but... does it also finish the web server process (or whatever is used here) as well after the request is serviced?
No, it doesn't automatically close the process. Two options I can think of: Application exit when it's done with its thing or RuntimeMaxSec to make it close after a while.

systemd passes the socket on to the application so I don't think it has any reference to it anymore, so it wouldn't be able to know when the socket closes.

systemd-cgi :^)
Everything old is new again.

The next big thing will be a web server where you don't need to use the command line to deploy your project, just sync your workspace folder and it will automatically execute the file matching the URL.

It was/is inetd[1] actually

[1] https://en.wikipedia.org/wiki/Inetd

Socket activation means that every application must be modified so that it can run both with activation or without. So you need to patch every application for compatibility with systemd. And if tomorrow there will be an alternative system daemon, you will have to patch everything again?