[dextercd]

You can use systemd-run with --shell (or a subset of options enabled by --shell) and -p to specify service properties to run commands interactively in a similar environment as your service.

This can help troubleshoot issues and makes experimenting with systemd options faster.

I think there's been some talk about adding a built-in way for systemd-run to copy settings out of a .service file, but it doesn't exist yet.

I've written Perl/Python scripts to do this for me. They're not really aimed at working with arbitrary services, but it should be possible to adapt to different scenarios.

https://gist.github.com/dextercd/59a7e5e25b125d3506c78caa3dd...

There are some gotchas I ran into. For example, with RuntimeDirectory: systemd deletes the directory once the process exits, even if there's still another process running with the same RuntimeDirectory value set.

[matheusmoreira]

I use systemd-run very often to impose CPU usage limits on software. Awesome feature.

[mpyne]

It's also really useful for doing parallel builds of modules that may actually consume all available memory when you can't force the build system to use fewer cores than you have available.

Both in terms of artificially reducing the number of CPUs you expose, but also in terms of enforcing a memory limit that will kill all processes in the build before the broader kernel OOM killer will act, in case you screw up the number of CPUs.

[Imustaskforhelp]

woah that's actually awesome. I feel like adding uh storage usage limits could also be easy as well.

But the one thing that I always wonder is about (virtualization?) in the sense of something like docker just for containerizing or some sort of way of running them in some sort of sandbox without much performance issues or something, I am kinda interested in knowing what might be the best way of doing so (is podman the right way or some other way like bubblewrap?)

Edit: just discovered in the comment below the (parents parents?)comment that there is systemd isolation too, that sounds very interesting and the first time I personally heard of it hmm

[ChocolateGod]

You can achieve similar results with podman and bubblewrap, but podman handles things like networking, resource and image management that bubblewrap doesn't by itself

Bubblewrap really is more for sandboxing "transient" containers and being able to separate specific things from the host (such as libraries), with other applications handling the image management, which makes sense because its primary user is Flatpak and Steam. Once the application inside the container is exited, the sandbox is destroyed, it's job is done.

Podman is a Docker clone, it's for development or persistent containers. It will monitor containers, restart them, can pull image updates, setup networks between them etc.

They both use namespacing and cgroups under the hood, but for different results and purposes.

Your right that systemd has sandboxing too, and it also uses the same features as the kernel. Podman can also export it's services to be managed by systemd.

There's literally so much choice when it comes to making containers on Linux.

[godelski]

  > but podman handles things like networking, resource and image management

Btw, you can do all of this with systemd too

  > the sandbox is destroyed, it's job is done.

I think most container systems have an ephemeral option. If you're looking at systemd then look at the man pages for either systemd-nspawn or systemd-vmspawn and look under Image Options. More specifically `-x, --ephemeral`. It's a pretty handy option.

  > Podman can also export it's services to be managed by systemd.

But in that case, why not just use systemd? ;)

  > There's literally so much choice when it comes to making containers on Linux.

Despite my joke above, I actually love this. Having options is great and I think it ends up pushing all of them to be better. The competition is great. I'm hyping systemd up a bit but honestly there's gives and takes with each of the different methods. There's healthy competition right now, but I do think systemd deserves a bit more love than it currently gets.

[ChocolateGod]

> But in that case, why not just use systemd? ;)

Because some stuff is shipped and supported as a docker image and it's just easier to run it via podman which supports nearly all the same options.

[godelski]

Yeah, that is a big drawback. But as mentioned elsewhere and by others, there is `importctl`. So you can ship these images as well. Meaning only one person needs to make that image for others to be able to get the same convenience as pulling a docker image.

I'm unsure if someone has made a tool to convert docker images to systemd. If not, that'd be a pretty handy one.

[Imustaskforhelp]

podman + systemd integration seems really nice now.

given that podman can also have a (nicer?) transition to docker is a plus as well.

There are a lot of paas nowadays which use docker under the hood. I think I would love seeing a future where a paas actually manages it using systemd.

I think this might be really nice giving an almost standard way of installing software.

I really want to try to create something like dokku or some gui for making systemd management easier but I will see some alternatives currently, thanks for sharing it!

[yjftsjthsd-h]

I'm fairly confident that systemd, docker, podman, bubblewrap, unshare, and probably other tools are all wrapping the same kernel features, so I'd expect a certain degree of convergence in what they provide.

[godelski]

I wrote my comment before I saw yours, but you'll probably be interested in it[0].

The best thing about systemd is also the worst thing: it's monolithic. You can containarize applications lightly all the way to having a full fledged VM. You can run as user or root. You can limit system access like CPU, RAM, network, and even the physical hardware. You even have homed which gives you more control over your user environments. There's systemd mounts[1], boot, machines, timers, networks, and more. It's overwhelming.

I think two commands everyone should know if dealing with systemd services is:

  - `systemctl edit foo.service` to create an override file which sits on top of the existing service file (so your changes don't disappear when you upgrade)
  - `systemd-analyze security foo.service` which will give you a short description of the security options and a score specifying your exposure level.

These really helped me go down the rabbit hole and I think most people should have some basic idea of how to restrict their services. A little goes a long way, so even if you're just adding `PrivateTmp=yes` to a service, you're improving it.

I've replaced all my cron jobs with systemd jobs now and while it is a bit more work up front (just copy paste templates...) there are huge benefits to be had. Way more flexibility in scheduling and you're not limited to such restrictions as your computer being off[3]

[0] https://news.ycombinator.com/item?id=45318649

[1] I've found mounts really helpful and can really speed up boot times. You can make your drives mount in the background and after any service you want. You can also set timeouts so that they will power down and automount as needed. That can save you a good amount on electricity if you got a storage service. This might also be a good time to remind people that you likely want to add `noatime` to your mount options (even if you use fstab)[2].

[2] https://opensource.com/article/20/6/linux-noatime

[3] You can have it run the service on the next boot (or whenever) if it was supposed to run when the machine was powered off.

[nullpoint420]

Kubernetes is also this but for the cloud. CPU/Mem/Storage limits

[dextercd]

Here's the Python version I've been using: https://gist.github.com/dextercd/3bd65c1e32635b9e7bebf287b52...

Another issue I just ran into is that a colon separated value for ExecSearchPath doesn't work in systemd-run/-p. You have to specify each path as a separate -p argument.

There are some minor annoyances like that, but it's not too hard to work around.