[pojntfx]

bootc and OSTree are both very neat, but the leading edge of immutable Linux distros (GNOME OS, KDE Linux) is currently converging on a different proposal by systemd developers that's standardized by the UAPI Group (https://uapi-group.org/specifications/). It fixes quite a few of the complexities with OSTree (updates are handled by `systemd-sysupdate`/`updatectl` and are just files served via HTTP) and is quite a bit easier to extend with things like an immutable version of the Nvidia drivers or codecs thanks to system extensions handled by `systemd-sysext` (which in turn are just simple squashfs files overlayed over `/usr`) and configuration via `systemd-confext`. `mkosi`, also by systemd, is quickly becoming _the_ way to build custom images too, and is somewhat tied to these new standards.

[jcastro]

> is currently converging on a different proposal by systemd developers that's standardized by the UAPI Group

We're working in this space with Project Bluefin: https://github.com/projectbluefin/dakota

Both approaches are indeed competitive, but you can also leverage both to achieve the same thing. We're experimenting with a pure ddi Bluefin, a buildstream/GNOMEOS one that spits out a bootc image, as well as a Bluefin that is just a systemd-sysext on top of GNOME OS. Chef's choice!

There will be many ways to slice this problem -- my opinion is that in the end it will be how you design the infrastructure to make these and not the artifacts themselves.

We already have CentOS/Fedora builds alongside these, long term we'll see which ones end up being the most efficient. Buildstream is a tool which people should look at in this space too: https://buildstream.build/index.html

[znpy]

From https://uapi-group.org/ :

> Contributing members include people from Ubuntu Core, Debian, GNOME OS, Fedora CoreOS, Endless OS, Arch Linux, SUSE, Flatcar, systemd, image-builder/osbuild, mkosi, tpm2-software, System Transparency, buildstream, BTRFS, bootc, composefs, (rpm-)ostree, Microsoft, Amazon, and Meta.

Note systemd, (rpm-)ostree and bootc.

My understanding is that uapi is another initiative but not completely separated from bootc and ostree. Maybe complementary.

[pojntfx]

Sorry, not completely separate, yes, but some of the parts of the standard (e.g. systemd-sysext for layering "packages") and closely related things like systemd-sysupdate do actually replace parts of this (esp. ostree).

[aaravchen]

sysexts are indeed a very interesting feature, though it really only complements some other whole-system solution since it can only affect files under a non-root folder location.

I'm struggling to see how sysupdate is really equivalent to bootc or ostree though. Sysupdate is just the sw-update tool from Yocto that's been around for 10+ years with a little more syntax sugar, which itself was just a common shared implementation of what all embedded systems had been rolling-thier-own of for almost 20 years before that. It says it requires an A/B/.../N partitioning scheme, which is exactly what bootc/ostree/etc is designing to avoid.

If you don't use the whole disks update thing from sysupdate, then instead you're just talking about a transactional package manager that is still in its infancy and hasn't addressed the many gotcha and corner cases that the dozens of more mature package managers have. It's not actually "transactional" in the sense of undo for example, it's "transactional" only in that you won't get partial install, which hasn't been a problem with any existing package managers for almost 40 years. All thier listed things you can list together for a "transaction" association are either things that are already linked via existing package maager packages, or are only useful for embedded systems.

I'm not saying sysupdate isn't useful, upper end of embedded design is pushing into the space where systemd is standard now so it could be useful for those devices, but it's not really equivalent at all to bootc/ostree, and doesn't really have amt applicability outside initial system imaging from a live disk, or embedded devices.

[aaravchen]

Ironically the first implementation of ostree required an HTTPS server to serve the ostree commits, allowing a much smaller subset of what's needed to be transferred. However that became an adoption hurdle since it required unique infrastructure. Ostree switched to using containers because zstd allows compressed chunks now rather than the old all-or-nothing image layers, and the existing widespread container image registry infrastructure could be reused without modification. And both utilize layers for their construction so there were possible benefits there (that never really materialized, but are still available to pursue).

[bigpod]

yea it switched to a in general much better distribution method then custom https server, for one thing authentication is built in something many of these https solutions forget(look at every linux package manager)

[rurban]

Typo: (CoreOS and Fedora Silverblue) are the bleeding edge of immutable distros. Those mentioned are just users.

[pojntfx]

GNOME OS uses BuildStream and as a result has no concept of packages at all and no relationship to any distro, KDE Linux is based on Arch. There is no relationship between the two. GNOME OS used to be OSTree based but switched to systemd-sysupdate a while ago.

[aaravchen]

CoreOS is in a weird space. It's been desperately playing catch up with it's sibling products for the last few years, but it also is where a lot of the Fedora/RHEL developers in this space are focused primarily.

[debugnik]

It's confusing, but there're apparently distros called "GNOME OS" and "KDE Linux" now.

[smashed]

> the bleeding edge of immutable Linux distros (GNOME OS, KDE Linux)

These are words but they don't make sense.

[pojntfx]

Corrected - I meant leading edge.

Context re:distros mentioned:

GNOME OS: https://os.gnome.org/ KDE Linux: https://kde.org/linux/

[7777777phil]

Doubling progression-free survival (17.6 vs 7.4 months) is a large effect size for recurrent prostate cancer.

[Scipio_Afri]

I’m pretty sure Linux doesn’t have a prostate, even with all the changes in the leading edge distros, and you’re commenting in the wrong post.

[7777777phil]

Pretty sure I didn’t want to post that here. But then I got rate limited and upon coming out of rate limit jail blindly pasted this comment where my page reloaded - my bad should have been here: https://news.ycombinator.com/item?id=47193047

[coderedart]

Kdelinux uses pacman for now, but the eventual goal is systemd-sysext based mkosi images.

They are also considering moving to buildstream and join gnome.

[n42]

"some of the newer ideas happening in this space are in the GNOME OS project and the KDE Linux project"

[hparadiz]

My Gentoo box is immutable. Right up until I run emerge.

[bigpod]

system extensions are just as usable on bootc based systems but bootc based systems allow for much simpler composition, you dont have to know about all the intricacies of linux filesystem tree to install a set of packges for your own image as you can base uppon what someone else did easily using industry standard semantics and use actual package managers. And on top of all that it uses a far better distribution system, the oci's distribution spec

[anglesideangle]

I often see bootc and/or buildstream uncritically presented as the future of the linux desktop, and find it somewhat surprising because, in my experience, they are both more complex _and_ less capable than nixos.

To elaborate, I will separate linux operating systems into three categories:

1. traditional mutable package management (debian, fedora, arch, etc) - Bad model for obvious reasons, I won't get into this.

2. immutable images (embedded deployments, android, chromeos) - A build system (yocto, nix, buildroot, buildstream) procudes a disk image. System boots the disk image and handles updates using A/B root partitions with `systemd-sysupdate` or similar.

3. store-based atomic (nixos, ublue) - The system keeps a mutable store of hashed objects (/nix/store, /var/lib/containers/storage, /ostree/repo) and boots a specific system generation. Updates add new objects to the store, which must be automatically cleaned, and create a new systemd generation to boot into.

In the case of categories 2 and 3, a build system of some kind is used to produce the image or packages that are turned into the new system generation.

The bootc project, which falls into category 3, attempts to use the standardized and highly adopted OCI image format (layered filesystem changes stored in a content addressed store) as a medium for distributing linux systems. The major limitation here is that these systems are very complicated to build and extend. While it may not seem that way compared to nixos if you have prior experience writing dockerfiles, configuring your system with imperative statements that build on previous state is _really_ tedious. For example, you can't just re-use work with multiple `FROM` statements in the same layer, so you instead need to copy files between images. This is incredibly jank, look at [the docs](https://blue-build.org/how-to/minimal-setup/) for bluebuild's module system. Additionally, for a motivated user to change their system internals, they need to make the jump to hosting it with CI and pulling the images.

As jcastro mentioned elsewhere in this thread, there is work in the ublue project, which focuses on bootc images, to instead build their systems from source using buildstream, the same way GNOME OS works. The idea is that this is no longer a "distribution" and doesn't have "packages" anymore, since the entire system is built from source into an image that can be updated to atomically. While this model is simpler and way less jank than assembling your OS in a dockerfile, the model of separating packaging from distribution makes things harder for users for no benefit over the alternative.

To elaborate: Buildstream is a build system created by gnome developers that works as follows:

- Various element are defined in yaml files with their dependency and build steps

- Buildstream forms a tree of build elements to evaluate

- Each element is evaluated in a sandbox with access to only its dependencies and the output is placed in a content-addressed store for caching

If you have experience with nix, you may recognize that this is almost the same way nix works, with the difference that buildstream dependencies are mounted to regular directories (e.g. /usr) in the build sandbox instead of directly dealt with as /nix/store objects, and that buildstream is much harder to extend. A nix build results in artifacts that live in the nix store and are symlinked to other paths in the nix store, while a buildstream build results in artifacts that are compatible with a more traditional directory structure.

The idea of GNOME OS being "distroless" and not having packages is misleading, as it does have packages. Only, the packages exist exclusively during the build process. In order for a user to modify their system, they must add a rule to the buildstream definitions their system is built from and rebuild the entire thing from source to generate a new system image, which is unnecessarily burdensome. This is because the content-addressed artifact store (buildstream's cache) that exists when building has no relationship to the content-addressed artifact store on the deployed system (ostree or oci storage). This is pointless indirection with no clear benefit. By not separating building from distribution, nixos (a project using the nix dsl to build a linux system) achieves the same benefits of this model without any of the drawbacks. Users can modify how their system works, use caches, add their own packages, and share/integrate these modifications freely without building their entire os again from source every time.

To put this power in perspective, adding a nginx server to your system on nixos amounts to adding to your nix configuration:

services.nginx.enable = true;

and then rebuilding, switching to the new system generation.

Imagine how painful it would be to do this via a dockerfile (based off a different image) or buildstream definitions...

Finally, I'd like to clarify that nixos is not perfect. There are many areas of that need improvement (documentation, evaluation speed, evaluation caching, ifd, error messages, etc). However, I believe nixos is a fundamentally better and simpler model than the one being pushed by a lot of the linux desktop ecosystem at the moment. I believe a lot of the work on infrastructure like bootc and buildstream would be better focused on nix/nixos, or at least would benefit from learning from them.

[bigpod]

nixos may have a simpler moder on paper the implementation of it is so backwards that it ruins everything, and once you implement the model you see its not simpler and your conclussion is wrong nix and nixos has a lot to learn from bootc, like usage of standard industry tooling instead of inventing its own things for what to be honest is no benefit mostly drawbacks which are so huge. BootC uses primarily conventions already existing previously its just different way to deliver and define, better in almost every way.

first off i can start nginx locally i dont need to change a fancy config in a bad DSL(and yes its bad actually horrible),but if i want it in an image i add one line in my docker file RUN systemctl enable nginx and rebuild thats it does it take longer to rebuild WHO CARES CI takes care of it

[anglesideangle]

> nix and nixos has a lot to learn from bootc

nix is a build system, and has nothing to learn from bootc. However, I agree that nixos could adopt some of the developments around bootc, like composefs for verified boot.

> usage of standard industry tooling instead of inventing its own things for what to be honest is no benefit mostly drawbacks which are so huge

nix was first released in 2003, nixos and docker were first released in 2013, and the OCI was started in 2015. bootc was created ~2023. For the purposes of deploying environments, an input-addressed (hopefully content-addressed in the future) tree of build artifacts is just a conceptually _better_ model than coarse layers of filesystem changes. It's a shame it isn't industry standard, because the industry would be better off if it was.

> BootC uses primarily conventions already existing previously

As previously established, nix and nixos were created before docker, and all of them existed before bootc, even though you might not have personally encountered nixos before then. There's a lot of value in keeping an open mind to technologies you might not be familiar or comfortable with, even though it might be tempting to arbitrarily hate them because they are different from what is commonly used.

> better in almost every way

refer to my previous post

> first off i can start nginx locally

yes, but you have to install and configure it. You cannot add nginx as a system package on a bootc system without building a new layer. Configuring it can be done by manually editing /etc/nginx and /var/www on _both_ nixos and bootc distros, but configuring it in a manner reproducible across machines requires either writing all your config in the dockerfile or something like ansible on a bootc system.

> bad DSL(and yes its bad actually horrible)

the nix language has plenty of areas for improvement, but it really is not that bad. I personally prefer it to a combination of yaml and python glue that alternatives like buildstream tend to use.

> WHO CARES CI takes care of it

good luck asking any user who wants to change something about their system to choose between setting up CI, essentially being responsible for a custom distribution, or waiting for a docker build on every update because their bottom layer's cache gets invalidated.

[lproven]

> bootc and OSTree are both very neat

May I rephrase that?

bootc and OStree are both Cthulhoid nightmare horrors that only exist because of corporate politics, but the leading edge...

[doug-moen]

GNOME OS and KDE Linux are both specialized distros that primarily exist to test GNOME and KDE. They aren't for general users, and both web sites warn you not to rely on them. And they impose limitations on your ability to install arbitrary 3rd party software, whereas Fedora Atomic Desktop lets you customize the system without such limitations. Fedora Atomic lets me install arbitrary RPMs into the base system.

"quite a bit easier to extend" sounds good to me, but the "easier" here refers to the internal system implementation details? I am an end user, not a Linux distro system architect, and I care more about the user experience. I will be interested in test driving a general purpose OS based on this technology, whenever that happens in the future. Since Red Hat is involved in the UAPI project, perhaps Fedora Atomic Desktop will migrate to this technology in the future?