[ar-nelson]

I really wanted to like FreeBSD. ZFS is cool, jails are cool, pkg has such a wide variety of packages that it's comparable to Arch.

In a world before Docker and btrfs, this would be enough to make installing FreeBSD worthwhile. I tried it anyway, for a home server, and it was mostly painful.

Jails are tricky and require third-party (?) tools and scripts to manage them effectively. And they don't give you much that Docker doesn't already. And, although it's not as bad as Arch, FreeBSD does update quickly and if you aren't frequently updating your server you can be left behind; it's not noticeable until you try to update and a bunch of packages break, or your jails are no longer running the same major version as their host and that causes subtle bugs.

I recently replaced my FreeBSD home server with a basic Debian machine running a docker-compose cluster, and it's so much simpler. I wanted to like FreeBSD. At its core, it's probably a better-designed system. But it requires too much additional knowledge and maintenance for something that Linux and Docker do just as well.

[thijsvandien]

For me it's the opposite really. Every time I touch Linux, I'm overwhelmed by how messy it is. I spent hours upgrading an old Ubuntu box serving as a Ubiquity UniFi Controller and ended up in some kind of dependency hell. While trying to fix all that, disk space ran out. Then I had enough, whiped the whole thing and in 5 minutes installed the FreeBSD base system. A handful of binary packages and a couple of edits to config files later everything worked perfectly. It all feels so clean, with a minimal number of processes running.

[yetanotherloss]

My experience dealing with Linux over the years can be summed up by the exclamation "oh my fucking god can you people just PICK ONE."

There are certainly arguments for why the variety of replacements for simple, old, well trod and tested system tools are improvement and yet my 20 year BSD installations get on mostly fine without them and I'd be hard pressed to describe what they're missing in practical terms other than very specific stacks like Docker.

If you live in devops I can see that being a big deal but for the service functions themselves there is something to be said for leaving minor things at "good enough."

[ransackdev]

Running out of disk space isn’t the operating system’s fault

> Then I had enough, whiped the whole thing and in 5 minutes installed the FreeBSD base system. A handful of binary packages and a couple of edits to config files later everything worked perfectly.

There seems to be some confirmation bias here. Is writing an iso, installing an OS, installing a “handful” of packages, and configuring everything not somewhat equivalent to the “messy” alternative of upgrading Ubuntu? I bet they were very close to the same amount of work, but one might have seemed like more, or more daunting, due to your experience and preference for bsd. Just throwing it out there

As for Ubuntu upgrades, apt-get update, apt-get dist-upgrade, and do-release-upgrade have always worked well for me, though I hardly ever do a major version upgrade and opt to do a fresh install after the first minor point release. If you were jumping major versions or a few major versions, of course packages are going to give you a hard time. I imagine a fresh install might have gone as seemingly smooth as the bsd route you took. Anyway, hacking is hacking, keep at it!

[thijsvandien]

You know, 10 GB should be plenty for what that machine is supposed to be doing. On Ubuntu, initially it was, but after several upgrades (despite cleaning package caches), not any longer. FreeBSD takes less than a fifth, and I know that isn't going to change much. Am I desperate to save a few gigs? No, it merely illustrates how much of everything there was that I didn't want there to be. Now the output of top fits in my small console window. It's about feeling in control.

Surely experience must be factored in, but I consider it a fact that Linux is less coherent and more prone to changes. Evolving quicker can be a benefit, yet for my server applications it simply never is.

[toast0]

> FreeBSD does update quickly and if you aren't frequently updating your server you can be left behind; it's not noticeable until you try to update and a bunch of packages break, or your jails are no longer running the same major version as their host and that causes subtle bugs.

In modern times, point releases tend to be about once a year, which doesn't seem too frequent to me? But if you use pkg, and the new point release added a syscall, you will have problems installing new packages (or upgrading) once the package builders update; I don't quite remember the timeline, but it's usually a few months after the point release. (Although, I think there was a time where pkg would default to a faster updating repository, and you might see problems sooner)

My upgrade process is a bit laborious (freebsd-update -r ... upgrade; freebsd-update install; shutdown -r now; freebsd-upgrade install; pkg update; pkg upgrade; reboot). Might be another pkg command I forgot, but it will likely prompt you.

If you're not interested in upgrading everything together at once, ports is probably a better choice. Sometimes after an OS upgrade, you'll need to install a compat port to continue to run old software; it's possible to do that with pkg, but IMHO, it's difficult to get the compat package installed without upgrading other packages, and then you may as well upgrade everything.

Personally, I don't find docker-compose simple. Probably from lack of experience, but I find it hides so much that makes it much more difficult to debug. Most of the time, I'd rather run everything in a single host, but I'm encouraged to use docker-compose by my workplace, so...

[kevans91]

> [...] But if you use pkg, and the new point release added a syscall, you will have problems installing new packages (or upgrading) once the package builders update; I don't quite remember the timeline, but it's usually a few months after the point release. [...]

The previous release in the branch gets dropped after ~3 months, probably aligned to the end of the month that its EOL lands in; then pkg builders switch to the newer release in the branch.

[hn8305823]

> shutdown -r now

I know this is in lots of textbook examples but I didn't think anyone ever actually typed this. I've always used 'halt' or 'reboot'

[rjsw]

Using 'halt' or 'reboot' won't run all the scripts needed to shut down cleanly, the correct command to use is 'shutdown -r now'.

[red-iron-pine]

IIRC, 'reboot' is aliased to 'shutdown -r now' in several modern distros

[toast0]

reboot and halt have a history of being less graceful on some systems, and I've been bitten. For FreeBSD, I think reboot skips shutdown scripts, doesn't notify all terminals, etc; just kill everything, sync the disk and reboot.

That's not ungraceful, but it's not exactly pleasant.

OTOH, my openwrt device doesn't have a shutdown binary, just a reboot binary.

[JohnFen]

I use "shutdown -r now" instead of "reboot" because "halt" or "reboot" is not universally available, but shutdown mostly is.

[znpy]

> In a world before Docker and btrfs, this would be enough to make installing FreeBSD worthwhile

FreeBSD is really missing out in not implementing something to run containers, and it shouldn’t even be impossible.

The podman people basically reimplemented the docker daemon command line (and apis), and I remember a talk by Bryan Cantrill about how joyent reimplemented the docker API to spawn Illumos/SmartOS zones backed by zfs volumes (i guess either datasets or zvols). And there was some person reimplementing the docker apis using jails as a backend… but I guess that project never went anywhere and didn’t get the attention (and funding/developer time?) of the freebsd foundation.

[stonogo]

I find your assessment odd, since Docker itself is basically a third-party tool to manage cgroups, namespaces, and filesystems. I suppose it's another in a long line of what are essentially marketing failures on the part of FreeBSD...

[ar-nelson]

FreeBSD jails don't have a one-command way to install a preconfigured jail for a specific service. And they don't have a way (that I know of) to write a config file describing a bunch of jails and services, then run that file and install all of those services on a new machine.

Docker is less elegant than jails in a lot of ways, but not having to figure out how to install each service on your own is a huge advantage.

I tried using Ansible to set all of my services up, and it really didn't work. Probably because I was using a different machine as the Ansible controller, and there is no Ansible plugin to connect to jails on a remote machine. I tried writing my own, and it just barely worked, but with a lot of bugs I could never fully track down (because Ansible's internals aren't documented at all). Eventually it just wasn't worth it to keep trying to use FreeBSD.

[craftkiller]

> FreeBSD jails don't have a one-command way to install a preconfigured jail for a specific service

FreeBSD does have that tool, its BastilleBSD: https://bastillebsd.org/

For example, this is the Bastillefile for running consul: https://gitlab.com/bastillebsd-templates/consul/-/blob/maste...

> And they don't have a way (that I know of) to write a config file describing a bunch of jails and services, then run that file and install all of those services on a new machine

Sounds like you want to use pot with nomad: https://klarasystems.com/articles/cluster-provisioning-with-...

[tcmart14]

To add on, CBSD may also provide of these functions. So there are options.

[1] https://github.com/cbsd/cbsd

[stonogo]

It has both, but like Docker, they don't ship in base. ezjail was the tool of choice for many years, and cbsd (which also manages the virtualization capabilities) has emerged as a mature replacement for the past decade or so.

I'm not disagreeing with your assessment -- nobody knows better than you which software you want to use! -- so much as listing options for anyone reading.

But I find it interesting how far ahead of its time FreeBSD was on this, and how badly it appears to have fumbled this ball. The official documentation still recommends ezjail, which is not good. Just another example, I guess, of "writing the code isn't enough."

[Gud]

What’s wrong with ezjail? I still use it, works great

[znpy]

Honest question: what do you mean with “docker-compose cluster” ?

[all2]

Docker compose allows you to specify multiple docker containers all running on a single host. It makes setting up multiple services on a single piece of hardware relatively simple and repeatable.

[gjvc]

Would also like to know what this is.

[kjs3]

Jails are tricky and require third-party (?) tools and scripts to manage them effectively.

Docker is a third party tool. People use third-party tools and scripts to manage them effectively.

Having done jails for years and now having to get comfortable with Docker for $DAYJOB, there's not a huge difference in complexity, just different complexity, as long as you actually learn the tools.