[nimbius]

>ZFS on Linux

its been said in the thread already but this was always a non-starter. Torvalds even said so himself. CDDL was the last poison pill of a dying giant who couldnt pull its foot from the well.

What we, er, the linux community, chose instead, was BTRFS. It isnt ZFS, but its made incredible strides. for most use cases, it is a reasonable and working replacement for ZFS.

[laumars]

> for most use cases, it is a reasonable and working replacement for ZFS.

That is a huuuuge overstatement for the current state of Btrfs. In some specific domains it is a working replacement. But for most domains it still falls far behind ZFS in terms of stability, resiliency or even ease of use.

By all means if you want to use btrfs then go for it. But the favourable comparisons people make when comparing btrfs to ZFS is a combination of wishful thinking and not having really bullied their fs into those extreme edge cases where the cracks begin to show. And frankly I’d rather depend on something that has had those assurances for 10+yrs already than have the hassle of explaining downtime to restore data on production systems.

[josteink]

> What we, er, the linux community, chose instead, was BTRFS. It isnt ZFS

Speak for yourself. As a part of “the Linux community” I gave btrfs a fair chance, but stopped using it because it constantly failed on me in ways no other fs had done before and didn’t protect my data.

ZFS is rock solid and I’ve never had any of the issues I had with btrfs.

So as a member of “the Linux community” you claim to unilaterally represent I put such petty license-politics aside and choose the file system which serves my needs best, and that is ZFS.

[guenthert]

BTRFS is the only fs I recall having lost data to (rather it corrupts data, i.e. data of one file is found in another) as late as 2018.

[_hyn3]

I lost data and had to restore from 12 hour old backups one too many times with BTRFS. XFS + Ext4 for me from here on out, but that's one of the great things about Linux: lots of choices.

[acomjean]

Lamentably, for those of us that just use linux, the lots of choices seem weird and frankly a little scary (Stories of data loss with BTRFS I have heard before.) I use the default Ext4? I think?

But as a believer in open source, I really would love it if the choices we had were so much superior to the proprietary stuff that was out there that it made using an open source OS a no brainer.

[nix23]

XFS all the way down...or ZFS (on FreeBSD)

[ahl]

What is the latest on RAID-5/6 support for BTRFS. RAID-Z has its issues (saying this as the triple- and double-parity author), but it's been stable.

[simcop2387]

As far as Ive heard it's still pretty iffy. Since you were involved with the zfs side of things what are your thoughts on the upcoming zfs draid bits? I don't have specific need for them myself but they look really attractive for building a new pool to replace my aging drives.

[ahl]

Unfortunately I've been completely out of the loop on the draid stuff.

[guardiangod]

>What we, er, the linux community, chose instead, was BTRFS.

Isn't that putting politic before technical excellence, something the Linux crowd is proud of? Other than in place volume expansion, there is no technical reason to choose BTRFS over ZFS (for now.)

I don't really see a killer feature from BTRFS that would persuade me to take a chance with it.

[blihp]

It's being pragmatic. Linux has typically placed freedom ahead of pretty much everything else.[1] All else being equal, sure you want the best technical solution. But if it doesn't fit the definition of freedom that Linux requires, how otherwise good a solution is doesn't matter. So the main 'killer' feature of BTRFS is that it fits the licensing requirements for integration into Linux. Linux has a great many problems, but being sticklers for a particular type of license isn't one of them IMO.

[1] This isn't just idealism. See Oracle v. Google for an example of what happens if you play fast and loose with licenses and a malicious actor. Google eventually won, but how many millions of dollars did that victory cost them? Oracle would love Linux developers to blunder their way into the receiving end of a lawsuit.

[rnd0]

>Isn't that putting politic before technical excellence, something the Linux crowd is proud of?

It's not unprecedented. The adoption of systemd was forced on distros through political pressure, and not for technical reasons.

If you want a truly non-political OS community these days, I think you're basically stuck with OpenBSD. No CoC, no systemd, no political BS at all -just pure tech.

(there's other problems with OpenBSD -performance, mostly; that's why I use windows and Ubuntu instead. But the way they run things is admirable IMO. Blatant BS isn't tolerated.)

[dale_glass]

systemd wasn't "forced" on distros. The distros adopted it because they liked it.

The thing is that systemd did something quite clever -- it sold itself to the people actually building distributions, which are the people that actually matter the most in regards what system software gets used. It made their jobs easier and less annoying in many ways.

As somebody who's done a lot of packaging and writing of SysV scripts, I can tell you that it's a tiresome and annoying task even for a small amount of software, let alone a whole distro. At that point the unix philosophy loses its luster quite a bit.

[bsder]

> It's not unprecedented. The adoption of systemd was forced on distros through political pressure, and not for technical reasons.

Sorry, I'm going to slag on this.

Anyone could have put in the work to make a better init experience. No one put in that work.

System Management Facility (SMF) existed on Solaris since 2005 (systemd didn't appear until 2011?). launchd on OS X dates to a similar time. Someone could have copied them--no one did.

Even once it became clear that systemd was going through, still nobody could muster the work to put together a viable alternative.

Where's the "meritocracy through code" Linux mantra in all of this?

You can say what you want about Poettering, but he put in the work to write the code. Nobody else did.

Perhaps the problem is that an init system is a metric boatload of finicky code that nobody had the guts or skills to drive to completion?

And for the old-init bigots, sorry, that wasn't working in spite of what you claim. The fact that Windows, OS X, Solaris, etc. (and then systemd) all converged on essentially the same design is because of common needs on modern computers.

[the_why_of_y]

There was upstart, but it didn't work very well, mostly because it pre-dated the kernel mechanisms like cgroups that enabled a reliable service manager to be implemented. So it was only used in its sysvinit-compatible mode, not its native upstart mode.

https://bugs.launchpad.net/upstart/+bug/406397/comments/21 https://bugs.launchpad.net/upstart/+bug/447654/comments/6

[bombcar]

The Linux Community is working on BTRFS but if ZFS emerged with a GPLv2 compatible license tomorrow BTRFS would likely be moribund.

[znpy]

Yup. If openzfs was gpl-ed tomorrow morning, btrfs would be dead by tomorrow at lunch time.

[jjav]

When filesystem integrity matters, the filesystem matters more than the OS.

While I mostly use Linux these days, for file servers it must be ZFS, which means whichever OS has first-class support for ZFS. I'm still on Illumos but perhaps will move to FreeBSD at some point.

[arid_eden]

This is why FreeBSD rebasing its ZFS fork on ZFS-on-Linux made me so scared for the future of FreeBSD. Their one major advantage over Linux and they didn't have the developers to maintain their fork themselves.

[laumars]

ZFS will always be a smoother experience on FreeBSD as opposed to Linux because FreeBSD endorses it. Thus the user land and documentation is written assuming you’re running ZFS. As opposed to Linux where some distros might ship pre-compiled binaries but everything is written assuming you’re not running ZFS. Thus everything takes that extra couple of steps to set up, fix, and maintain.

For example, if you want to use ZFS as a storage for containers on Linux, you have to spend hours hunting around for some poorly maintained 3rd party shell scripts or build some tooling yourself. Whereas on FreeBSD all the tooling around Jails is built with ZFS in mind.

This is why platforms like FreeBSD feel more harmonious than Linux. Not because Linux can’t do the job but because there are so many different contributors with their own unique preferences that Linux is essentially loose Lego pieces with no instructions. Whereas FreeBSD has the same org who manage the kernel, user land and who also push ZFS.

And I say this as someone who loves Linux. There’s room for both Linux and FreeBSD in this world :)

[mappu]

I think "smoother experience on FreeBSD" is a myth -

The standard volume manager on FreeBSD is vinum/geom; ZFS ships its entire separate volume manager to the host OS, so you can't use mount/umount to control mounting a ZFS volume. Maybe it would be okay to move entirely over to ZFS's volume manager but it only supports ZFS's own filesystem, you can't use the ZFS volume manager with a normal FreeBSD UFS2 partition.

In both Linux and FreeBSD, ZFS's bolt-on ARC competes with the kernel's actual page cache for resources instead of properly integrating with it.

It's an out-of-tree filesystem for both OSes. Sure FreeBSD periodically imports it into master from OpenZFS (née ZoL), but all development happens elsewhere, and the SPL is still trying to emulate a Solaris interface on top of both OSes.

Is there any more concrete example of how ZFS is actually better integrated on FreeBSD compared to Linux, say Ubuntu? It takes ZFS snapshots automatically during apt upgrades, root-on-ZFS is a default installer option, etc.

[laumars]

Coincidentally there was a discussion about this yesterday. I agree with a lot of what was posted in it so might be easier to share that: https://news.ycombinator.com/item?id=27059551

This branch in particular addresses your points: https://news.ycombinator.com/item?id=27062069

Has the latest version of Ubuntu finally made mirrored ZFS root pools painless? Because that was anything but a native out of the box experience (compared to setting up the same on FreeBSD) and that has bit me several times.

I've use ZFS on both FreeBSD and Linux for years and while Ubuntu is closing the gap, ZFS has been the default recommended file system on FreeBSD for close on 10 years already. So it's bound to feel more like a native experience on FreeBSD.

> In a review in DistroWatch, Jesse Smith detailed a number of problems found in testing this release, including boot issues, the decision to have Ubuntu Software only offer Snaps, which are few in number, slow, use a lot of memory and do not integrate well. He also criticized the ZFS file system for not working right and the lack of Flatpak support. He concluded, "these issues, along with the slow boot times and spotty wireless network access, gave me a very poor impression of Ubuntu 20.04. This was especially disappointing since just six months ago I had a positive experience with Xubuntu 19.10, which was also running on ZFS. My experience this week was frustrating - slow, buggy, and multiple components felt incomplete. This is, in my subjective opinion, a poor showing and a surprisingly unpolished one considering Canonical plans to support this release for the next five years."

This was Ubuntu's latest LTS release, which is less than a year old. Granted not all of the criticism levelled against it are ZFS related and granted that's just another persons anecdotal report but it mirror the same experiences everyone else, aside from yourself it seems, raises when switching between Linux and FreeBSD for ZFS storage.

I don't post this as a hater though. I, like others, do still run Ubuntu Server + ZFS for some systems (particularly where I wanted ZFS + Docker) and those systems do run well. But I can't deny that everything requires just a little more effort to get right on Linux because there isn't the assumption you're running ZFS where as on FreeBSD it more or less pre-configured to use ZFS right out of the box because that's the expectation. eg FreeBSD containers tooling is already written to support ZFS where as Linux container tooling isn't.

This is why people talk about a smoother experience on FreeBSD. The file system itself is the same code base and performs largely the same. But it's all the stuff around the edge that is built with the assumption of ZFS on FreeBSD that makes things feel a little less hacked together with duct tape.

[arid_eden]

My main issue with ZFS is the integrated nature - like systemd for filesystems. My 'alternative' for ZFS isn't BTRFS (awful performance characteristics for my workloads) but LVM coupled with ext4 and mdraid. I get snapshots, reliability, performance and a 'real UNIX' composable toolchain. I miss out on data checksums.

[e12e]

In principle I dislike the coupling of volume manager, raid and filesystem.

But I still think zfs gets most things right; I see the argument for a concistent system managing caching/logs, volumes, data integrity, discard support, compression, snapshots and encryption.

The fact that it's the first serious, open, cross platform solution (Linux, bsd, Mac, winnt) that provides encryption, integrity and filesystem is a nice bonus.

And the integration of snapshots and fs dumps via zfs send/receive is beautiful.

I think zfs makes sense like one fat layer - networking can go below (drdb, iscsi) or on top (iscsi, nfs, cifs).

Encryption need to be somewhat holistic - for making sane performance and data leaking tradeoffs.

[salmo]

Having run all these thing in prod (except BTRFS, it ate a mirror on my desktop), I’ll say that even the LVM + + is so much more hacky than geom on FreeBSD which feels much more ‘Unix’ with a designed composable interface.

Although, I do prefer the durability of XFS or ext4 (depending on workload) vs UFS, and the setup you described is totally maintainable.

[riku_iki]

No compression..

[bombcar]

You could use VDO (I never have) https://access.redhat.com/documentation/en-us/red_hat_enterp...

[nix23]

>Torvalds even said so himself.

Torvald's comment about ZFS was as uninformed as it gets...and he calls himself an FS-Guy ;(

[arid_eden]

His comment was more on the wisdom (or otherwise) of running an out of tree filesystem. I think its hard to disagree with him. He went on to say you would never be able to merge the ZFS tree with Linux. Again he's the one who would know what code gets in Linux. His only actual comment against ZFS was that benchmarks didn't look great - which is unsurprising given all the extra work ZFS is doing wrt data integrity than other filesystems in production use.

https://www.realworldtech.com/forum/?threadid=189711&curpost...

[wang_li]

From the link:

>[ZFS] was always more of a buzzword than anything else, I feel,

This is deeply ignorant. I feel that Linux has been handicapped by the fact that many developers have never done any serious enterprise administration and thus not having clear understanding of the needs of a set of their users.

[nix23]

Not everything needs to be in the Linux Kernel (and honestly i don't care if it is), looking at the past "linux-sound-system-tragedy" i would say, to make something outside linux is often much better (not 1000's different peoples who thinks it's better the other way around, and you are full of sh* anyway).

>His only actual comment against ZFS was that benchmarks didn't look great

What benchmark? Mines are looking pretty good, with a preheated Arc and with L2 especially...actually much better than any HW-Raid. Compared to Linus, there are Institutes with a bit more than a single 3GB git repository, and the crazy stuff...they need verified backups.

https://computing.llnl.gov/projects/zfs-lustre

If he ever has to improve a Lustre Filesystem of 55 petabyte with ZFS, he can come back, otherwise Linus...shut-up* and be happy with your ext4 (nothing against that one).

* A homage to the old linus-style of having a discussion.