[cyphar]

You can grow and add/remove raid1 devices (mirror vdevs) in ZFS without any significant work or downtime. Shrinking does require a bit more work, but depending on your setup it can be done fairly painlessly with send/recv (and shrinking is usually not something which is a very common administrative operation).

[xzcat]

"fairly painlessly" and "without significant work or downtime" doesn't sound like it lines up with btrfs's, which I would describe as "one command and zero downtime (just some io load if you rebalance immediately)" for both operations. btrfs is also mainline, which increases how painless it is to use.

BTRFS does have some scary stories from earlier in its development, and true raid5 seems like it's unlikely to be safe for quite a while, but raid1 and "normal" fs usage has been rock solid in my experience. The only time I've ever had an issue was probably 4 years ago at this point, and it was solved by just booting an Arch live iso and running a btrfs command that was basically "fix exactly the bug that your error message indicates". I don't remember exactly what it is, something about two sizes not matching, but googling the text it showed at boot led me directly to the command to fix it. Certainly dramatically less trouble than I've ever had when hardware RAID goes south.

I do agree that modern lvm does probably compete with btrfs, but again you're trading how dang simple btrfs raid1 is to manage for monkeying with partitions in lvm in exchange for ~some? performance.

IMO ZFS is in a weird spot where I don't know where I'd use it. It's too complicated/annoying to admin for me to want to run it in my basement for myself/my family, and for anything bigger or more professional I'd use ceph or a problem-domain-specific storage system (HDFS, clickhouse, aws, etc).

[cyphar]

The first operations I mentioned (adding or removing a device from a vdev, or adding a new vdev) are one command with no downtime:

  % zpool attach <pool> <vdev> <device> // add to a vdev
  % zpool detach <pool> <vdev> <device> // remove from a vdev
  % zpool add <pool> <vdev> <devices...> // add a new vdev

In the newest ZFS versions, you can also remove mirror and singleton vdevs (this does require some time -- because the data needs to be copied from the drives) but it's all done in the background:

  % zpool remove <pool> <vdev>

Shrinking a pool "the old way" (which is still sometimes necessary depending on what you're doing) is definitely more involved -- you have to create a new pool with the layout you need and then do a zfs send/recv from your old pool to the new one. This does only take a handful of commands but I would definitely consider it to be a much more complicated affair than the operations I mentioned above.

I would not (nor did I) compare LVM (or md-raid) to btrfs or ZFS -- those technologies have fundamental limitations regarding the integrity of your data that ZFS (and btrfs) don't have. And don't get me wrong -- I don't have a problem with btrfs (I run btrfs on all of my machines except my home server -- which runs ZFS), I just disagree with GP's point that ease of use is an argument for btrfs over ZFS. There are many arguments for either technology.

> btrfs is also mainline, which increases how painless it is to use.

I agree that this is one argument to pick btrfs over ZFS (though on most distributions it isn't really that hard to install ZFS, the fact that btrfs requires zero extra work to use on Linux is a benefit).

[3fe9a03ccd14ca5]

How? My understanding is that you create a new vdev and add the old vdev as a device, basically recursively creating volumes with each new device you add.

[cyphar]

Which operation are you asking about? [1] is a sister comment which I posted that outlines how to do most of the operations I mentioned.

[1]: https://news.ycombinator.com/item?id=22168494