[jcalvinowens]

I've been running homebuilt NAS for a decade. My advice is going to irritate the purists:

* Don't use raid5. Use btrfs-raid1 or use mdraid10 with >=2 far-copies.

* Don't use raid6. Use btrfs-raid1c3 or use mdraid10 with >=3 far-copies.

* Don't use ZFS on Linux. If you really want ZFS, run FreeBSD.

The multiple copy formats outperform the parity formats on reads by a healthy margin, both in btrfs and in mdraid. They're also remarkably quieter in operation and when scrubbing, night and day, which matters to me since mine sits in a corner of my living room. When I switched from raid6 to 3-far-copy-mdraid10, the performance boost was nice, but I was completely flabbergasted by the difference in the noise level during scrubs.

Yes, they're a bit less space efficient, but modern storage is so cheap it doesn't matter, I only store about 10TB of data on it.

I use btrfs: it's the most actively tested and developed filesystem in Linux today, by a very wide margin. The "best" filesystem is the one which is the most widely tested and developed, IMHO. If btrfs pissed in your cheerios ten years ago and you can't figure out how to get over it, use ext4 with metadata_csum enabled, I guess.

I use external USB enclosures, which is something a lot of people will say not to do. I've managed to get away with it for a long time, but btrfs is catching some extremely rare corruption on my current NAS, I suspect it's a firmware bug somehow corrupting USB3 transfer data but I haven't gotten to the bottom of it yet: https://lore.kernel.org/linux-btrfs/20251111170142.635908-1-...

[QuiEgo]

I use mergerfs + snapraid on my HDDs for “cold” storage for the same reason: noise. Snapraid sync and scrub runs at 4am when I am not in the same room as the NAS.

The drives stay spun down 99% of the time, because I also use a ZFS mirrored pool on SSDs for “hot” files, although Btrfs could also work if you're opposed to ZFS because it's out of tree.

Basically using this idea, but with straight Debian instead of ProxMox: https://perfectmediaserver.com/05-advanced/combine-zfs-and-o...

I also use mergerfs 'ff' (first found) create order, and put the SSDs first in the ordered fstab list of the mergerfs mount point. This gives me tiered storage: newly created files and reads hit the SSDs first. I use a mover script that runs nightly with the SnapRAID sync/scrub to keep space on the SSDs open.

https://github.com/trapexit/mergerfs/blob/master/tools/merge...

[vardump]

I have had zero issues running ZFS on Linux for the last 10 years. (Not saying there were no issues that have annoyed or even caused data loss.)

[WarOnPrivacy]

I was wondering what the parent's beef was with ZFS on Linux. I have a box I might change over (B-to-L) and I haven't come across any significant discontent.

[jcalvinowens]

No beef: I just simply don't run out of tree kernel code, I've been burned too many times. Linux ZFS is mostly used by hobbyists and tinkerers, it doesn't get anything close to the amount of real world production testing and follow up bugfixing with linux that a real upstream filesystem like btrfs does today.

If ZFS ever goes upstream, I will certainly enjoy tinkering with it. But until it does, I just don't see the point, I build my own kernels and dealing with the external code isn't worth the trouble. There's already more than enough to tinker with :)

All my FreeBSD machines run ZFS, FWIW.

[queenkjuul]

I've even been using ZFS on Linux with USB enclosures for 5+ years with no issues.

[jcalvinowens]

This is the first time I've ever had a problem with the USB enclosures. And its fantastically rare, roughly one corrupt 512b block per TB of data written. With a btrfs-raid1 it's self-correcting on reads, if I didn't look at dmesg I'd never know.

I've figured out it only happens if I'm streaming data over the NIC at the same time as writing to the disks (while copying from one local volume to another), but that's all I really know right now. I seriously doubt it's a software bug.

[whatevaa]

Btrfs raid was also the one who had data loss bugs.