[_flux]

My personal grievances with btrfs are multifaceted.

- I never agreed with the btrfs default of root raid 1 system not booting up if a device is missing. I think the point of raid1 is to minimize downtime when losing a device and if you lose the other device before returning it to good state, that's 100% on you.

- Poor management tools compared to md (though bcachefs might be in the same boat). Some tools are poorly thought, e.g. there is a tool for defragmentation, but it undoes sharing (so snapshots and dedupped files get expanded).

- If a drive in raid1 drops but then later comes back, btrfs is still quite happy.

- Need of using btrfs balance, and in a certain way as well: https://github.com/kdave/btrfsmaintenance/blob/master/btrfs-... .

- At least it used to be difficult to recover when your filesystem becomes full. Helps if you have it on LVM volume with extra space.

- Snapshotting or having a clone of a btrfs volume is dangerous (due to the uuid-based volume participant scanning)

- I believe raid5/6 is still experimental?

- I've lost a filesystem to btrfs raid10 (but my backups are good).

- I have also rendered my bcachefs in a state where I could no longer write to the filesystem, but I was still able to read it. So I'm inclined to keep using bcachefs for the time being.

Overall I just have the impression that btrfs was complicated and ended up in a design dead-end, making improvements from hard to difficult, and I hope that bcachefs has made different base designs, making future improvements easier.

Yes, the number of developers for bcachefs is smaller, but frankly as long as it's possible for a project to advance with a single developer, it is going to be the most effective way to go—at the same time I hope this situation improves in the future.

[viraptor]

> I never agreed with the btrfs default of root raid 1 system not booting up if a device is missing.

Add "degraded" to default mount options. Solved.

[jeltz]

Bad defaults is a huge issue even when you can change the config to something sane.

[PlutoIsAPlanet]

Defaulting to degraded is a bad default. Mounting a btrfs device array degraded is extremely risky, and the device not booting means you'll actually notice and take action.

[_flux]

Md devices do "degraded" by default and it seems fine. Indeed, I believe this is the default operation of all other multi-device systems, but of course I cannot verify this claim. I dislike all features that by default prevent from booting my system up.

The annoying part of this is that if you do reboot the system, it will never end up responding to a ping, meaning you need to visit the host yourself. In practice it might even have other drives you could configure remotely to replace the broken device. I use md's hot spare support routinely: an extra drive is available, should any of the drives of the independent raids fail.

Granted, md also has decent good monitoring options with mdadm or just cat /proc/mdstat.

[tiberious726]

Defaults should favor safety over convenience. Fail fast and fail hard. Md-raid's defaults are simply wrong.

[_flux]

Based on this logic, should the default mode of operation be to stop I/O when a disk dies?

[jeltz]

As someone else pointed out: how is that different from losing a disk while running? Do you want the file system to stop working or become read-only if one disk is lost while running too? I think the bahviour should be the same on boot as while running.

[yencabulator]

You want a flaky system to pick a random disk at every boot and pretend that's the good one?

[mst]

The usual reason I've seen RAID 1 used for the OS drive is -so- it still boots if it loses one.

Not doing so is especially upsetting when you discover you forgot to flip the setting only when a drive fails with the machine in question several hours' drive away (standalone remote servers like that tend not to have console access).

I think 'refusing to boot' is probably the right default for a workstation, but on the whole I think I'd prefer that to be a default set by the workstation distro installer rather than the filesystem.

[viraptor]

That sounds like the right default then. If you're doing a home install, you get that extra little bit of protection. If you're doing a professional remote server deployment, you should be a responsible adult understanding the choices - and run with scrubbing, smart and monitoring for failures.

[mst]

"Will my RAID configuration designed so my system can still boot even if it loses a drive not actually let it still boot if it loses a drive?" is not a question that I think is fair to expect sysadmins to realise they need to ask.

Complete Principle of Least Surprise violation given everything else's (that I'm aware of) RAID1 setups will still boot fine.

Also said monitoring should then notify you of an unexpected reboot and/or a dropped out disk, which you can then remediate in a planned fashion.

If this was a new concept then defaulting all the safety knobs to max would seem pretty reasonable to me, but it's an established concept with established uses and expectations - a server distro's installer should not be defaulting to 'cause an unnecessary outage and require unplanned physical maintenance to remediate it.'

[cesarb]

> and the device not booting means you'll actually notice and take action.

How often do people reboot their systems? IMO, if it's running (without going into "read-only filesystem" mode) with X disks, it should boot with the same X disks. Otherwise, it might be running fine for a long time, and it arbitrarily not coming back in case of a power failure (when otherwise it would be running fine) is an unnecessary trap.

[curt15]

Since RAID is not a backup, isn't availability the main point of RAID1?

[cherryteastain]

Good luck doing that after the disk shuts down okay but never comes back online

[viraptor]

This option is only needed of you can mount the filesystem, but only degraded/to. If you're in that situation, you can remount easily. If not, that's not the solution you need.

[edward28]

Add to the grub kernel option, rootflags=degraded.