Hacker News new | ask | show | jobs
by MurrayHill1980 1130 days ago
If I recall correctly, log structured file systems and in general, making file systems more robust was a real breakthrough. Before that, fsck after a crash seemed a little too exciting. I can't recall the details, but didn't Microsoft go through a similar evolution in the early days of win32 where they recruited a prominent LSFS academic researcher to help them with this problem. The article mentions this but it seemed much more important at the time.
2 comments

> Before that, fsck after a crash seemed a little too exciting.

It was more the opposite IME (90s-era linux): fsck after a crash was boring and time-consuming, and we grew increasingly impatient as capacities grew.

Prior to having a journaling filesystem and log replay @ reboot/mount, the filesystem was often plain conservatively implemented using methods like two-phase-commits and arguably kept more consistent due to the repeated exhaustive checking process at every dirty mount.

We went through a phase of putting a lot of trust into the hardware keeping things consistent/no-bitrot and more or less assuming everything was fine after a journal replay. Only relatively recently did we start getting checksums of metadata to detect such failures, that's still not a universal feature.

The lack of frequent fscks creates opportunity for hidden corruption from either bugs or hardware errors to go unnoticed until it's far too late to recover from... There's a reason filesystems tend to have a max mount count before a fsck is forced, even if not dirty.

Checksums for integrity at all.

Scrubs / checks / etc for online consistency check.

Did log-structured filesystems really take off for mainstream computing? IIRC they suffered from write amplification and required garbage collection.

Journaling did take off in a big way, with NTFS, HFS+, ext3, etc. supporting it. This mostly removed the long fsck on most modern operating systems.

Of course, traditional filesystems + journaling are now rapidly being replaced by CoW filesystems.

ZFS is effectively log-structured. It's what the BSD 4.4 LFS wanted to be. In many ways the two are remarkably similar. ZFS merely solved the problems that had been left unsolved (like garbage collection / free space management), dropped the "all the blocks in a transaction must be contiguous" part, added the ZIL for speed, added a notion of volumes that contain multiple datasets, and reified snapshots.

Log-structured and CoW are very closely related.

My personal recollection is that long FSCK times on ever-increasing disk sizes was the driving force; I remember switching to ReiserFS simply because my 160 GB mp3 drive would take forever to FSCK if the computer lost power or had to do a hard reset. I didn't even mind the amusing file corruption you'd get from tail-packing or whatever it was because I needed that speed.

I later manually did the needful to get XFS working as it was faster on deletes, iirc.

Depends on your definition of mainstream computing. Samsung's f2fs has been in the linux kernel for many years and is used on their Android devices. It can be used on desktop or server SSDs and gives impressive results on some workloads compared to more traditional ones like ext4, but I've been steering clear of it because at least in 2019 it had really bad error detection (just silently discarding most i/o errors, not something you'd want in a filesystem):

https://www.usenix.org/system/files/atc19-jaffer.pdf

It may have improved since then, who knows.