[russdill]

Crashes in the middle of the night are not what worries me. Who cares. It's silent data loss that can go unnoticed for a very long time. And not just a single bit. If the flip hits file system structures or file layout you can have massive silent data loss.

[MrFoof]

Yep. It's why ZFS, BTRFS, Ceph and Gluster matter. Being able to detect that data at rest has gone wrong, and being able to reconstruct the original state is a big deal.

I'd like to think that as NAND continues to scale up in capacity and lower in cost, that we'll see some real shakeup to filesystems and storage where self-healing mass storage can be genuinely commoditized -- not something that's only accessible to businesses (and computing enthusiasts) due to cost and complexity.

[kstenerud]

Absolutely, but these are only half the solution. You still have to be sure that the data you're passing to the filesystem is not already corrupted in memory.

[MrFoof]

Indeed. I have a very low power storage server with tons of ECC, and the better consumer grade NASes also tend to have it. Again, the issue here is its functionally limited to enthusiasts today because of cost and the average person being completely unaware of the impact.

My hope as we move into more advanced fabrication nodes is the increasing shift to HBM in the data center space starts to at least create an HBM option in the consumer side of things. I expect at least AMD to try that push in 2027 and beyond, and I’m sure Intel is looking hard at it too. Granted, Apple is already there with its higher end silicon.

Granted, I still expect there to be product line segmentation with ECC, as it’s a good lever to push a buyer into a higher end product. Though when it’s done on package, you at least eliminate the need for a main board to actually have the traces, and the external modules to have the extra memory. So it might be the easier route to get to more ECC in the consumer space, at least for mid-range and up personal computers.

[bartvk]

My most precious personal data are my family pictures. I protect them with par2. These are basically checksums for your files, but they provide so much added information that you can also repair your files if they are damaged.

Once I coded a shell script that verified all my photos, but I don’t bother with that anymore. I just back everything up, and if there’s ever a problem, the parity files provide an additional safety net.

[semi]

How do you protect/ensure the integrity of your par2 files?

[bartvk]

In case you’re not joking: I don’t. It’s the one and only line of defense.