[wtallis]

I've seen a lot of people repeat that assertion about using ECC RAM with ZFS, but I've never seen it explained in detail. Is ZFS actually more susceptible to data loss if RAM gets corrupted as compared with other file systems, or is it just an overblown piece of advice about not forgetting your RAM when trying to make a backup system reliable?

[tachion]

This is overblown at all, ZFS needs ECC as it doesnt detect memory corruption on its own, and here's some explenation: http://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-r...

[stormbrew]

I think the question is, all else equal, is ZFS less reliable than alternative filesystems/software-RAID on non-ECC?

[Amezarak]

Read the linked FreeNAS forum thread. The answer is yes.

On a non-ZFS file system, a bit getting flipped in memory will result in the file being written incorrectly. That sucks. You can read the file ten times and nothing more will happen. Your file is corrupted, maybe even unreadable. If you were really unlucky, maybe some file system metadata was corrupted.

On ZFS, the same single bit being flipped can result in ZFS writing _more_ bad data when the file is read, because ZFS will try to fix the file but can never do it successfully since the memory is bad (and the original checksum was wrong). Hypothetically that could keep happening forever, trashing more and more bits, all because of ZFS.

Now, the real question is how often that scenario can arise and does that justify the cost of ECC RAM in your mind.

[craigyk]

So you're saying if you aren't using RAID on your ZFS vdevs than ZFS with non-ECC is no worse than anything else... and there is a valid scenario for doing so. I use ZFS on offline backup drives so that I can periodically mount them and scrub them too see if they need to be replaced updated. In my mind this is faster and more reliable than any custom verification scheme I might have come up with.

Also, if I understand correctly, your disaster scenario requires the RAM to keep repeatedly fing up in a way that allows ZFS to screw up repeatedly during corrections but coincidentally won't cause the system to outright crash? It's possible, but in my experience most of the ECC errors I've seen are pretty transient.

[takeda]

It looks like it is all speculation by cyberjock, and his assumption that data could shift by 3 bits or that metadata doesn't get corrupted is wrong.

I experienced memory corruption first hand (in my case it was not really bad memory, but bad PCI device which caused random bit flips).

Yes, it caused data corruption, and eventually I got whole pool broken. At one point when I removed a snapshot, my system crashed, and was panicking every time I tried to import the pool.

But I also had data corrupted on SSD that was using UFS2 (it was on FreeBSD).

Actually if anything, in my case ZFS saved me, because of its extensive memory usage (when you have 4GB+ RAM it enables ARC) the system was crashing every 1-2 days

[wtallis]

> because ZFS will try to fix the file but can never do it successfully since the memory is bad (and the original checksum was wrong)

It sounds like this applies to a stuck bit of memory, not just a once-off corruption. If so, then you're still not making a fair comparison - if your memory is broken, then any filesystem will continue to write bad data when flushing cache residing on that broken chip, and if the broken piece of memory is used to store more critical data structures then the potential damage is unbounded. But this is orders of magnitude less common than the transient memory errors that are the actual reason for having ECC.

So once again, assuming the only reasonable scenario of a bit being randomly corrupted in a fully-functional memory module, how much more fragile is ZFS than other modern file systems? Do other filesystems re-read and re-parse their on-disk data structures more often where ZFS caches them in memory? If ZFS writes a corrupted block to disk with a non-matching checksum (or a correct block with corrupted checksum), is that corruption contagious and capable of spreading to other blocks or other files as a result of trying to read the corrupted block back from the disk?

[bsder]

> Read the linked FreeNAS forum thread. The answer is yes.

These are the same guys who took how long (15+ months!) to run down the memory barrier failures on Linux on ARM that gradually corrupted every single filesystem?

Pardon me if I discount the self-righteousness a touch.

That having been said, ZFS has some odd corner cases that people do not expect. The fact that you really want to kill access time modification is one of them.

[craigyk]

I think the answer is No. ZFS users are just extra paranoid and looking for the next SPOF.