Hacker News new | ask | show | jobs
by smarx007 1751 days ago
I am not sure. Quite possible bad data was written over SMB, not necessarily that the drives failed. As you see in the article, the damage to images is systemic and drives don't seem to be failing. More like Lightroom/macOS/macbook RAM/Samba wrote (accepted) bad data.
1 comments
sieve 1751 days ago
His entire approach to storage of critical data was a disaster waiting to happen. Storing data on FAT-formatted USB drives and then transferring them to a Btrfs file system over Samba? Why?

If you run an OS with first-class ZFS support and your files arrive on the platform as soon as possible over as few intermediaries as possible, the chances of such mishaps are greatly reduced.

link
smarx007 1751 days ago
Okay, let’s see how we can practically improve this (I am interested in a good solution as well):

1. Camera stores images/videos on an SD card.

2. When the card is full, images are copied in the field on the USB drive. I imagine the user has a 128G SD card and multiple photo sessions will exhaust internal storage quickly, especially on a mac with 256G-512G SSD, hence an external drive. APFS may be used instead of FAT but then the drive won’t be readable on Windows or Linux. Mac doesn’t allow writes to an NTFS filesystem unless you buy a 3rd party driver of unknown reliability. I guess one can try to combine your linked guide with https://github.com/spl/zfs-on-mac to get ZFS on a USB drive but all of that is done at your own risk.

3. When the user gets home, they can plug the drive into a Linux/BSD machine with ZFS. They copy the files locally.

4. They proceed with their favorite workflow on another OS accessing files on ZFS over SMB. Bad writes from macOS over the network can still bork files on ZFS, though you can apply a readonly policy to RAW images.

Step 2 and 4 still look quite hard to work around for me. Bad SD card reader and bad RAM or bugs in any of the 3 machines are still a risk.

link
sieve 1750 days ago
> get ZFS on a USB drive but all of that is done at your own risk.

I don't consider ZFS on portable devices to be more risky than FAT32. You can always copy the files to two devices instead of one. Or carry around multiple SD cards.

> though you can apply a readonly policy to RAW images.

Snapshots will protect you if files are corrupted after they make their way to the file system.

> Bad SD card reader and bad RAM or bugs in any of the 3 machines are still a risk.

Sure they are. But they are not in your control (except RAM, where you can always pay for ECC). I would worry about things that are.

link