Hacker News new | ask | show | jobs
by ed25519FUUU 2031 days ago
I noticed that enabling compression on zfs made a huge difference with the source size of some of my largely text file petitions. I never turned on deduplication because I don’t want to bother with the memory overhead, but I bet that would help even further.
1 comments

Most ZFS howto's now recommend against dedup on the prolongued memory cost consequences. Yes, you would get some block level compression outcome. But, you enter the cost/benefit hell of balancing CPU and memory at runtime.
Can't you periodically run the dedup out of band (for example whenever you scrub)? https://btrfs.wiki.kernel.org/index.php/Deduplication
I was just googling this myself and I think this is a feature that btrfs has over zfs. There’s no way to do native offline deduplication as far as I could find.