[tpetry]

Zstd compression with configurable levels is really interesting: You could write every block first with a level comparable to lz4 for very fast performance. And if a block has not been rewritten for some time you recompress them with a compression level allowing more compression and comparable decompression performance.

So cold data (cold write, cold/hot read) will take less and less space over time while still having the same read performance.

[KMag]

That would be an even more interesting feature for NILFS2, as I understand it, its ring buffer structure requires moving the oldest unmodified blocks as the ring buffer write frontier approaches. Any blocks that are forced to be copied are by definition old and unmodified, and need to be moved anyway, so why not recompress? AFAIK, there are no plans for compression in NILFS, but I think it's an interesting idea.

[rcthompson]

My understanding is that for ZFS, things like this would require a mythical feature called "block pointer rewrite", the same feature required to implement out-of-band deduplication.

[rincebrain]

You are correct - ZFS hardcodes the assumption that data's location on disk will never change once written very deeply, and offline dedup/data migrating of any sort would require that.

(It would also be a performance nightmare - you'd have a permanent indirection table you'd need to use for _everything_, and if you've ever seen how ZFS dedup performs with its indirection table not on dedicated SSDs, you can understand why this is terrible.)

[tpetry]

The block could still be rewritten from the view of zfs as long as it does not update the last-written timestamp (does zfs have this?). I was just describing how it would look like from the birds eye.

[rdc12]

Directly no, but if you moved the data to a new dataset, with a command that preserves the timestamp that would work (rsync -a or zfs send/recv), which could be run from a cronjob.

Compression settings are set at a per dataset level, so applying this to only some files in a dataset isn't practical.