Hacker News new | ask | show | jobs
by KitDuncan 1969 days ago
I have a ZFS pool with exclusively video files. Probably won't see any benefit in enabling LZ4 there right?
4 comments

Probably not, but ZFS is actually smart enough to store the uncompressed version if compression doesn't save space. In other words: zfs will try lz4 compression on those video files, notice that it doesn't gain anything and store it uncompressed.

[0] https://klarasystems.com/articles/openzfs1-understanding-tra...

Assuming you have a larger record size than block size (with media files you probably want a 1M record size), and that you probably have ashift set to 12 (4k) or 13 (8k), then I believe you need to enable compression in order to prevent the final record from using the full record size. IOW, ZFS will pad out the final record with zeros to the full record size, then use compression on it (if enabled) so that the zeros don't need to be written to disk.

This article refers to the padding on the final record as "slack space" and states that you need to enable compression to eliminate the slack space.

https://arstechnica.com/information-technology/2020/05/zfs-1...

See also:

https://old.reddit.com/r/zfs/comments/gzcysy/h264_h265_media...

LZ4 is not going to help, if you already have compressed data.

For image, video and audio there are more efficient compressions taking advantage of those formats.

I compression not on by default nowdays? Anyhow, I would not run ZFS with compression disabled completely. There are edgecases where you want it. The meta data? I can't remember the details. At least active the compression that just compresses zeros.
I enabled it for my media storage, knowing that it wouldn't matter much, and indeed it doesn't. Compressratios are 1.00, 1.00 and 1.01.
Since you can control compression settings per dataset, I'd just use a separate dataset for the videos directory with compression disabled.