[Spooky23]

It’s a bad implementation. You can run inline compression on latency sensitive workloads like VDI without issue.

Compression makes a lot of sense as the cost for fast high capacity SSD is usually much higher than the extra CPU cycles required to decompress.

[stefan_]

It's not squashfs fault, it's the snap people that just have absolutely no clue. squashfs is designed for embedded systems with say 8 or 16 MiB of very slow NOR flash, so you maximize compression ratio at the expense of speed (because the flash is probably still slower).

[stingraycharles]

And decompression is typically very fast. What I don’t understand is why they’re not using something like zstd if they care about speed. It’s a supported compression algo for squashfs, but still they insist on using a single threaded compression (xz iirc?) algo.

[cmurf]

The kernel code for reading zstd squashfs image has been merged for some time. But zstd is only a recently supported algorithm in upstream squashfs tools for creating the squashfs image.

In my testing with OS installs that depend on squashfs+xz, there is a significant lzma hit for decompression, resulting in significant latencies. And the higher the compression level used, the more the hit when decompressing. While compression computational hit for zstd is in the same ballpark as xz to achieve the same compression ratio, (a) decompression computational cost is far less with zstd, translating into faster reads; (b) is fairly consistent regardless of compression level.

Another factor for squashfs is the block size. The bigger it is, the better the compression ratio, but the greater the read amplification. I haven't looked at it, but it might be they're overoptimized for space reduction with too little consideration for performance. Since this isn't a one time use image, like for an installation, but intended to be read over and over again, erofs might be an alternative worth benchmarking.

https://linuxreviews.org/images/d/d2/EROFS_file_system_OSS20...

https://www.usenix.org/system/files/atc19-gao.pdf

https://lkml.org/lkml/2018/5/31/306

[Spooky23]

I think there is a cultural bias in this type of application that favors disk space to reduce overheads on mirrors.