Hacker News new | ask | show | jobs
by tonyarkles 3522 days ago
An interesting angle that I read a paper on in grad school a few years ago but don't have handy: because of the asymmetric growth of CPU speed vs. hard drive speed, you can actually get performance gains by enabling filesystem compression. It seems counter-intuitive, but it boils down to "can I compress this data faster than the disk can write it?"

If you're blasting highly compressible data to disk, compressing it on the fly can, in some circumstances, have a net bandwidth greater than just writing the data to disk (and greater than the disk alone is capable of). Yes, you incur more CPU load, but it's a net win. It's not universally true, YMMV etc.

2 comments

One counter to this argument is that, while the CPU has thousands of cycles to do compression while an old 5400 RPM spinning rust drive seeks or writes, a modern SSD like the new Samsung 960 Pro can write data at 2100 MB/s, giving the CPU perhaps one cycle per byte, which makes compression difficult.

The counter to my counter, of course, is that specialized silicon for compression can easily keep up with even these speeds. In fact, Sandforce SSD controllers build in compression to boost read and write speeds!

Thanks for the reminder that while grad school doesn't feel that long ago, spinning rust was definitely the name of the game at the time. SSDs were a thing, but way too expensive and small for non-exotic purposes... :D
Sandforce does compression for minimizing writes, not for speed. They are slower than non-compressing controllers for some tasks because of that, but it should increase the device's lifetime.
I can say that's true from empirical experience - doubly so if you're targeting an environment where the data's likely to be stored on a NAS or some other fairly slow storage.

(And yeah, natch, insert recitation of the Liturgy of the Optimizer as a charm against the appearance of a certain Knuth quote.)