[lrm242]

Blosc is an outstanding project. I have used it with great success in finance and general data science in production with very large total datasets (one custom binary format and one leveraging protobufs).

It really shines first and foremost as a meta compressor, giving the developer a clean block based API. Once integrated (which really is quite easy) you can experiment easy with different compressors and preconditioners to see what works best with your dataset. These things can be changed at runtime and give you great flexibility.

Francesc has been advancing blosc consistently with a steady vision for years and years. It is one of the most underrated tools around IMO.