[mtanski]

We do something similar with compressing integers. Unlike the article we pick from 3 different compression schemes: SIMD FastPFOR, Bitpack (max value) and delta encoding.

Which one we pick depends on the distribution of the integers. FastPFOR we use when the values are mostly smallish but there's an occasional large value. Works great. Bitpack we use when there is a tight band (ex. all values are ~5 -> ~7 bits). Delta works well for (mono) increasing values.

Also, we've adapted most of these to 64bit integers since we work with those more commonly.

[cruppstahl]

That's interesting - are the 64bit versions available for download?

[mtanski]

My junior (at the time) coworker created the 64bit implementations by reading the original papers. But you should be able to find other implementations on github now, since the paper has been out for a while. Our code is pretty tied to software (our allocation / buffer handling).

In hindsight, it might not have been the best idea to say. Hey you just started working with C++ and never worked with SIMD, so yeah... I need to extend this SIMD Fast PFOR scheme to 64bit. But he ended up doing quite well.