Hacker News new | ask | show | jobs
by TacticalCoder 1381 days ago
> When a set is sparse, traditional bitmaps compress poorly.

They waste space. But if you were to compress them, they'd compress very well. As in, for example, if you were to compress them using roaring bitmaps.

2 comments

If you compressed them using some naive scheme, they would be smaller but they would lose their fast set-wise operations like lookup and union.
Maybe the author means they compress poorly under generic, common compression algorithms. This is an exotic compression.
Yeah pretty much. General purpose compression algorithms do a good job but pale in comparison to what Roaring achieves.

I wouldn't use the word exotic but rather specialized and/or optimized.

Infact later versions of Roaring use run-length-encoding (RLE) containers in addition to array and bitmap containers which is a technique shared with more traditional/general compression algorithms.

In effect rather than achieving "very good" compression, Roaring is very close to "optimal" compression for sparse bitmaps.