Hacker News new | ask | show | jobs
by ignoramous 2345 days ago
BloomFilters make for good indices, too. My first encounter with them was from the BigTable paper.

We used it in front of a Radix Tree that at places had too much depth. The filter took something like 2MiB at 7% false-positive rate vs 100+ MiB for the Radix Tree. We later bitmap-compressed the Radix Tree [0] to under 5MiB at the cost of making the lookups very very slow (P99 at 4ms, P50 at 1ms compared to P99 at 400ns before, iirc) but added a TinyLFU-based [1] lean-HAMT [2] cache of 512KiB in the front to speed lookups of popular items and a Robinhood cache [3] of 256KiB to take care of the long-tail KVs.

[0] https://news.ycombinator.com/item?id=2348619

[1] http://highscalability.com/blog/2016/1/25/design-of-a-modern...

[2] https://blog.acolyer.org/2015/11/27/hamt

[3] https://blog.acolyer.org/2018/10/26/robinhood-tail-latency-a...