Hacker News new | ask | show | jobs
by kordlessagain 1378 days ago
Instead of storing values, like "dog", "cat", or "mouse" it stores (in this example) three binary numbers:

000 - whatever needs to associate with animals, but has no associations currently

001 - whatever it is is associated with having a "mouse" included

111 - whatever it is is associated with having a "dog", a "cat" and a "mouse" included

In the past, high cardinality data sets weren't good for storing in binary form, or a binary index, but nowadays there are ways around this. So, that list of animals could be quite large.

The primary reason it's so much faster is that many CPUs nowadays can do 10s of lookups in a single instruction cycle. That makes them extremely fast.