Hacker News new | ask | show | jobs
by aprdm 1534 days ago
Could you expand what you mean by keeping pointers and basic index in memory?
1 comments

Pointers are tuples of (Id, LogOffset) and are used to map logical identities to positions of those objects in the append-only log.

Indexes are usually a tuple of (Some64BitKey, Id) and are used to map physical business keys to logical object identities. These entries are only candidates in the case where the key material needs to be hashed and inspected for actual equivalence.

One big advantage with this approach is that you can stream big blobs directly out of the log to a caller-supplied buffer or stream. No intermediate allocations required aside from some small buffers.

Awesome, thank you for sharing!