Hacker News new | ask | show | jobs
by debugnik 3 days ago
> compressed[1]

You missed adding the reference to [1], I'm curious now.

1 comments

I was going to elaborate and say that even though typical columnar databases are already compressed with some variant of dictionary lookup compression, I'd like to see a database engine where large objects (bulk text or binary data) is stored efficiently by default. If I were to wave my hands about, I'd say something like a Merkle or Prolly tree of large ~256KB chunks stored in deduplicated external blob storage, where the individual chunks are compressed with a modern throughput-optimised algorithm.