Hacker News new | ask | show | jobs
by stargrazer 893 days ago
This reminds me of HDF5, which, even thought the data is written/appended in row format, there is an API to chunk the data, organize into columns, compress based upon column regularities, and write to storage.

On reading the reverse happens.

This becomes the compute/space conundrum: space is reduced with column based regularity, but time is increased due to the extra overhead of columnar compression.