|
|
|
|
|
|
by mwlon
1817 days ago
|
|
|
I made this open source compression algorithm as part of a database I'm creating. It typically compresses columns of numerical data to ~25% smaller than alternatives (think .snappy.parquet or .gzip.parquet) at similar or cheaper compute cost. It decompresses 15-100 million 64-bit numbers per second on a single i5 CPU per second. I also made a blog post that introduces the idea more from the math perspective: https://graphallthethings.com/posts/quantile-compression |
|
|