Hacker News new | ask | show | jobs
by cm2187 801 days ago
you mentioned NLJSON and CSV, which would require to read all columns from the disk.
1 comments
orthoxerox 801 days ago
Yes, but you would usually have to read at least two columns anyway. What are the datasets that are too large to be ingested completely, but too small for a proper columnar format?

If ZSV is meant to occupy the gap between CSV/NLJSON (smaller datasets) and Parquet/DuckDB (larger datasets), this niche is actually really small, if not nonexistent.

link
cm2187 801 days ago
yes it's unclear to me what is the advantage over parquet with compression. And there are enough file formats flying around already.
link