Hacker News new | ask | show | jobs
by co0lster 216 days ago
650GB relates to size of parquet files which are compressed in reality it’s way more.

32 GB of parquet cannot fit in 32GB of RAM
2 comments
m00x 216 days ago
You don't need it to if you just need specific columns. This is the advantage of columnar storage.
link
barrkel 216 days ago
This would speed things up since it looks like the bottleneck here is I/O.
link