Hacker News new | ask | show | jobs
by aynyc 119 days ago
Yes, but parquet hates small files.
1 comments
dionian 119 days ago
You can't compact? i.e. iceberg maintenance
link
aynyc 118 days ago
We might be doing something wrong, but we saw significant performance degradation for both ingestion and query when doing compaction when it comes to finance data during trading hours.
link