Hacker News new | ask | show | jobs
by TheGuyWhoCodes 1658 days ago
You could already run SQL on Parquet with DuckDB (even on Java). I believe it already used Arrow under the hood to read the Parquet files, could be wrong tho, but this is a more memory performant integration which is great.
2 comments

DuckDB has its own Parquet reader
DuckDB can read parquet directly - however the interesting bit is that the results of SQL queries can be returned as arrow objects into python for further processing (by pyarrow/pandas).