Hacker News new | ask | show | jobs
by overbytecode 815 days ago
How do you decide when to reach for Polars vs DuckDB?
2 comments
jasonjmcghee 815 days ago
In Python, I think of them as, DuckDB is for getting the data you want, in the form you want it, from a file / remote place. Polars is for when you want to do something with that data, like visualize it.

`duckdb.sql("SELECT foo, bar * 2 as bar_times_2 FROM ...").pl()` (now in polars format) -> other stuff

In Rust, it's a bit fuzzier to me, as DuckDB is a pretty heavy dependency. I'm looking more and more fondly at DataFusion.

link
nozzlegear 815 days ago
Do you mean Polars depends on/uses DuckDB pretty heavily in Rust? I'm only just now dabbling in Rust myself so I'm not familiar.
link
jasonjmcghee 814 days ago
No not at all. Polars is not dependent on DuckDB.

DuckDB is a heavy dependency in terms of size. It's written in C++, so you can't work with it like a native rust dep.

link
joelschw 815 days ago
Or you use Ibis and switch between the two at will!
link