[ritchie46]

I do think code should be shared when you are benchmarking. He could be using Polars' eager API for instance, which would not be apples to apples.

[mwc360]

Hi - Miles Cole here… I used lazy APIs where available. I.e. everything up to write_delta() is lazy in the Polars (Mod) variant.

Yeah I was debating whether to share all of the source code. I may share a portion of it soon.

[ritchie46]

Great! A small correction on your post. Polars does have SQL suppor. It isn't the main usecase, so it isn't as good as that of Spark and DuckDB, but it does exist and is being improved on.

[mwc360]

Ritchie - thx for graciously correcting some things I got wrong, will get it corrected!