[sceadu]

You might be able to do it with ibis. Don't know about the performance though

[abirch]

Thank you, I'll look into this. Yes performance is the main driver when some data frames have millions of rows.