[goatlover]

The Polars code puts me off as being too verbose and requiring too many steps. I love the broadcasting ability that Pandas gets from Numpy. It's what sceintific computing should look like in my opinon. Maybe R, Julia or some array-based language does it a bit better than Numpy/Pandas, but it's certainly not like the Polars example.

[thijsn]

Polars is indeed more verbose when coming from pandas, but in my experience it is an advantage for when you're reading that same code after not having touched it for months.

pandas is write-optimized, so you can quickly and powerfully transform your data. Once you're used to it, it allows you to quickly get your work done. But figuring out what is happening in that code after returning to it a while later is a lot harder compared to Polars, which is more read-optimized. This read-optimized API coincidentally allows the engine to perform more optimizations because all implicit knowledge about data must be typed out instead of kept in your head.

[goatlover]

I don't agree that more verbose code is necessarily more readable when the shorter code looks like familiar math. All you have to do is learn how operators broadcast across array-like structures, how slicing and filtering works. Perhaps with more complicated examples the shorter code becomes harder to read after months away? Mathematicians are able to handle a lot of compact equations.

No doubt some of this comes down to preference as to what's considered readable. I never really bought that argument that regular expressions create more problems than they're worth. Perhaps I side on the expressivity end of the readability debate.

[thijsn]

Oh I don't mean to say verbose makes it more readable by default, I agree with you on that. I mostly meant that because the API is declarative (more geared at describing the result you want instead of the operations) it is easier to understand what's going on. A side effect of that is that it might be more verbose, which is the case of Polars vs pandas. In the end it's a personal thing which one you like the most. I do believe that if your deliverable is insights you get out of your analysis I can imagine that a less verbose API is practical to get things done quickly. But if you create pipelines that your colleagues have to quickly understand (or you in a couple of months) a read-optimized one makes more sense, even though it might take slightly more effort to write.

[thereisnospork]

Likewise, I was considering trying Polaris until I saw that example. The pandas example is a good approximation of how I think and want to transform/process data even if it is ugly under the hood. I do occasionally find numpy and pandas annoying wrt when the return a view vs a copy but the cure seems worse than the disease.