Polars has been my go-to for six months now. The performance gains for what I do are pretty great, and the ability to write out Parquet with minimal hassle is just dandy.
If it weren’t for resume-driven architectures, it would make a great alternative to Spark for high volume bronze-to-silver cleanup and normalization steps.
It's interesting that they are explicitly not committing to holding the API constant (which seems quite common & expect for a "1.0"). I generally like that philosophy, but I wonder if it will lead to some unexpected whiplash in the future for people now expecting the core to be stable.
It's a bit unclear in the docs, but I think what they're saying is that if there's a need for a breaking change, they're "not afraid to release version 2.0", rather than saying they're "not afraid to make a breaking 1.x release".
Yes only found the announcement [1] that the Polars team and NVIDIA engineers are working on a GPU engine, but other than that no concrete examples. Github issues also don't provide any hints on the status, only one open item [2] where most comments are prior to the announcement.
If it weren’t for resume-driven architectures, it would make a great alternative to Spark for high volume bronze-to-silver cleanup and normalization steps.