[flowerlad]

Python is a second class citizen in the world of Spark. Perf issues for UDFs. Some functions are only available through Scala. Python support for new features is always late. And so on. It is good to have .NET support but I will stick with Scala for the same reasons I switched from Python to Scala.

[pjmlp]

That is the exact reason that I learned to stay with platform languages for production code, even if there are more interesting ones trying to plug into it.

FFI, extra debugging layers and lesser tooling integration don't pay off a couple of language feature bullet points.