[jstephan]

Thanks for the feedback! It's possible to run Spark on Kubernetes using just OS tools - in fact our platform builds upon and contributes to many of these tools. But it's not easy enough, in our humble opinion, you need to build a decent level of expertise on Spark and k8s just to get started, and even more to keep it operational/stable/cost-efficient/secure in the long term.

Regarding costs. By autoscaling the cluster size and minimising our service footprint, the fixed cost for using our platform is around $100/month, which is negligible compared to the cost of most big data projects. We have some ideas on how to drive this fixed cost to zero, and offer a free hosted version of our platform too. It's in the roadmap!