[tjhunter]

This article has valid points but does not understand the perspective of companies. Companies do not buy technologies. Companies buy solutions.

- Companies do not buy Spark, they buy the ability to process their data and to have multiple personas collaborate (data scientists, data engineers, ...)

- You can do it yourself. It will be cheaper but it will require time, expertise and money, all things that companies do not give easily

- Snowflake and Databricks are elastic: you can start small and grow as you need. This is much easier than justifying the upfront cost of hiring specialized people or asking for trust that your ad-hoc solution will respect whatever enterprise governance rules

(disclaimer: I worked at Databricks for 6 years and talked to hundreds of prospect and actual Databricks users and customers)