[falaki]

To be fair Apache Spark, which started long before either company existed, was built on the assumption that compute and storage should be separate. Unlike Hadoop, Spark did not come with any storage system and could read from any source.

[d-d-d]

> To be fair Apache Spark, which started long before either company existed

Databricks was founded before Spark 1.0 released by Spark's creators.

Hadoop was created at a time when network and disk were much slower, RAM was less abundant. Bringing compute to the data made sense, but it typically doesn't anymore.

[doppelganger1]

Hadoop was built on the notion that commodity hardware, when pooled together, can be extremely cheap and powerful. The problem is, to manage it, is a nightmare. Cloudera/HWX and others were unable to reduce the management burden and their inability to pivot to a cloud based architecture really sunk their ship.