[buttaphingas]

I've used Snowflake for the past few years, and it's worth pointing out that when it comes to overall cost, there's a lot you get with Snowflake for free. For example, they have HA across 3 AZs out of the box, included in the price and with no configuration required.

If I'm reading what Databricks published correctly, it seems that they've only used 1 driver node for this benchmark, in other words it's a dev setup. If they want to compare apples-to-apples then they should configure, and price, a multi-AZ HA set-up.

I'm not sure if this is still applicable to Photon, however - can anyone confirm?

[sagarm]

The _data_ should be replicated, but the compute infrastructure doesn't need to be. Many companies I suspect would be fine having to restart pipelines on driver failure (increasing tail latency, basically) if it yields a substantial cost reduction.