[Tanjreeve]

Ok you've got me there it's not 100% the exact same code Databricks are using there are some optimisations (that normally do end up downstream anyway). But I think it's getting a bit philosophical to say it's not open when you can run a delta lake "on-prem" and shuffle data between databricks and your own setup with few/no changes. Now Databricks SQL product afaik is not open and that's a proprietary C++ engine comparable to Snowflake so I think these discussions might get a lot more confusing in the future when databricks doesn't just mean various flavours of spark.

[imslowbutnice]

Yes Photon is completely proprietary. Databricks does have a "delta" version, but it is actually completely baked into the databricks runtime. So we are both correct. Ali (Databricks CEO) actually has gone on record to say Databricks is 90% proprietary code. There is an open source version, but it is not as good. The culture within Databricks though, is completely open source. Unlike Snowflake, the culture is definitely not open source. I think it affects the culture too.