|
|
|
|
|
|
by saj1th
1678 days ago
|
|
|
> Why would Spark trounce Snowflake. What makes it inherently so much faster at 100TB jobs? These are the slides from a talk one of the co-founders (@rxin) gave at Stanford.
https://web.stanford.edu/class/cs245/slides/LakehouseGuestTa... It goes into the details of how this performance is achieved(and not just at 100TB). Part of this could be attributed to innovations in the storage layer(delta lake), and part of it is just the new query engine design itself. |
|
|