Hacker News new | ask | show | jobs
by georgewfraser 2840 days ago
We've gotten a lot of passionate feedback in this category, and we actually did a comparison with sort and dist keys in the extended version. They don't make as much difference as people think, these charts would be ~15% shifted with sort and dist keys.
2 comments
vgt 2840 days ago
Did you guys implement cluster keys in BigQuery as well?

(BQ PM)

link
georgewfraser 2840 days ago
No we just did a comparison between Redshift with no tuning, conservative tuning, aggressive tuning. The point was just to assess "how much does tuning matter anyway?"

You are correct that if we wanted to do a full-on "tuned comparison", we would need to implement partitioned tables in BQ, Snowflake and Presto.

link
vgt 2840 days ago
(and clustering in BQ! [0])

[0] https://cloud.google.com/bigquery/docs/clustered-tables

link
maldeh 2840 days ago
That actually sounds like a solid improvement. Do the head-to-head comparisons change with sort- and dist-keys? What was the tuning used?
link
georgewfraser 2840 days ago
You can see the whole back-and-forth here:

https://github.com/fivetran/benchmark/issues/1

The tuning was pretty aggressive, some of the sort keys were a little unrealistic for a real-world scenario where you're doing a lot of different queries.

link