Sorry to nitpick (document seems solid), on page 32:
> Due to a TPC-internal error during the production of 3.2.0 of the TPC-DS kit, the benchmark execution had to use version 2.13 of the kit. It was confirmed by the TPC that the only changes between these two versions of the kit is the version number set in the tools/release.h parameter file.
How can there be that much of a delta of major/minor versions without a change? The only way that I see this happening is if 'change' being defined as the specific benchmark which was run, rather than the kit.
Yes, I wasn't saying they were lying about their tpc.org posted results. I'm saying both companies made use of clever indirection, wording, presentations of stats, etc. Like price/performance, and which of your competitor's tier's to select when doing that, and which of your own. Or over-provisioning the competition's setup, for example.
My guess is that they know the result won't look terrific. And they also know Snowflake works well in production for people despite that. So, little upside.
All leaders in a space take this approach. Little be gained, a fair bit to lose if you are ALREADY leading without having to debate / do a benchmark etc.
Anyways, the benchmark is only one part of the overall story for these solutions.
> Due to a TPC-internal error during the production of 3.2.0 of the TPC-DS kit, the benchmark execution had to use version 2.13 of the kit. It was confirmed by the TPC that the only changes between these two versions of the kit is the version number set in the tools/release.h parameter file.
How can there be that much of a delta of major/minor versions without a change? The only way that I see this happening is if 'change' being defined as the specific benchmark which was run, rather than the kit.