[hilariously]

Yeah, the tl;dr is that benchmarking is freaking hard because what you actually care about is "does my workload today and in the future run better or worse given current setup?" but identifying what your workload actually is, what systems you are going to be allowed to run it on, what tweaks would even be possible if you know the interiors of a system and how it aligns with your hardware, and it all comes with the price tag of "and if you do anything different tomorrow with any of these variables it might not hold."

[fragmede]

Yeah, but also, I want to know the p50 warm performance, not just the p99. Run the same query twice in a row after cold start. And then another 10 times. Then do another different set of queries and at the end of the day, or a week, still have no real idea how the system will perform in prod for your particular use case.

Benchmarking is hard, no argument from me!

[hilariously]

Yep, I actually want to know the system has some sort of baseline performance that only hockey sticks under conditions I can monitor and control... but also the business wants to try new feature X and vendor is promising new performance for feature Y, and new patches are coming in affecting ???.

[fragmede]

And oh, you just have to set this one setting that you need to pay a consultant to know all the settings in order to tweak and tune the database for your use case. Still the TPC competitions such that oracle won because they managed to hand two assembly from five instructions down to four as an impressive bit of computer engineering nerditry.