[PeterZaitsev]

This response illustrates important point - if you're expert in technology A and compare it to technology B, you're not expert in, comparison is very likely to be unfair.

I very much would like to see vendors at least to follow Journalist ethics and reach out to their competition for optimization comments and suggestions before publishing it, so others are given a chance to suggest optimizations

[DeathArrow]

>This response illustrates important point - if you're expert in technology A and compare it to technology B, you're not expert in, comparison is very likely to be unfair.

To have fair benchmarks, one organization can manage the test requirements and submissions while expert in a said technology can submit their best efforts to be measured and compared to others.

Like Tech Empower Benchmarks used to work.

[hodgesrm]

Agree. Or just load test on your own software, publish how you did it, and let other vendors respond for themselves.

[klysm]

Yeah this happens a lot. I like it when people maintain a repo that accepts changes for the comparison

[snikolaev]

Then you should like https://db-benchmarks.com/

[PeterZaitsev]

Great idea.