Hacker News new | ask | show | jobs
by gauravphoenix 1468 days ago
I have always felt that DB benchmarks are useless, always use your own dataset

https://gauravkumar.blog/performance-benchmarks-are-useless....

4 comments

All benchmarks are always useless, in 90% of the cases. They could maybe give some baseline understanding, but it's important to always do your own benchmarks as your performance can be very different than what the benchmark showed, simply because the data/data structures are slightly different.

Do your own benchmarks people!

I wouldn't go as far as to say that benchmarks are useless, but I agree that when looking at a benchmark, it's important to be aware of how similar its data distribution and query patterns are compared to your own.
That has the confounding variable of how good you are at configuring each database option.

I can configure Postgres fairly well. I have little chance of knowing if I’m getting good performance out of most others without a serious time investment.

This is why the commercial offerings do not allow you publish benchmarks.
Which is horrible thing. Even bad benchmarks often create create discussions
Not true at all. Most people take benchmarks as gospel because they value their time.