[jdcryans]

My thoughts exactly, this is useless. Benchmarking is something everyone can do, so people do it, but it doesn't provide any real value unless you put an insane amount of time into it, and even then you'll get a lot of hate mail :)

For example, look at the "Yahoo! Cloud System Benchmark" AKA YCSB paper. Those researchers spent a lot of time designing the benchmarks and properly configuring each database that they were testing. They even dedicated a researcher per database to sit down with the developers to review the configurations and the test runs. I was part of this process for the Apache HBase as a dev. In the end, everyone was still critical of the results once they saw the graphs comparing the DBs.

But, I'm still glad they did it as I use YCSB as one of my tools to benchmark HBase.