[CPlatypus]

I really wish they'd give more information about their methodology. For example, I've used the SoD SSD servers for some of my own testing, and they pull 30K IOPS for small random synchronous writes. How does that translate to "361.62"? WTF are the units here? What workload were they even testing? Yes, I know they list the random grab bag of tools they're using, but most of those are capable of generating many different kinds of I/O and they don't say what arguments they used. "361.62" seems very precise. I'm sure the two digits after the decimal point really impress the pithed snails or MBAs who are the benchmarkers' apparent target market. However, given both the bogosity of combining disparate measurements like this and the well known variability over time of cloud performance, that precision is not justified. Numbers that are more precise than they are accurate or meaningful are just decoration.

P.S. I expect someone will ask for more specifics, so here are a few. First, Bonnie++ sucks. Many of the numbers it produces measure the memory system or the libc implementation more than the actual I/O system. I've seriously gotten more testing value from building it than running it, so its very presence taints the result. Second, fio/hdparm/iozone might be redundant, according to which arguments are used. Or the results might be non-comparable. Either way, the aggregate result could only apply to an application with exactly that (unspecified) balance of read vs. write, sequential vs. random, sync vs. async, file counts and sizes, etc. Did they even run tests over enough data to get past caching effects? That's particularly important since they used different memory sizes on different providers. Similarly, what thread counts did they use on these different numbers of CPUs/cores? Same across all, or best-performing for each component benchmark? With such sloppy methodology anything less than an order-of-magnitude difference doesn't even tell you which platforms to test yourself.