Why are they using MIPS as a metric for comparing integer CPU performance? When you're looking at two different compiler backends targeting two (very) different ISAs, it seems pretty thoroughly uninformative.
In benchmarks MIPS usually means "we arbitrarily say that reference machine X runs this benchmark at Y MIPS". Then, for a different machine Z, Z_MIPS = Y * (X_TIME / Z_TIME).
For example, the 7zip benchmark used in this article is "normalized with results of Intel Core 2 CPU" [1], or the famous Dhrystone integer benchmark's "VAX MIPS" [2].
Still I agree it's weird terminology. It would make more sense to use the true units of the benchmark, or just say "performance relative to reference machine".
Yep, with you 100%. Anandtech (especially Anand himself) usually does pretty rigorous benchmarks but this is either a poorly executed comparison or intentionally biased towards POWER8 (and I say this as POWER fan and former IBMer). I'm really let down by this review.
Another HNer went into detail as to why the comparison is flawed but basically when you compile with a compiler made by the chip designer, especially for numerical computation, the performance is almost definitely going to slant towards that direction. GCC and Clang have to focus on dozens and dozens of backends. Anandtech has more than enough money to spend 1k for an ICC license.
For example, the 7zip benchmark used in this article is "normalized with results of Intel Core 2 CPU" [1], or the famous Dhrystone integer benchmark's "VAX MIPS" [2].
Still I agree it's weird terminology. It would make more sense to use the true units of the benchmark, or just say "performance relative to reference machine".
[1] http://www.7-cpu.com/
[2] https://en.wikipedia.org/wiki/Dhrystone