Hacker News new | ask | show | jobs
by wtallis 99 days ago
You're seriously posting to HN a link to your Slashdot post linking to your year-old blog post complaining about Geekbench 6's multi-threaded test without ever mentioning Amdahl's Law?

Pretending that everything a CPU does is an embarrassingly parallel problem is heinous benchmarking malpractice. Yes, Geekbench 6 has its flaws, and limitations. All benchmarks do. Geekbench 6 has valid uses, and its limitations are defensible in the context of using it to measure what it is intended to measure. The scalability limitations it illustrates are real problems that affect real workloads and use cases. Calling it "broken" because it doesn't produce the kind of scores a marketing department would want to see from a 96-core CPU reflects more poorly on you than it does on Geekbench 6.

3 comments

Possibly the underlying reason for the indirection is because that particular domain is banned on HN; check https://news.ycombinator.com/from?site=dev.to with showdead turned on.
Yeah, that's the main reason. Not sure why the ban, medium etc are much worse...

The second reason is I've never had a slashdot submission accepted and I saw that after posting this the page suggested I share it to increase chances of the editors picking it up, but I don't really use social media so I though hey why not HN.

Amdahl’s Law is descriptive, not exculpatory.

It explains why a workload with a large serial/contended fraction won’t scale.

It does not prove that the workload’s serial fraction is representative of the category it claims to stand in for.

So when a benchmark’s “text processing” test over ~190 files barely gets past ~1.3x on 8 cores, that’s not some profound demonstration that CPUs can’t parallelize text work. It’s mostly a demonstration that this benchmark’s implementation has a very large serial bottleneck.

That would be fine if people treated GB6 multicore as a narrow benchmark of specific shared-task client workloads. The problem is that it is labelled as a general multicore CPU metric, and is used as such, including for 18-core vs 96-core comparisons. That’s the misuse being criticized.

TL;DR: Amdahl’s Law explains the ceiling; it does not justify treating an avoidably low ceiling as a general measure of multicore CPU capability.

EDIT: Also, submitter, I'm not sure why parent is upset that you submitted. Thanks for sharing. I've been wondering for years why GeekBench was obviously broken on multicore. (comes up a lot in Apple discussions, as you know)

From your post I can tell you did not read the "year-old blog post". It starts with the scaling, but goes further and explains a lot. I am a software engineer specializing in algorithms and optimization, Amdahl's law is part of the usual training I give our junior developers. It has nothing to do with Geekbench 6 being a surprisingly bad benchmark, especially for big CPUs.
If you’re in the market for a 96 core cpu and you’re using Geekbench to guide your purchasing decision, the fault lies with you.