[wtallis]

Please stop framing the multi core scaling issue as a matter of the benchmark being good or bad. Geekbench 5 scores scale better than Geekbench 6 scores because Geekbench 5's multicore test runs N independent copies of the same workload while Geekbench 6 runs one workload that has to be split across the available cores, with non-zero coordination between threads.

The Geekbench 5 approach of pretending Amdahl's Law doesn't exist is sometimes a valid benchmarking strategy, but generally is the wrong choice for benchmarking consumer workloads and devices, and that's what Geekbench is ostensibly targeting.

The fact that Geekbench 6 scores don't increase linearly with the addition of more CPU cores is not a weakness of the benchmark, it's the benchmark demonstrating an important real-world effect.

The change that Geekbench 7 makes to exclude some subtests from the multicore suite entirely will definitely have the effect of making the overall multicore score scale better with the addition of more cores, but most of the audience for those scores is going to miss out on the fact that the multicore test now measures a narrower range of tasks than the single-core test suite.

[osti]

I agree. For me personally I mostly only care about single thread geekbench variant, I believe it's an excellent proxy for general performance of a CPU. Multi thread geekbench (or other benchmarks) for most purposes and for most people, it's kinda useless. You just need to know that you have a quite a few cores on your computer and that it will have enough concurrency for what you do. But single thread will make whatever you do actually faster.

[adrian_b]

There are plenty of things for which the results of single-threaded benchmarks are completely irrelevant, like the compilation of a big software project.

There are also things that are well-correlated with the single-threaded Geekbench result, like how responsive will appear an Internet browser, but for many professional uses the multi-threaded performance is much more important than the single-threaded performance.

For many CPUs, and depending on the computers in which they are installed, the single-threaded and multi-threaded results are completely uncorrelated, because during the multi-threaded benchmark the clock frequency in some computer may drop even 4 times (there are Intel CPUs with a so big ratio between the maximum turbo frequency and the base frequency), while in another computer it might drop only by 20%.

[jchw]

Hmm, I don't fully agree. There are a whole lot of embarassingly parallel processes that people interested in many core workstations are interested in; compiling large C++ codebases for example tends to scale very well since each translation unit is independent. This is still going to not quite give you linear scaling so it's definitely still valuable to know what that looks like.

[wtallis]

Geekbench is not intended for benchmarking large workstations. Look at how the web site actually describes the benchmark. It's for phones and laptops and similar devices. A test mode that uses a machine in a way that is only relevant to high-end workstation or server use would be actively harmful to the suitability of Geekbench for its intended audience of consumer hardware and consumer users.

[aurareturn]

No idea why you're downvoted. Geekbench's focus is consumer workloads. In consumer workloads, having 20 cores or more is nearly useless. Most normal people will not need more than 8 in 2026. You're better off with a few fast cores over many slow cores.

Geekbench isn't a server CPU benchmark that stresses 128 core CPUs.

[jchw]

I dunno why people are downvoting at all. I don't get what nerve this is striking so hard.

The real truth is that if consumers don't benefit from knowing how well multicore workloads scale, then they can just look only at the single-core score. Skewing it on purpose by including things that genuinely only use ~1 core doesn't seem to serve to add value but only obfuscate the truth about how well a CPU is able to scale on multicore workloads, which does matter sometimes (like with modern video games.)

Yes this means that 128 core Xeon servers will maybe look especially good, but idk, what consumer hardware has 128 cores? Consumers are not considering a Mac Mini versus a fucking NVIDIA Vera rack, you know?

I fully accept that you could view it either way, but if you're going to have a multicore score at all, in my opinion, it may as well actually show you what multicore workloads look like. It is OK that consumers can't necessarily understand how this translates. If all they ever do is single core workloads, they will never have any benefit to looking at the multicore workload score for any Geekbench score on any Geekbench version, because otherwise the multicore score overstates the performance, whereas if they were to look at a version that has multicore including ~single threaded workloads, it would understate the multicore scaling.

Benchmarks are necessarily imperfect no matter how you shake it. For example, it is impossible for a general compute benchmark to show how much better the UX of two cores is over having just one core that is a bit faster.

Though, now that I say all of this, I get the strange feeling the nerve it's striking is fanboyism. What else would make people care this much, but perhaps the hardware vendor they are fans of showing up higher in the numbers? Well it ain't me. I couldn't care less how Apple and AMD and Intel compare in the Geekbench scores. Could be wrong, but somehow this just feels like a good bet.

[aurareturn]

Given that Geekbench is a consumer workload benchmark, it makes sense to factor in Amdahl's Law. Period.

Now the question is whether having a 3rd CPU score for embarrassingly parallel work makes sense for Geekbench. It probably does since many professionals also use Geekbench nowadays.

[adrian_b]

Amdahl's law is one thing, and an inefficient implementation of a concurrent program is another thing.

We do not know how much of the bad scaling of Geekbench 6 was due to Amdahl's law and how much was caused by an inept implementation.

Moreover, Amdahl's law is really important only for very short computational tasks, i.e. which do not need more than a few seconds or minutes for completion.

The bigger is the job that must be accomplished, the more likely it is that it can be reorganized in a way that tends to be embarassingly parallel, so that Amdahl's law has little importance.

So for jobs as big that they need hours, days or weeks to be completed by a computer, if they are well planned it is frequent to reach scaling factors not much lower than the number of available cores, even if that may be much more difficult to achieve for short jobs.

[wmf]

If they had an embarrassingly parallel test, Intel would be exclusively promoting that misleading number just like they do with Cinebench. Let's not create even more totally unrepresentative benchmarks.

[jchw]

> Given that Geekbench is a consumer workload benchmark, it makes sense to factor in Amdahl's Law. Period.

To be honest, they are factoring in Amdahl's law, because they're actually not only including embarrassingly parallel workloads, they're just not including ones where the speedup factor is roughly 0. Frankly I should have acknowledged this in my initial response.

> Now the question is whether having a 3rd CPU score for embarrassingly parallel work makes sense for Geekbench. It probably does since many professionals also use Geekbench nowadays.

Including workloads that are almost entirely single-threaded does not tell you anything about the "multi-core" performance of the processor. It just biases the multi-core score to be lower, period.

It is true that even without that, the multi-core score will still be biased towards the rough average speedup factor of all of the workloads combined, but that's just the nature of an aggregate benchmark. If you wanted to know how it would perform on a specific task, you'd need a specific benchmark anyways. If they've done a good job, the multi-core scores should still be useful for comparison across processors for your average multi-core workload.

I think two things are true:

- If consumers almost only ever use single threaded workloads, then the multi-core score will always be misleading to them if it is anything above what the single core score is. They should be paying more attention to the single-core score.

- If consumers do have multi-threaded workloads, then a multi-core score that averages in a bunch of tasks with no multi-core speedup will also not be representative for them, as it will understate the performance benefits they can see.

If you wanted a more useful third number it would be a weighted overall score, but what is the objectively correct weight between single and multi-core? Even for consumers this is a challenging number to come up with.

But I know one thing: it is almost certainly not whatever weight you just happen to wind up with by just arbitrarily including whatever number of single-threaded tasks your benchmark happens to have in a multi-core benchmark score.

[jchw]

Well, apparently they must not agree.

[sroussey]

In that case, just assume linear scaling? You only need pay attention to a single core.

[osti]

You can use geekbench 5 in that case. But given that they deprecated that, it might be harder to compare to others.

[GeekyBear]

They do mention that the change is meant to reflect how real world programs handle various types of workloads.

> In Geekbench 7, a workload only runs in multi-threaded mode if the task it models actually runs multi-threaded in real applications. For example, the HTML5 Browser test isn’t included in the multi-threaded suite because web browsers are single-threaded (or lightly threaded).

[wtallis]

Yes, they have a valid reason for the change. But I think it's still making a tradeoff, and has a clearly identifiable downside because the content of the multi-core test suite is no longer as closely aligned with the content of the single-core test suite.

(One could just as easily argue that the single-core test suite ought to exclude any task typically done with multiple threads in real-world usage.)

[zamadatix]

It bothers me because there is no reason it can't just give 3 results: single, mixed/low thread, all core and yet there is often debate on which 2 numbers in such benchmarks instead.

But, of all of the 2-pick-only options, choosing a custom mix and calling it multi-core requires guessing your actual workload on both the content of the mixed test suite as well as the scaling profile of the CPU to reason with. On the other hand, just testing "single core" and "all core" at least only requires you to guess based on what you think the scaling profile of the CPU is.

Nothing beats just testing your actual workload, but that doesn't mean all other ways of testing have to be equally good.

[wtallis]

> But, of all of the 2-pick-only options, choosing a custom mix and calling it multi-core requires guessing your actual workload on both the content of the mixed test suite as well as the scaling profile of the CPU to reason with.

Yes, that's pretty much the whole point of Geekbench: to offer non-experts a way to measure CPU performance and produce scores that are relevant to a specific class of devices and users and use cases, rather than artificially inflated.

Benchmarking experts can already use something like SPEC, which does offer single-thread and both types of multi-thread test modes (SPECrate and SPECspeed).

[zamadatix]

It has nothing to do with whether it's an expert benchmark or not, even the dumpster fire that is UserBenchmark manages to show the three categories separately. It's just a bad practice among any kind of benchmark, basic or not. After all, the Geekbench is already doing the work needed during the tests.

[wtallis]

Geekbench is not just intended to be used to benchmark consumer systems, it is intended to be used by consumers to benchmark their systems and to have its scores interpreted by consumers. For that audience, the most important thing that a multi-core score needs to do is counteract the naive assumption that a 20-core CPU will be twice as fast as a 10-core CPU. Having one of the top-line scores be a perfectly-scaling multi-core score that you know will be widely misinterpreted by your audience and used by vendors to mislead consumers would be serious benchmarking malpractice.

[zamadatix]

You start by saying it's used by the users and then end by saying it'd be abused by the vendors. You can't play both sides as reasons at the same time.

Besides, UserBenchmark is clearly intended to be used by users as well. It's right in the name. There has not been confusion with it, if anything labeling the scores as what they are instead of hiding it under how you want a user to see it is both clearer and avoiding benchmark malpractice - not the other way around.

[adrian_b]

While it would be nice to have a multi-threaded benchmark that demonstrates a realistic scaling towards many threads, I have no reason whatsoever to trust that the Geekbench implementers are capable to optimize the inter-thread communication well enough for reaching the limits imposed by the hardware.

If they use inefficient thread synchronization methods, like those offered by the C++ compiler, or by the pthreads library on Linux, or by some of the older Win32 functions, their results will be much too pessimistic for the CPUs with many cores, creating the wrong impression that they are not much better, or not better at all, than CPUs with less cores.

For Geekbench 6, I have seen enough results where CPUs that are much better for real multi-threaded workloads appeared to be almost the same as clearly inferior CPUs.

For multi-threaded benchmarks, if they are embarassingly parallel the quality of the implementation does not influence the results, but otherwise it is not possible to trust the results without having access not only to the source program of the benchmark, but also to the compilers used by it.

I have seen a lot of benchmarks of locking methods used for thread synchronization, where the results could not be interpreted correctly without seeing the assembly language instructions generated by the compiler for the locking sequences, because a bad assembly generation could matter more than the high-level algorithms whom the benchmark was supposed to compare.

[wtallis]

> While it would be nice to have a multi-threaded benchmark that demonstrates a realistic scaling towards many threads, I have no reason whatsoever to trust that the Geekbench implementers are capable to optimize the inter-thread communication well enough for reaching the limits imposed by the hardware.

You're contradicting yourself. Do you want realistic scaling, or the kind of highly-tuned hand-optimized scaling that almost no real-world applications (especially in the consumer space) ever get?

In both of your comments here, you make very narrow assumptions about what constitutes a "real" workload, and you're excluding ordinary everyday consumer PC software from what you consider "real". Highly optimized software running on large servers aren't the only "real" workloads, and they're not the kind of "real" workload that Geekbench is trying to represent. For the kind of hardware and use cases that Geekbench actually targets, using the platform/compiler default synchronization primitives and doing work in short bursts interspersed with significant idle time is reasonable and realistic.

[adrian_b]

A very frequent real-world workload is the compilation of a software project, which scales almost perfectly with the number of cores.

For the other multi-threaded tasks, like running certain CAD/EDA applications, I use well-optimized programs, either by others or by myself.

What I do is typical for a professional use of a computer.

The Geekbench multi-threaded results do not provide any useful information about how a computer will behave for such applications.

I agree that the Geekbench multi-threaded results may be representative for how a smartphone will behave if it will have small bursts of multi-threaded activity caused by casually written applications, interspersed between single-threaded execution.

This would be fine, if it would be more clearly specified when the Geekbench benchmark is distributed.

I know that I should trust only the single-threaded Geekbench results and for multi-threaded I should search other published benchmarks.

But others appear to not be aware of this and I have seen frequently erroneous claims on the Internet that some laptop or even some smartphone would match or exceed the performance of a desktop or of a bigger laptop, even one with a CPU with more cores, based on the published Geekbench multi-threaded results, while in reality that desktop or big laptop has a much greater throughput.

[wtallis]

> A very frequent real-world workload is the compilation of a software project, which scales almost perfectly with the number of cores.

Geekbench has that as a subtest.

> For the other multi-threaded tasks, like running certain CAD/EDA applications, I use well-optimized programs, either by others or by myself.

Those seem pretty clearly out of scope. You want a workstation benchmark, not a consumer benchmark.

> This would be fine, if it would be more clearly specified when the Geekbench benchmark is distributed.

The descriptions of what Geekbench is are adequate. You're asking for Geekbench to exhaustively list what it isn't to stop you from imagining it to be a one-size-fits-all benchmark, which couldn't possibly exist.

> But others appear to not be aware of this and I have seen frequently erroneous claims on the Internet that some laptop or even some smartphone would match or exceed the performance of a desktop or of a bigger laptop, even one with a CPU with more cores, based on the published Geekbench multi-threaded results, while in reality that desktop or big laptop has a much greater throughput.

Again you seem to be unable to stop asserting that workstation or server use cases are the only ones that are real.

[wmf]

Honestly I think the world would be better off if Geekbench just refused to run on any machine over 28C. Then people could benchmark their workstations using... workstation benchmarks.