[ljhsiung]

Even putting aside security aspects aside, in general I've been seeing research pop up over the years criticizing SMT's performance claims of ~30%.

Hell, even Amazon's Graviton CPUs don't have it (though I'm sure that's a product of being ARM derived rather than a design decision).

[tux3]

ARM vendors must be feeling pretty good about themselves yeah, but if you take AMD's cores... SMT might not be a huge win in every benchmark, but you just can't keep that wide backend fed from a single hyperthread (at least I can't!).

So turning SMT off is at the least wasted potential for those cores, the way they've been designed

[wmf]

Apple Firestorm is even wider but it doesn't have SMT. I guess they just don't care.

[marcan_42]

Probably because they have an 8-wide decoder and a massive reorder buffer, so they can actually keep the backend fed.

The problem with x86 is decoding is hell and requires increasingly large transistor counts to parallelize, so you end up with a bottleneck there. ARM doesn't have that problem.

[sitkack]

This the single largest driver of M1 performance.

Variable length, over lapping instructions has made x86 instruction decoding intractable. The obvious answer is make it tractable, the unobvious answer is how to do that and hopefully remain backward compatible.

[hajile]

The performance claims are true for all the worst reasons.

Let's say you can queue up 100 instructions. This yields the following

    1 port 100% of the time
    2 ports 60% of the time
    3 ports 30% of the time
    4 ports 10% of the time
    5 ports 2% of the time

Increasing the buffer to 200 instructions yields the following

    2 ports 80% of the time
    3 ports 40% of the time
    4 ports 15% of the time
    5 ports 4% of the time

As in that made-up example, doubling the window you can inspect doesn't double performance. You really want those extra ports because they offer a few percentage IPC uptick, but the cost is too high. So you keep increasing the window size until the extra ports become viable. As an aside, AMD Caymen switched from VLIW5 to VLIW4 because the fifth port was mostly unused. A few applications suffered from the slightly lower theoretical performance, but using that space for more VLIW 4 units (along with other changes) meant that for most things the overall performance went up.

Now comes the x86 fly in the ointment -- the decoders width gives rapidly diminishing returns (I believe an AMD exec mentioned 4 was the hard limit to keep power consumption under control). This limits the size of the reorder buffer that you can keep queued up. Since you have a maximum instruction window size, you have a hard port limit.

So you add a second thread. Sure, it requires it's own entire frontend and register sets, but in exchange you get a ton more opportunities to use those other ports. There are tradeoffs with the complexity and extra units required for SMT, but that's beyond our scope.

As you can see, SMT performance is DIRECTLY related to how inefficiently the main thread can use the resources. In less interdependent code, SMT performance increases are worse because finding uses for those extra ports on the main thread is easier.

Now, let's consider the M1 and one reason why it doesn't have SMT. Going 5, 6, or even 8-wide on the decoders is trivial compared to x86. Apple's M1 (and even the upcoming V1 or N2) have wider decode. This in turn feed a much larger buffer which can in turn extract more parallelism from the thread (this seems to be taking about as many transistors as the extra frontend stuff to implement SMT). Because they can keep most of their ports fed with just one thread, there's no need for the complexity of SMT.

IBM POWER does show a different side of SMT though. They go with 8-way SMT. This isn't because they have that many ports. It's so they can hide latency in their supercomputers. It's kind of like MIMT (multiple instruction, multiple thread) in modern GPUs, but even more flexible. They help to ensure that even when other threads waiting for data that there's still another thread that can be executing.

[feffe]

The memory latency hiding also works with 2-way SMT. I worked on a networking software doing per packet session lookup in large hash tables. SMT with a Sandybridge core in this application gave 40% better performance which is higher than usually mentioned. So for memory bound (as in cache misses) applications, SMT is a boon.

[injinj]

I have a graph for this:

https://github.com/raitechnology/raikv/blob/master/graph/mt_...

The CPU in this case is a Threadripper 3970x, 32 cores, 64 SMT.

My experience is this: When the L3 cache is effective, then the memory latency hiding via memory prefetch works well across SMT threads. If the hashtable load requires a chain walk, the SMT latency hiding is less effective because the calculated prefetch location is not the actual hit. I couldn't get prefetching multiple slots as the load increased to be as effective as prefetching a single slot.

[magicalhippo]

I tested this some years ago on a raytracer, and got a tad over 50% more speed when enabling HT compared to disabling it.

As you say, the ray tracer did a lot of cache missing , interspersed with a fair bit of calculations. I'm guessing this is close to the ideal workload, as far as non-synthetic benchmarks go.

[gpderetta]

The reorder buffer size is practically limited by branch prediction probability, not decode bandwidth though.

[jcelerier]

When doing audio processing I'm getting ~20/25% more oomph with HT enabled