In the article, Mitchel mentions how this doesn’t always work. In fact, as someone who’s worked in compiler development, I can say it’s a small miracle when it does work.
Case-in-point, the example in my own post doesn't auto-vectorize with LLVM or GCC at highest optimization levels. Basically, compilers will never auto-vectorize loops with an early loop break afaik.
You would have to give the compiler some help to allow it to auto-vectorize this. Warning, untested: https://godbolt.org/z/b358bMWzG. This unrolls the loop by 16 times. The trick is using `&` instead of `and` so that there's no short-circuiting. All 16 elements are read on each iteration of the loop. This gives the compiler the freedom to replace these reads with a single 16 byte load.
> compilers will never auto-vectorize loops with an early loop break afaik.
You need to let the compiler know that there are at least 4 or 8 elements to process. This may require padding data and/or having a second loop after the main one that processes the remainder <4 or <8 elements.
You start the post with:
> There is an opportunity to use SIMD. SIMD turns those into this:
>
> for (8 byte chunk in bytes) { /* ... */ }
If you actually wrote that loop, there is a good chance the compiler (gcc specifically) will auto-vectorize.
In any case, the more manual SIMD optimizations I have seen require reworking the data altogether, not just processing N elements at a time. For example, instead of packing two 4-vectors into two registers to do a dot product, pack the XXXXs, YYYYs, etc. into 4 vectors and compute 4 dot products for the price of one. That not only requires having 4 vectors to process, but also thinking how exactly they are packed in registers.
I don't know why qurren is downvoted. You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.
> You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.
> I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model.
> The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit.
> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.
> With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.
How is that different from any other compiler optimization?
And what does the last statement in the quote mean anyway? When is performance "predictable"; do you freeze the entire toolchain?
And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?
If you're writing C++, you're already rolling on decades of compiler optimization. You return by value because it makes code more readable and safer and rely on RVO. You write functions to abstract and rely on the compiler inlining. When it doesn't work for your specific target/toolchain, you may decide to handroll stuff. The proof that you're banking on the compiler is that if you run a debug build of any non-trivial program, it runs like absolute dogshit.
> How is that different from any other compiler optimization?
When it can be single-handedly responsible for an 8× speedup, you might not want to rely as much on the compiler as in the case of smaller, cumulative optimisations.
> And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?
Most scalar-to-SIMD conversion requires changing the design of data structures and algorithms to be effective. Compilers are required to exactly reproduce the specified data structures in a deterministic way for obvious reasons.
Even if compilers were clever enough to transform your data structures and algorithms for SIMD (they're not), the data structures are a contract that can't be unilaterally modified.
And -march=native or at least -march=x86-64-v3 or similar, alternatively identifying relevant functions and manually invoking FMV and uarch specialization via target_clones. Plus non-integer code can generally not be autovectorized in normal-math mode since FP is non-commutative.
A good introduction to SoA (for anyone curious) are the two most famous Data-Oriented Design talks by Mike Acton (game engine dev) [1] and Andrew Kelley (Zig lead dev) [2] respectively.
I read a book book about DoD [3] really which confused me at first with all its talk about database table design (in a book about a high-performance C++ game engine?), but when it finally clicked it was amazing. The point is that you want to think hard about your access patterns and what could constitute good "primary keys", then model it accordingly. SoA ends up being useful a lot of the time, because having your data in homogeneous arrays/vectors is great for cache locality and branch elimination. Even without SIMD you can get huge speedups from that, but that's also where your compiler (or you as a programmer) can get incredible SIMD gains.
SoA is not a silver bullet as it may not align well with your access patterns, but it can great to add to your toolkit.
I have no idea why you're being downvoted. HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers are really good at vectorizing on whatever hardware you're targeting, including SVE.
> HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
What if the “explicit acceleration library” for what you need to do doesn’t exist?
I mean, utter bullshit. If you "need SIMD" you know exactly the programming pattern to guarantee SIMD from the compiler. And the single and only people who "need SIMD" know these rules. It is only the hobbyist "SIMD is neat" community that upvotes these ridiculous articles.