[yunnpp]

You need to let the compiler know that there are at least 4 or 8 elements to process. This may require padding data and/or having a second loop after the main one that processes the remainder <4 or <8 elements.

You start the post with:

> There is an opportunity to use SIMD. SIMD turns those into this: > > for (8 byte chunk in bytes) { /* ... */ }

If you actually wrote that loop, there is a good chance the compiler (gcc specifically) will auto-vectorize.

In any case, the more manual SIMD optimizations I have seen require reworking the data altogether, not just processing N elements at a time. For example, instead of packing two 4-vectors into two registers to do a dot product, pack the XXXXs, YYYYs, etc. into 4 vectors and compute 4 dot products for the price of one. That not only requires having 4 vectors to process, but also thinking how exactly they are packed in registers.

I don't know why qurren is downvoted. You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.

[spider-mario]

> You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.

Counterpoint: https://pharr.org/matt/blog/2018/04/18/ispc-origins

> I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model.

> The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit.

> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.

> With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.

[yunnpp]

How is that different from any other compiler optimization?

And what does the last statement in the quote mean anyway? When is performance "predictable"; do you freeze the entire toolchain?

And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?

If you're writing C++, you're already rolling on decades of compiler optimization. You return by value because it makes code more readable and safer and rely on RVO. You write functions to abstract and rely on the compiler inlining. When it doesn't work for your specific target/toolchain, you may decide to handroll stuff. The proof that you're banking on the compiler is that if you run a debug build of any non-trivial program, it runs like absolute dogshit.

[spider-mario]

> How is that different from any other compiler optimization?

When it can be single-handedly responsible for an 8× speedup, you might not want to rely as much on the compiler as in the case of smaller, cumulative optimisations.

> And what is the alternative? Handroll manual SIMD code for every possible architecture you may target?

Use a library like Highway? https://github.com/google/highway

[spider-mario]

Forgot to add: using Highway also means that you can detect and take advantage of e.g. AVX2 or AVX-512 in one binary that still works on CPUs without those instruction sets, whereas relying on autovectorisation would mean having to build with `-mavx2`, `-mavx512`, etc. which means that pretty much no one who relies on prebuilt binaries would get them.