[stabbles]

For the particular case of the 5 delimiters '\n', '.', '?', '!', and ';', it just happens to be so that you can do this as a single shuffle instruction, replacing the explicit lookup table.

You can do this whenever `c & 0x0F` is unique for the set of characters you're looking for.

See https://stoppels.ch/2022/11/30/io-is-no-longer-the-bottlenec... for details.

[akoboldfrying]

This is a really neat technique, well explained at your link.

Now that I understand it, I'd describe it as: For each byte, based on its bottom 4 bits, map it to either the unique "target" value that you're looking for that has those bottom 4 bits, or if there is no such target value, to any value that is different from what it is right now. Then simply check whether each resulting byte is equal to its corresponding original byte!

Not sure if the above will help people understand it, but after you understand it, I think you'll agree with the above description :)

[bhavnicksm]

Hey! Author of the blog here.

This is pretty cool~ Thanks for suggesting this, I will read this in detail and add it to the next (0.5.0) release of memchunk.

[CyberDildonics]

Why does your title not have any context?

[dataflow]

Note your compiler might turn that _mm256_set_epi64x into a load from memory, so there might still be memory accesses you don't expect.