[watmough]

This is really cool.

I just got through doing some work with vectorization.

On the simplest workload I have, splitting a 3 MByte text file into lines, writing a pointer to each string to an array, GCC will not vectorize the naive loop, though ICC might I guess.

With simple vectorization to AVX512 (64 unsigned chars in a vector), finding all the line breaks goes from 1.3 msec to 0.1 msec, so a little better than a 10x speedup, still just on the one core, which keeps things simple.

I was using Agner Fog's VCL 2, Apache licensed C++ Vector Class Library. It's super easy.