Isn't the better abstraction here to use a higher level library in the style of pandas/polars that will operate as vectors, compose and feel readable and inuitive, while (almost?) maxing out SIMD?
That can easily cause you to traverse your data several times when once would be enough. Let’s say you wanted to compute “mean of array divided by max in absolute value”.
NumPy-like:
mean = np.mean(array)
maximum = np.max(np.abs(array)
return mean / maximum
As far as I’m aware, that will be evaluated as three traversals.
Highway:
HWY_FULL(float) d;
using V = decltype(hn::Zero(d));
V sum = hn::Zero(d);
V max = hn::Zero(d);
hn::Foreach(d, values, N, hn::Zero(d), [&](auto d, auto v) HWY_ATTR {
sum = hn::Add(sum, v);
max = hn::Max(max, hn::Abs(v));
});
const float mean = hn::ReduceSum(d, sum) / N;
return mean / hn::ReduceMax(d, max);
Just one pass.
When the actual computation is made so much faster by SIMD, memory bandwidth starts to be a significant bottleneck.
Thanks, that's insightful. I haven't thought about it that way but in retrospect it's clear. In practice I think we've all seen that the numpy style construction is quick to write and performs (and reads) much better than "dumb loops", but if you really want to optimize your code, the highway version will give you more control (and will read similar to the dumb loop with some decorations). I guess just more tools in the toolbox!
NumPy-like:
As far as I’m aware, that will be evaluated as three traversals.Highway:
Just one pass.When the actual computation is made so much faster by SIMD, memory bandwidth starts to be a significant bottleneck.