[jmull]

Here's some important accompanying advice:

* If you aren't profiling, don't bother

Otherwise, you are going to be wasting plenty of time on "optimizations" that don't actually do anything useful.

Just come up with at least a few test that represent important cases and time them. Dig in, see where the time is being spent, and focus on areas where significant time is spent, especially ones that look ripe for optimization.

Also:

* Think about laying out your data in ways that accommodate your access patterns and are cache-friendly. SIMD would typically follow from this, not be a starting point on its own.