[globular-toast]

One thing to keep in mind is there's three layers of optimisation:

1. The problem, 2. The algorithms, 3. Micro-optimisation.

The potential gains shrink rapidly as you descend this list. A lot of people start thinking at level 3 straight away, but this is pointless if you've left performance on the table at the higher levels. For example, no amount of clever bit twiddling will compensate for the wrong algorithm, and even the best algorithm is pointless if you're solving the wrong problem.

[owlbite]

I'd add an addendum - look for stuff you don't need to do (e.g. at a low-level can you avoid that memcpy, zeroing things or multiple allocations when one would do, but even better at a high-level if you can avoid doing the whole thing altogether). It's better to just not do it rather than spends time trying to optimize it.