I feel that if you're one of the rare people who know how and when crafted assembly works better, it's almost always also a better use of your time to work on the compiler, rather than whatever you are working on.
Even years ago when the compilers were not as good this is exactly what most people did. Compile it look at the asm. Sometimes see better bits here and there and replace that function with an asm one. If you are in a space where that perf really maters you do this more often.
Even more difficult: for how long? - Compilers get better and CPUs get new instructions one might miss in the manual code. If you have your tight critical loop in assembler one has to regularly verify.