Exactly this bothers me a lot. I do bare metal performance optimizations, but I worry to never find a job outside of academia in that field. as I don't want to do high-frequency trading nor develop games.
You could look lower in the stack, e.g. at compilers, drivers, and libraries for GPUs and other accelerators. That's not a huge field, but still a decent number of jobs to go around.