[eru]

On the most basic level, you should be able to write a simple operating system from scratch. I heard http://pages.cs.wisc.edu/~remzi/OSTEP/ is good for an introduction to OS writing.

Some papers I read in no particular order:

Synthesis OS (http://valerieaurora.org/synthesis/SynthesisOS/) might be interesting for you. They do lots of runtime code synthesis.

Exokernels (follow links from https://en.wikipedia.org/wiki/Exokernel#Bibliography). And more recently Mirage (https://mirage.io/) and HaLVm (https://github.com/GaloisInc/HaLVM)

(I assume you already know how to program. Otherwise, brush up on that as step 0. C is still the canonical choice for OS work. But if you are feeling adventurous there's more choice.)