[thematt]

Not a stupid question. What you're referring to is the blackout period during reentry. It's a challenge not just for missiles but spacecraft too. Lots of research has gone into it:

https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201000...

Beyond the advanced research concepts though, I believe the leading solution is typically inertial navigation (or a variation thereof):

https://en.wikipedia.org/wiki/Inertial_navigation_system

https://ui.adsabs.harvard.edu/abs/2016SPIE10158E..13L/abstra...