INS chips providing enough precision for autonomous operation over last few kilometers are rather cheap nowadays too.
A small-ish plane frame could easily deliver a grenade from sleepy suburbs 10km straight to the Oval Office with only few minutes of warning (assuming somebody could actually detect it in time at all). We're just lucky nobody seems to be bothered to do that.
Well, if you managed to detect and track an incoming drone you could ‘jam’ its camera with a high-powered laser - or something that threw up a curtain of flares or burning magnesium, or whatever.
A drone swarm dropping hand grenades is like targeted artillery. Experience from WW 1 is that it's straightforward to defend against artillery fire, just retreat into a shelter. A target that's worth five dozen drones with grenades can afford a bunker.
A small-ish plane frame could easily deliver a grenade from sleepy suburbs 10km straight to the Oval Office with only few minutes of warning (assuming somebody could actually detect it in time at all). We're just lucky nobody seems to be bothered to do that.