How do you propose the keys would escape a HSM? You wouldn't keep this stuff connected to the network, so someone would have to get physical access to even attempt to attack it.
How would a government lose control of keys generated and stored within a offline HSM in a shielded room, without actually losing possession of that HSM? This is very obviously not unfeasible to implement in a secure manner.
So far you've only given examples of secrets which are really easy to lose control of, this simply isn't one.
I'm not trying to argue that this should be done, you claimed this can't be done which is obviously false