[dredmorbius]

And if you lose / damage that physical key?

Look at the physical world analogue. If you lose a house key, you can force the lock, break down the door, re-key the door, pick the lock, or any number of other means of re-gaining access, up to and including going through the wall / ceiling / floor.

If you lose an authentication key, you're S.O.L.

[mozzieman]

That is true, there is no silver bullet. For the people like me that know and have multiple keys in multiple secure locations are probably mostly fine, other solutions on government och bank websites is that they need somekind of formal verification before giving you a new key. There are always tradeoffs regardless of solution, depends on what you value.