[lxgr]

That should be very noticeable to the victim though, right?

Their own key would not work (unless the attacker persistently MITMs them and swaps their own credential in for every subsequent authentication) or they'd see multiple credentials being present in their account.

It's also a good idea to send out an email for every new credential added.