[thewakalix]

As I understand it, your SSH client gives out all your public keys, but doesn’t authenticate with all of them. That might be the crux.