Okay, but why is that something to be concerned about? How would one be able to probe to see if a server recognizes a public key? Why does that matter?
If I have your public key I could probe a server to see if it recognizes that public key. Which would tell me information you might not want to be leaked.
I think what the other comment is getting at is that you're essentially leaving footprints behind on various servers and that information could be used to narrow down who you are, where you work, etc. For example, if you used the same key pair for a work machine at the big social media site example.com as you did late.sh, they could probe example.com's machine and see that you'd been there.
At the very bottom of the website they give you a command that would not leak your public keys.
`ssh-keygen -t ed25519 -f ~/.ssh/late_throwaway && ssh -o IdentitiesOnly=yes -i ~/.ssh/late_throwaway late.sh`
this would only send the late_throwaway public key