It's not directly a security problem as in your computer is gonna get hacked, but it can ruin your opsec if you don't want people to realize that you're the person controlling a specific key.
You are definitely right in that the public keys themselves don’t present a threat. I think on a penetration test this would show up as “Information Revealed” rather than something that is “Exploitable”. By sharing a username - you might be able to correlate this person to other platforms and technically increase your attack surface area.