[xorcist]

Perahaps not directly, but it does add to cognitive overhead and makes securing the system as a whole more difficult.

If your environment contains multiple ssh servers running on non-standard ports you would not notice when something out of the ordinary happens.

Something listening to an unexpected port should be something you must look into, not shrug at.

[romeisendcoming]

So standardize on a non-standard port, document it and when you do have ssh probes assume it's targeted rather than just another l33t child running du-jour crack(x).

Moving to non-standard means more sophisticated discovery and possibly a more sophisticated attempt.