For SSH, changing to a random port number resulted in zero connection attempts from bots for months on end. It seems bots just never bother scanning the full 65535 port range.
For most of my VMs there's no ssh running. I use wireguard to connect to a private IP. I haven't done this on the bare metal yet but I might. Though barring exploits like we had recently nobody is getting into a server with either strong passwords or certificates. Fail2ban in my eyes is a log cleaner. It's not useful for much else.