I'm using RSA8192 with SSH and not seeing any noticable performance problems. ECC is generally faster than RSA, so just always using Ed448 or even larger should be a no-brainer.
The comments about Ed25519 versus Ed448 were meant for the general case of their use, and not (necessarily) strictly in the use for SSH.
However, the reason for Ed25519 use in OpenSSH is because the keys are much smaller, and easier to cut/paste/etc, for the same security factor. The fact that they are faster for signing is a bonus.
All of the above correspond to roughly the security of AES 128. If you want AES 256 equivalence, double the size/length.
I'm in the habit of using ssh-copy-id(1), but there are a lot of places where you have to copy-paste in a web interface (and lots of instructions tell you to copy-paste into your remote ~/.ssh/authorized_keys file).
"Please note that, due to the structure of the underlying ASN.1 structure, RSA PEM bodies start always with the same characters: MIG for 1024 bit keys, MII for 2048 and 4096 bit ones."
But if you're handling hundreds or thousands of operations per second (e.g., a web site), then the delays add up.