[mytec]

The only sluggishness I noticed, and was unexpected, was generating a SSH key.

ssh-keygen -t rsa -b 4096

This was considerably slower on the M1 Mac Mini than my 2017 iMac.

[macintux]

Interesting; possibly running under Rosetta? I could see crypto operations being very poorly emulated, and if it had to be translated first (if you only ran it once), that would obviously be even worse.