If you set the appropriate linux capabilities flag on a binary such as sshd at bootup then unprivileged users can bind to 22, no problem.
setcap 'cap_net_bind_service=+ep' /usr/sbin/sshd
Could even run it as a daemon unprivileged from a home directory with "systemd --user"
That said if you have multiple users and want every user to have their own sshd reachable on port 22 on the same machine you probably want to listen on vhost namespaced unix sockets and have something like haproxy listen on port 22 instead. Haproxy could of course also run unprivileged provided it has read access to all the sockets.
The way many including me manage systems without root privileges at runtime is by compiling immutable rootfs images that run in ram with kernel, init, mounting filesystems and assigning any users and privilege assignments, then drop to user privs.
That stuff needs to change very seldom, so when you do need to change it you just generate a new tiny rootfs image in a few seconds and reboot to pivot to it or maybe have a kexec trigger if you are feeling fancy.
For my primary workstation the entire disk is my home partition and I boot my latest rootfs from a flash drive. In other cases network boot.
For that you really only need CAP_NET_BIND_SERVICE.
The bigger issue is that if you want to install or update system-wide packages, many of those will be used by privileged processes. Suppose you want to update /bin/sh. Even if the only permission you had is to write binaries, that'll get you root.
setcap 'cap_net_bind_service=+ep' /usr/sbin/sshd
Could even run it as a daemon unprivileged from a home directory with "systemd --user"
That said if you have multiple users and want every user to have their own sshd reachable on port 22 on the same machine you probably want to listen on vhost namespaced unix sockets and have something like haproxy listen on port 22 instead. Haproxy could of course also run unprivileged provided it has read access to all the sockets.