[kruffin]

Reading again, it does look like the installation is limited in some way if you choose to use public keys and your user doesn't have the privileges (without sudo) necessary for some of Cockpit's commands it tries to run.

Note that when a user is authenticated in this way [public key] the authentication happens without a password, as such the standard cockpit reauthorization mechanisms do not work. The user will only be able to obtain additional privileges if they do not require a password.