My company Foxpass (YC S15) has a product to manage SSH keys. It serves as access control too -- the keys are only available on the hosts where a user should have access.
We've been using Foxpass for more than a year now and can definitely recommend. We have Amazon Machine Images with the required packages installed and configured; we use the web interface to grant/revoke access to users and add SSH keys. So each user logs in as themselves.
Aren has been awesome with responding to emails and helping us set it up too.
I didn't mention it in the original post, but Foxpass also has an LDAP endpoint so it can manage users and groups on your linux machines.
This means you can set up a linux group with sudo capabilities (sudo or wheel, usually) in /etc/sudoers. Then using Foxpass you can manage the membership of that group by adding users on a permanent or temporary basis.
Aren has been awesome with responding to emails and helping us set it up too.