[cnvogel]

The "canonical" way of solving this is of course with a certificate hierarchy. You can configure your target system to allow authentication whenever your certificate is signed by the "golden" key, and by creating the certificates with expiration dates it's easy to restrict the time that a certificate (which maybe got stolen or was made public inadvertendly) can be used for login.

Adding auxiliary data (comparable to X509 certificate usage restrictions only for code-signing or signing for emails) would allow you to have a (central hardened) machine hand out certificates to your "semi trusted" administrators that only allow login from a specific IP-address, to a specific target-system, up to a certain date or even time. Putting this functionality in a script (to acquire the certificate and login to a system) such a wrapper could most likely be used as a drop-in replacement for the normal ssh-client.

If your restriction is to use only the default openssh already deployed to the target machines, one quick solution to hide the critical private key from the developer/admin/service machines would be to place it on a (hopefully somewhat hardened) machine running the ssh-agent, and access the agent by creative use of tunelling. In essence then the hardened machine would act as a kind of "smartcard" for all your admins.