[nathan7]

What you're describing is called a smartcard, and readily available on the market. I keep my PGP key on one.

[hollerith]

Does your PGP key stay on the smartcard or is a copy of it transferred to your computer on occasion?

[nathan7]

The key can be generated on the smartcard, and it's not possible to transfer it out of the smartcard by design. (anything that calls itself a smartcard but allows this isn't a smartcard)

[Nursie]

If it's a properly designed smartcard system then the key never leaves the card.