Find a guide on using Yubikey with gpg (such as this one [0]) and if you configure it it will work seamlessly.
Enabling touch-to-decrypt [1] can also reduce risk of decrypting stuff without you noticing.
[0]: https://www.yubico.com/support/knowledge-base/categories/art...
[1]: https://developers.yubico.com/PGP/Card_edit.html#_yubikey_4_...