[viraptor]

That's very much not my experience. I agree that the setup is harder than necessary and if you get something wrong the errors are not helpful. But once it works, it works. I've been using yubikey with gpg-agent for ssh auth for years and it generally just works. I've not seen the stability problems you experienced.