I thought about using Qubes OS on my new computer but it seemed like a lot of work and tweaking and still having to live with the limitations so I abandoned the idea.
If you're really paranoid, Qubes has a "USB qubes" vm to handle untrusted USB devices. Then you only need to trust that the attacker can't tamper with PS/2.