[gruez]

>Will the solution to this, then, be to have some sort of "smart card enabled device"? For example, assuming TOFU, you manually accept all device's public keys (and all devices, including cables and stuff will have one of these). Then, the computer will have to verify all actions done by those devices by sending a challenge for each action.

Even that's not enough. If you're feeling extra-evil you could tamper with the keyboard switches/traces to do whatever evil stuff you want. It's not like you can authenticate the on/off state at a switch level.