You actually can. Well, you can't "sue" them, but you can impeach and remove them. It's simply never actually happened before.
http://en.wikipedia.org/wiki/Samuel_Chase#Impeachment