[EggsOnToast]

It sounds like what's missing is a bad faith mechanic. Assuming it could be implemented in a way that's not terrible or outright ineffective, it would be interesting to see a legal framework for barring a party from filing patent suits if they accumulate enough verdicts that rule their case to be outrageous enough to earn them a black spot.

[rplnt]

It exists in some/many/dunno European jurisdictions. If the claim fails in court, the party that lost will pay for all legal expenses (up to a limit). You still need money to defend yourself, but you'll get them back once the case is thrown out/won.

[krageon]

As a rule you won't actually get back all the expenses if the case takes a (few) year(s), it's more a symbolic amount.

[mnw21cam]

I think the term you are looking for is "Vexatious litigant" [0]. People who are declared as such by a court must apply for permission (which is rarely given) before launching any court cases.

[0] https://en.wikipedia.org/wiki/Vexatious_litigation