[botexpert]

Games, and social sciences.

It's a branch of maths that formalized a bunch of things and allows lovely proofs for certain interesting settings that economists, sociologists and politologists observe and analyze.

Other than that it currently isn't used that much in computer science.

[codingmyway]

Getting all the selfish incentives of stakeholders and attackers to enable and secure something like Bitcoin is the example. Whether it works out in the end or not is an open question. The current blockage could a be a failure of the game theory to predict the short term tactics of miners at the expense of the long term.