[EvgeniyZh]

> I wasn't trying to invent anything. Just describing what you would obviously have to do if you were to take a "scientific" or "objective" approach: Sound experiments, reproducible, free of financial incentives.

But how is it different from what arena or matharena does?

> That sort of thing is not showing broad intelligence anymore than a person both knowing a chess player and a poet is having broad intelligence.

The claim is that these problems require somewhat broad intelligence by themselves, as opposed to specialization into specific task while unable to do anything else.