[kadoban]

One way to handle this might be to have the data hidden, but verifiable in the future. That is: publish a signed hash of the benchmark questions, and every X amount of time swap them out and publish the old ones.

> I don't have a solution. I'm just saying this is complete bullshit, the idea that we are starting to exclaim "yay, hidden data! I can trust that!" just cannot be acceptable.

This feels just unkind. If you acknowledge that it's a hard (impossible?) problem, you should give some leeway to people doing their best until a consensus on the right approach(es) exists.