[isomorphic_duck]

No, the purpose was to create a (automated) test set in the first place. The author builds an LLM judge which can score the LLMs participating during test-time. That would be why the author used the strongest model (Opus 4,7 at the time) as the judge.

[brigandish]

Yes, by giving hints it verifies the test because if it can't find it with hints then it shouldn't find it without hints, else there is some other mechanism at work (which would also be useful to know, like access to other tools).