[knowriju]

ARC-AGI has a hidden private test suite, right ? No model will have access to that set.

[variadix]

I doubt they have offline access to the model, i.e. the prompts are sent to the model provider.

[xlbuttplug2]

Even if the prompts are technically leaked to the provider, how would they be identified as something worth optimizing for out of the millions of other prompts received?