[yomismoaqui]

That's really interesting... can you give more details about the problem you are using?

This sounds like in there will be a race between this kind of booby trap tests and AIs learning them.

[gritzko]

Long-tail problems are not reiterated in the dataset. Making LLM remember that can be difficult.