[XCSme]

But most benchmarks are not about that...

Are there even any "hallucination" public benchmarks?

[andrepd]

"Benchmarks" for LLMs are a total hoax, since you can train them on the benchmarks themselves.

[XCSme]

I would assume a good benchmark has hidden tests, or something randomly generated that is harder to game