[orly01]

Yes, monkeys could write Shakespeare works given enough time.

But in this case, it is really hard to know if a model is identifying "correct answers" reliably. A lot of answers are really hard to qualify as correct or not when written by humans, much more when written by a machine trying to trick readers into thinking the answer is correct. It can be done, but I doubt LLM are being trained to identify the subtle differences between those types of potential answers.