[mjcurl]

5/15, so the same as choosing the most common word.

I think I did worse when the prompt is shorter. It just becomes a guessing game then and I find myself thinking more like a language model.

[dalton01]

It says choosing the most common word was just 1/5 (and their best LLM was 4/15)

[toxik]

Yeah, it should be sentences that have low next token distribution entropy. Where an LLM is sure what the next word is. I bet people do real well on those too. By the way, I also had 5/15.