[gpugreg]

Same here. LLMs are great at spitting out well-known solutions to problems instead of the best one. The "long tail" of solutions is usually lost due to how tokens are sampled from the LLM's probability distribution.

What I found to help a lot is to ask for e.g. 10 different solutions to a problem and then choosing one of them. Sometimes, this even leads to borderline creative solutions if there aren't 10 different ones.

[btown]

In theory reasoning tokens should do the equivalent of this - explicitly create options outside of the quick-response probability space, so those can guide future generation.

In practice, models that do this won't be prioritized as much, because the economics of thinking tokens that stop by default at, say, one option plus a bit more planning (short of full alternatives) would be superior as long as billing is per-user instead of per-token. So we'll still need to play games with prompting!

[tliltocatl]

Without continuous feedback from real world, lower-probability token (and soon high-probability ones as well) will be complete garbage.

[exe34]

> LLMs are great at spitting out well-known solutions to problems instead of the best one.

I remember how Stack Overflow would close questions as duplicates just because somebody suggested the wrong answer that is also the right answer to the existing question. The best way to get a correct answer on Stack Overflow (and forums before that) was to post the wrong answer as part of your question.