[somenameforme]

*Assuming you don't mean mathematically prove.*

I can't test the bot right now, because it seems to have been hugged to death. But there's quite a lot of simple tests LLMs fail. Basically anything where the answer is both precise/discrete and unlikely to be directly in its training set. There's lots of examples in this [1] post, which oddly enough ended up flagged. In fact this guy [2] is offering $10k to anybody that create a prompt to get an LLM to solve a simple replacement problem he's found they fail at.

They also tend to be incapable of playing even basic level chess, in spite of there being undoubtedly millions of pages of material on the topic in their training base. If you do play, take the game out of theory ASAP (1. a3!? 2. a4!!) such that the bot can't just recite 30 moves of the ruy lopez or whatever.

[1] - https://news.ycombinator.com/item?id=39959589

[2] - https://twitter.com/VictorTaelin/status/1776677635491344744

[sfkgtbor]

Multiple people found prompts to make LLM solve the problem, and the $10k has been awarded: https://twitter.com/VictorTaelin/status/1777049193489572064

[imtringued]

The entire problem with LLMs is that you don't want to prompt them into solving specific problems. The reason why instruction finetuning is so popular is that it makes it easier to just write whatever you want. Text completion on the other hand requires you to conform to the style of the previously written text.

In a sense, LLMs need an affordance model so that it can estimate the difficulty of a task and plan a longer sequence of iterations automatically according to its perceived difficulty.