[hibernator149]

LLMs are really good at system 1 thinking (fast, intuitive) but can't do system 2 (planning, calculating) at all. Just like in Go/Baduk, neural net based AIs were good at tactical play but fell apart against pro players with good long term strategy. Until NNs were combined with Monte Carlo Tree Search and we got AlphaGo with great tactical and strategial play.

We need a good search strategy that fits LLMs before we can get AGI. Maybe Chain Of Thought can become such a strategy, but it always felt too clunky to me.

[pona-a]

We already do beam search, which allows LLMs to backtrack on branches until they find the highest probability path. Discounting things like Toolformer, this is about as good as it's going to get.