[lukev]

Completely agree.

From a computer science point of view: a single prompt/response cycle from a LLM is equivalent to a pure function; the answer is a function of the prompt and the model weights and is fundamentally reducible to solving a big math equation (in which each model parameter is a term.)

It seems almost self evident that "reasoning" worthy of the name would involve some sort of iterative/recursive search process, invoking the model and storing/reflecting/improving on answers methodically.

There's been a lot of movement in this direction with tree-of-thought/chain-of-thought/graph-of-thought prompting, and I would bet that if/when we get AGI, it's a result of getting the right recursive prompting pattern + retrieval patterns + ensemble models figured out, not just making ever-more-powerful transformer models (thought that would certainly play a role too.)

The LLM isn't the whole brain. Just the area responsible for language and cultural memory.

[kaba0]

Ideally a recursive execution would also be a pure function - maybe a better way to put it about current LLMs is that they are a single mathematical expression being built up from a fix number of nodes and only addition and multiplication.

[lukev]

yes, except the "reasoning" process should also be able to look up facts (retrieval) and invoke external tools, making it non-pure.

[Scubabear68]

Agree. I am not an expert on AGI, but I suspect this route is a dead end because I believe true AGI requires interaction with an external environment to learn eg there needs to be a feedback loop to external things that it can interact with and learn from.

“You learn best from your mistakes” comes to mind here.

And not just shallow feedback loops but bigger, more profound ones eg “I didn’t go to work for a month and so I was fired and and now I have no money and it will now be hard to be hired again”.

[hskalin]

Do you have any good resources for chain-of-thought prompting experiments?