[nuancebydefault]

The fact that the explaining part of the brain fills in any blanks in a creative manner (you need the shovel to clean the chicken shed), reminds me to some replies of LLMs.

I once provided an LLM the riddle of the goat, cabbage and wolf, and changed the rules a bit. I prompted that the wolf was allergic to goats (and hence would not eat them). Still the llm insisted on not leaving them together on the same river bank, because the wolf would otherwise sneeze and scare the goat away.

My conclusion was that the llm solved the riddle using prior knowledge plus creativity, instead of clever reasoning.

[captainmuon]

I believe LLMs are entirely analogous to the speech areas of the brain. They have a certain capacity of speaking automatically, reflexively, without involving (other) memory for example. That is how you are able to deliver quippy answers, that is where idioms "live". You can see this in people with certain kinds of brain damage, if they are unable to recall certain memories (or sometimes if you press somebody to recall memories that they don't have) they will construct elaborate stories on the spot. They won't even notice that they are making it up. This is called confabulation, and I think it is a much better term than hallucination for what LLMs do when they make up facts.

I feel this analogy confirmed by the fact that chain of thought works so well. That is what (most?) people do when they actively "think" about a problem. They have a kind of inner monologue.

Now, we have already reached the point that LLMs are much smarter than the language areas of humans - but not always smarter than the whole human. I think the next step towards AGI would be to add other "brain areas". A limbic system that remembers the current emotion and feeds it as an input into the other parts. We already have dedicated vision and audio AIs. Maybe we also need a system for logical reasoning.