[hanselot]

I think framing it this way could actually help us reach the next step with AI, if we ask how we could imbue it with those properties.

A human neural net is constantly bombarded with inputs from many different senses, which firstly, gets prioritized based on prior usefulness. That usefulness is updated all the time, constantly, and that's what any current AI implementation lacks.

1 - The continuous integration of data from all "senses". This one should be self-evident, as obviously, all our senses are constantly barraging our brain with data and it learns to handle that over time, in whichever way your genetic makeup + learned internal cognitive processes dictate it be handled.

2 - The network that decides which data requires which amount of attention, and whether to store it in short or long term memory. This is obviously tied in quite closely to 1, as you need massive amounts of data to understand underlying patterns, and which data is just spam, versus what's really valuable.

3 - And with these two things together come the emergence of improving of approximation of consistency. Which means, this itself is a metric which the agent running the other agents needs to be aware of. Its silly to think the human brain is a single agent. It makes way more sense to see it as various interacting agents that equate to a greater sum than its parts.

Now, that being said, I'm not an expert on AI or Data Science, but this is more or less how my understanding of computational theory of mind meets with biological computing and neural networks. My theory is that, the first actually intelligent AI will be one that is composed of a network that makes decisions on how to spend a unit of iteration. One iteration becomes "an instant" to the AI. Aka, the AI decides to spend one iteration thinking, or spawns a sub-agent (which it is aware will consume resources that other processes might also need access to, but it needs to be able to decide which action to pursue).

So in all honesty, its amazing to me that LLM's on their own have been able to achieve this level of "personhood" despite their being only a tiny subset of the whole that makes up a "conscious" entity.

Edit: Misunderstood parent's point.

[Symmetry]

Right. I sort of worry that because LLMs are able to be so coherent without anything resembling a human's working memory at all, then adding either a crude working memory still working with LLM tokens or adding a more sophisticated one assembled from "chunks"[1] will let an AI based on them ascend to clearly superhuman reasoning with just an architectural improvement and no need for any more flops invested than we already put into GPT4.

[1]https://en.wikipedia.org/wiki/Chunking_(psychology)