[MacsHeadroom]

>I do agree with you that memory and self-excitation seem like they are the last push thats needed to get to something more akin to "AGI"

"We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton, hence computationally limited. However, augmenting such models with a read-write memory creates the possibility of processing arbitrarily large inputs and, potentially, simulating any algorithm."

From "Memory Augmented Large Language Models are Computationally Universal"

https://deepai.org/publication/memory-augmented-large-langua...