[inciampati]

Alas, if it could only remember and precisely relate more than 4k or 8k or 32k or 64k words...

And if only scaling that context length weren't quadratic...

Indeed, we would really expect an AI to be able to achieve AGI. And it might decide to do all kinds of alien things. The sky would not be the limit!

We have more than 100 trillion synapses in our brains. That's not our "parameter" count. It's the size of the thing that's getting squared at every "step". LLMs are amazing, but the next valley of disillusionment is going to begin when that quadratic scaling cost begins to rear its head and we are left in breathless anticipation of something better.

I am not as worried, I guess, as your average AI ethicist. I can hope for the best (I welcome the singularity as much as the next nerd), but quadratic isn't going to get easier without some very new kinds of computers. For those to scale to AGI on this planet it's questionable if they'll have the same architecture we're working with now. Otherwise, I'd expect a being whose brain is a rock with lightning in it to have take over the world long, long ago. Earth has plenty of both for something smart and energy efficient to have evolved in all these billions of years. But it didn't and maybe that's a lesson.

That all said, these LLMs are really amazing at language. Just don't ask them to link a narrative arc into some subtle detail that appeared twice in the last three hundred pages of text. For a human it ain't a problem. But these systems need to grow a ton of new helper functionality and subsystems to hope to achieve that kind of performance. And, I'll venture that kind of thing is a lower bound on the abilitites of any being who would be able to savage the world with it's intellect. It will have to be able to link up so, so many disparate threads to do it. It boggles our minds, which are only squaring a measly 100T dimension every tick. Ahem.

[startupsfail]

You can only hold around 7 to 10 numbers in your mind well, in your working memory. Let me give you a few: 6398 5385 3854 8577

You have 1 second, close your eyes and add them together. Write down the result.

I’m pretty sure that GPT-4 at its 4k setting would outperform you.

[The point being, we have not seen what even GPT-4 can do in its optimal environment. Humans use paper, computers, google, etc. to organize their thoughts and work efficiently. They don’t just sit in empty space and then put everything into the working memory and magically produce the results. So imagine now that you do have a similar level of tooling and sophistication around GPT-4, like there is present around humans. I’m considering that and it is difficult to extrapolate what even GPT-4 can do, in its optimal environment.]

[inciampati]

Indeed, and maybe less than 7...

I'll point out that chatgpt needs to be paying attention to the numbers to remember them in the way I'm taking about. You will need to fine tune it or something to get it to remember them blind. I suppose that's not what you're talking about?

There is a strong chance that I'll remember where to find these numbers in a decade, after seeing and hearing untold trillions of "tokens" of input. The topic (Auto-GPT, which is revolutionary), my arguments about biological complexity (I'll continue to refine them but the rendition here was particularly fun to write) or any of these things will key me back to look up the precise details (here: these high entropy numbers). Attention is perhaps all you need... But in the world it's not quite arranged the same way as in machines. They're going to need some serious augmentation and extension to have these capabilities over the scales than we find trivial.

edit: you expanded your comment. Yes. We are augmented. Just dealing with all those augmented features requires precisely the long range correlation tracking I'm taking about. I don't doubt these systems will become ever more powerful, and will be adapted into a wider environment until their capabilities become truly human like. I am suggesting that the long range correlation issue is key. It's precisely what uniques humans from other beings on this planet. We have crazy endurance and our brains both cause and support that capability. All those connections are what let's us chase down large game, farm a piece of land for decades, write encyclopedias, and build complex cultures and relationships with hundreds and thousands of others. I'll be happy to be wrong, but it looks hard-as-in-quadratic to get this kind of general intelligence out of machines. Which scales badly.

[startupsfail]

When doing the processing GPT remembers these in the “working memory” (very similar to your working memory that is just an actuation of neurons, not an adjustment of the strengths of synaptic connections).

And then, there is a chance that the inputs and outputs of GPT be saved and then used for fine-tuning. In a way that is similar to long-term memory consolidation in humans.

But overall, yes, I agree, GPT-4 in an empty space, without fine-tuning is very limited.

[dilyevsky]

It doesn’t remember anything unless you mean that intermediate values in calculation of forward pass is “remembering”. The prompt continuation feature is just a trick where they refeed previous questions/replies back to it with new questions at the end

[GistNoesis]

>And if only scaling that context length weren't quadratic...

There are transformers approximations that are not quadratic (available out of the box since more than a year) :

Two schools of thoughts here :

- People that approximate the neighbor search with something like "Reformer" and O(L log(L) ) time and memory complexity.

- People that use a low-rank approximation of the attention product with something like "Linformer" with O(L) complexity but with more sensibility to transformer rank collapse

[chillfox]

So how many of those 100 trillion synapses are actually in the part of the brain that does the thinking? Because the brain has different regions (subsystems) responsible for different things.

[robwwilliams]

> But these systems need to grow a ton of new helper functionality and subsystems to hope to achieve that kind of performance. And, I'll venture that kind of thing is a lower bound on the abilitites of any being who would be able to savage the world with it's intellect. It will have to be able to link up so, so many disparate threads to do it. It boggles our minds, which are only squaring a measly 100T dimension every tick.

Agreed: LLM are just one of many necessary modules. But amazing nonetheless. The quadratic scaling problem needs an attentional-conceptual extractor layer with working memory. Hofstadter points out that this needs to be structured as a recursive “strange loop” (p 709 of GEB). Thalamo-cortico-thalamic circuitry is a strange loop and attentional self-control may happens by phase- or time-shifting activity of different circuits to achieve flexible “binding” for attention and compute.

I’m actually optimistic that this is not a heavy computational lift but a clever deep extension of recursive self-modulating algorithms across modules. The recursion is key. And the embodiment is also probably crucial to bootstrap self-consciousness. Watching infants bootstrap is an inspiration.