[kilroy123]

Yes, I don't understand their ridiculous AGI hype. I get it you need to raise a lot of money.

We need to crack the code for updating the base model on the fly or daily / weekly. Where is the regular learning by doing?

Not over the course of a year, spending untold billions to do it.

[tomohelix]

Technically, the models can already learn on the fly. Just that the knowledge it can learn is limited to the context length. It cannot, to use the trendy word, "grok" it and internally adjust the weights in its neural network yet.

To change this you would either need to let the model retrain itself every time it receives new information, or to have such a great context length that there is no effective difference. I suspect even meat models like our brains is still struggling to do this effectively and need a long rest cycle (i.e. sleep) to handle it. So the problem is inherently more difficult to solve than just "thinking". We may even need an entire new architecture different from the neural network to achieve this.

[chikere232]

> Technically, the models can already learn on the fly. Just that the knowledge it can learn is limited to the context length.

Isn't that just improving the prompt to the non-learning model?

[mike_hearn]

Google just published a paper on a new neural architecture that does exactly that, called Titans.

[KuriousCat]

Only small problem is that models are neither thinking nor understanding, I am not sure how this kind of wording is allowed with these models.

[ben_w]

All words only gain meaning through common use: where two people mean different things by some word, we influence each other until we're in agreement.

Words about private internal state don't get feedback about what they actually are on the inside, just about what they look like on the outside* — "thinking" and "understanding" map to what AI give the outward impression of, even if the inside is different in whatever ways you regard as important.

* This is also how people with aphantasia keep reporting their surprise upon realising that scenes in films where a character is imagining something are not merely artistic license.

[ninetyninenine]

I understand the hype. I think most humans understand why a machine responding to a query like never before in the history of mankind is amazing.

What you’re going through is hype overdose. You’re numb to it. Like I can get if someone disagrees but it’s a next level lack of understanding human behavior if you don’t get the hype at all.

There exists living human beings who are still children or with brain damage with comparable intelligence to an LLM and we classify those humans as conscious but we don’t with LLMs.

I’m not trying to say LLMs are conscious but just saying that the creation of LLMs marks a significant turning point. We crossed a barrier 2 years ago somewhat equivalent to landing on the moon and i am just dumb founded that someone doesn’t understand why there is hype around this.

[b112]

The first plane ever flies, and people think "we can fly to the moon soon!".

Yet powered flight has nothing to do with space travel, no connection at all. Gliding in the air via low/high pressure doesn't mean you'll get near space, ever, with that tech. No matter how you try.

AI and AGI are like this.

[dTal]

And yet, the moon was reached a mere 66 years after the first powered flight. Perhaps it's a better heuristic than you are insinuating...

In all honesty, there are lots of connections between powered flight and space travel. Two obvious ones are "light and strong metallurgy" and "a solid mathematical theory of thermodynamics". Once you can build lightweight and efficient combustion chambers, a lot becomes possible...

Similarly, with LLMs, it's clear we've hit some kind of phase shift in what's possible - we now have enough compute, enough data, and enough know-how to be able to copy human symbolic thought by sheer brute-force. At the same time, through algorithms as "unconnected" as airplanes and spacecraft, computers can now synthesize plausible images, plausible music, plausible human speech, plausible anything you like really. Our capabilities have massively expanded in a short timespan - we have cracked something. Something big, like lightweight combustion chambers.

The status quo ante is useless to predict what will happen next.

[b112]

By that metric, there are lots of connections between space flight and any other aspect of modern society.

No plane, relying upon air pressure to fly, can ever use that method to get to the moon. Ever. Never ever.

If you think it is, you're adding things to make a plane capable of space flight.

[dTal]

>By that metric, there are lots of connections between space flight and any other aspect of modern society.

Indeed. But there's a reason "aerospace" is a word.

>No plane, relying upon air pressure to fly, can ever use that method to get to the moon

No indeed. But if you want to build a moon rocket, the relevant skillsets are found in people who make airplanes. Who built Apollo? Boeing. Grumman. McDonnell Douglas. Lockheed.

[mlyle]

I feel like aeronautics and astronautics are deeply connected. Both depend upon aerodynamics, 6dof control, and guidance in forward flight. Advancing aviation construction techniques were the basis of rockets, etc.

Sure, rocketry to LEO asks more in strength of materials, and aviation doesn’t require liquid fueled propulsion or being able to control attitude in vacuum.

These aren’t unconnected developments. Space travel grew straight out of aviation and military aviation. Indeed, look at the vertical takeoff aircraft from the 40s and 50s, evolving into missile systems with solid propulsion and then liquid propulsion.

[b112]

AGI may use the same hardware, or same compute concepts.

But LLMs (like low/high pressure wing flight) will never result in AGI (you won't get to the moon with a wing).

You're making my point.

[Xmd5a]

The first plane to ever fly was in fact ignored by the general public for several years

https://bigthink.com/the-past/wright-brothers-ignored/

[ninetyninenine]

That’s not true. There was not endless hype about flying to the moon when the first plane flew.

People are well aware of the limits of LLMs.

As slow as the progress is, we now have metrics and measurable progress towards agi even when there are clear signs of limitations on LLMs. We never had this before and everyone is aware of this. No one is delusional about it.

The delusion is more around people who think other people are making claims of going to the moon in a year or something. I can see it in 10 to 30 years.

[b112]

That’s not true. There was not endless hype about flying to the moon when the first plane flew.

I didn't say there was endless hype, I gave an example of how one technology would never result in another... even if to a layperson it seems connected.

(The sky, and the moon, are "up")

People are well aware of the limits of LLMs.

Surely you mean "Some people". Because the point in this thread is that there is a lot of hype, and FOMO, and "OMG AGI!" chatter running around LLMs. Which will never ever make AGI.

[ninetyninenine]

You said you didn’t comprehend why there was hype and I explained why there was hype.

Then you made an analogy and I said your analogy is irrelevant because nobody thinks LLMs are agi nor do they think agi is coming out of LLMs this coming year.

[b112]

Actually, plenty of people think LLMs will result in AGI. That's what the hype is about, because those same people think "any day now". People are even running around saying that LLMs are showing signs of independent thought, absurd as it is.

And hype doesn't mean "this year" regardless.

Anyhow, I don't think we'll close this gap between our assessment.

[pizza]

And yet, the overall path of unconcealment of science and technological understanding definitely traces a line that goes from the Wright brothers to Vostok 1. There is no reason to think a person from the time of the Wright brothers would find it to be a simple one easily predicted by the methods of their times, but I doubt that no person who worked on Vostok 1 would say that their efforts were epochally unrelated to the efforts of the Wright brothers.