[dinfinity]

> It hasn't worked out like that

It seems quite premature to say that. We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive. The recursive self-improvement aspect that is necessary for the actual singularity is something we're only really starting to get into this year.

If the singularity is 5 years from now, that is still much sooner than most people (including me) previously expected it to happen.

[an0malous]

> We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive.

The “Attention Is All You Need” paper came out in 2017 and OpenAI released GPT-1 in 2018. ChatGPT was released in late 2022 but that was not the beginning of LLMs. Transformers and generative AI go back even further.

[pohl]

Can you pinpoint the revolution by pinpointing moments of invention and release? Revolution takes time because it’s more about uptake, downstream effects, and feedback.

[daveguy]

The revolution is way overblown.

[bojan]

> It seems quite premature to say that. We're 3 to 4 years into the LLM revolution and the rate of progress is still impressive.

When we were 3 to 4 years into mobile phone revolution it was already abundantly clear that it's changing the way people leave.

Ditto when mobile phones became smartphones, and for many other tech items we got in the past decades.

As for LLMs... outside of a few precious professions you could live your life and not notice they are there.

[treis]

This isn't accurate. The first mobile phone came out in 1983. It took at least a decade before they started to become common.

[pproe]

Mobile phones at that time were 800g and cost $4000. Using your logic you could say we are almost 20 years into the age of LLMs because Cleverbot was released in 2008.

[treis]

The first prototype of a cellphone was functional in the early 70s. Before that you had walkie talkies and farther back wireless communications.

We're just shy of 10 years since attention is all you need and just a few years past the first commercially viable LLMs. So in cellphones we are something like early 90s.

[tavavex]

What they're saying is that mobile phone tech didn't scale at the same rate as LLMs. How fast it takes a technology to unfold into full adoption varies depending on the industry and on what's blocking development at its inception.

Mobile phones were severely constrained by hardware that made them impractically expensive and unwieldy, and also by the time it took to set up wireless networks at a large enough scale. That's why it took mobile phones a few decades to really get going, and this is going to be different for every other technology.

LLMs, for instance, were not as hardware-constrained - we have a lot more compute now, but there wasn't a hardware paradigm shift or anything, things were already pretty good a few years ago - now it's just about scale. LLMs were also always relatively affordable - there wasn't a period where they cost $10/token, so we don't get the same kind of affordability scaling as with phones. They also utilize existing internet infrastructure and don't need to be rolled out in some new worldwide grid.

[treis]

I think this is off. LLMs are severely constrained by hardware and we've seen at least an order of magnitude improvement in flops since ChatGPT launched. Over the next couple of decades we are likely to see that level of improvement in memory capacity. There's likely at least a few step changes left.

Regardless, the point of the GP that we saw revolutionary changes in society within a few years of technology introduction is off. It took decades for the full impact of the pc, internet, or mobile to be seen. We're still in a very early phase of figuring out how to use LLMs.

[inigyou]

Everyone uses chatgpt instead of Google now, but at least half of that is on Google.

[goatlover]

Define "everyone". Google has an AI result before the search results. People do use that. People also still use search results when they're dissatisfied with the AI answer.

[phil21]

> As for LLMs... outside of a few precious professions you could live your life and not notice they are there.

And old folks said the same thing about the internet and smart phones. Fads that would pass. While the younger generations lived an entirely different lifestyle.

I’m personally seeing the same. I resisted AI being a “thing” for a long while, but since I’m around a lot of teens and young adults it was a choice of burying my head in the sand or deciding to take a look.

They are going to change society on a scale far beyond smart phones. In ways many will not see as positives, but change never goes the way one may expect.

[ThrowawayR2]

> "And old folks said the same thing about the internet and smart phones."

Ironic how LLM promoters like to hallucinate citations, much like the tools they espouse.

[mh-]

"Internet 'may be just a passing fad as millions give up on it'" - Daily Mail, December 2000

I recall seeing similar sentiment in other publications throughout the late 90s.

https://www.reddit.com/r/retrocomputing/comments/1m6i4ui/art...

(Apologies that this was the best image host I could find for it)

[ThrowawayR2]

Surely you jest. Since people here are fans of LLMs, the Google summary describes said publication as: "The Daily Mail is widely regarded as a sensationalist tabloid with low factual credibility and a strong right-leaning bias."

I was there in the 1990s too and we were deluged with headlines trumpeting how everyone, young and old, was rushing headlong onto the Information Superhighway as the dotcom bubble kept inflating.

[reinitctxoffset]

As a vocal advocate for the use of machine intelligence in certain contexts and a skeptic of claims they are useless or always hallucinate or whatever...

It hasn't been even close to even the recent claims! It's been like three straight years where it was supposed to utterly transform society any minute now.

I think we more or less have a fragile consensus that in like, computer programming and really very little else, that on a good day you're probably going to come out ahead with the AI assist. That seems relatively uncontroversial now. But it also seems like success with AI is largely about working hard to get good at it, as a first order concern. It is not at all obvious that blind, uncritical use by anyone is a net win.

It's pretty unclear, sone might say dubious, that anyone has made serious, aboveboard money net of debt and equity, other than the hardware vendors. There's some pretty serious revenue, but it's a drop in the proverbial bucket against the outlay. There like a two trillion dollar balance sheet hole in the US alone where investment into AI has gone.

While I personally don't agree with them, multiple S-tier machine learning researchers think we've got our wheels stuck in the mud, that autoregressive decoder architectures on a tokin-suffix pre-train is tapped out as a paradigm.

It's ok to say "AI is starting to get useful in pretty durable ways" and not sound like an Anthropic shareholder/employee, i.e. completely full of shit.

[uwamamama]

Uncritical use of today's AI beats all of the optimized use of yesterday's AI by far. I think people like the idea that their success is a result of their efforts, but this is just not the truth.

[reinitctxoffset]

I don't think anyone's disputing that the models are better than a year ago (though clearly all this, recursive self improvement stuff is utterly hypothetical and that's being generous, most of the progress has been on cost and fit and finish stuff). When it's on the plan, I'll use Fable for some stuff. If I'm paying for Opus? It's 4.5 or 4.6 which were dramatically better aligned and token efficient in the trace at a capability gap that's "you win some and lose some".

That can be true while it also being the case that to someone who has no idea how this stuff works under the hood, it's basically Dunning Kreuger in a box. The next person to go /u/PhdInEverything on me with Fable is getting an education in the history of hardware support for mixed precision training or something. Fable is a masterclass in refusal to ground and a dozen other alignment catastrophes.

So yeah, the models are still getting a little better, but from here out I think it's rapidly becoming a skill game.

[idiotsecant]

It's clear that while LLM progress is continuing, it's at something like a linear rate, or slower.

The singularity requires exponential improvement. That's not happening. Maybe we will develop something that sets that off, but there's no indication of that right now.

[dinfinity]

Yet. If the exponential improvement had already started, the singularity would already have happened or happen very, very soon.

The logic has always been that the AI would have to have significant tools and agency to do self-improvement for the singularity to occur. This is exactly the thing that a bunch of the AI labs are working on hard right now.

[daveguy]

Sorry, but labs working hard doesn't make non-polynomial problems suddenly polynomial.