[okhuman]

AI is going to continue to have incremental progress, particularly now in hardware gains. No one can even define what AGI is or what it will look like, let alone be something that OpenAI would own? Features progress is too incremental to suddenly pop out with "AGI". Fighting about it seems a distraction.

[root_axis]

There's also no reason to believe that incremental progress in transformer models will eventually lead to "AGI".

[snapcaster]

Yes, but I think everyone would agree that the chance isn't 0%

[root_axis]

I don't agree, I think many people would argue the chance is 0%.

[JohnFen]

I don't think the chance is 0%, but I do think that the chance is very, very close to 0%, at least if we're talking about it happening with current technology within the next hundred years or so.

[snapcaster]

Are you one of those people? how can you be so confident? I think everyone should have updated their priors after how surprising the emergent behavior in GPT3+ are

[root_axis]

I don't think GPT3's "emergent behavior" was very surprising, it was a natural progression from GPT2, and the entire purpose of GPT3 was to test the assumptions about how much more performance you could gain by growing the size of the model. That isn't to say GPT3 isn't impressive, but its behavior was within the cone of anticipated possibilities.

Based on a similar understanding, the idea that transformer models will lead to AGI seems obviously incorrect, as impressive as they are, they are just statistical pattern matchers of tokens, not systems that understand the world from first principles. And just in case you're among those that believe "humans are just pattern matchers", that might be true, but humans are modeling the world based on real time integrated sensory input, not on statistical patterns of a selection of text posted online. There's simply no reason to believe that AGI can come out of that.

[andoando]

I agree. I am baffled as to why there isn't more thought on developing AI starting from simple sensory input.

[nicklecompte]

Perhaps you should update your priors about "emergent behavior" in GPT3+: https://arxiv.org/abs/2304.15004

[famouswaffles]

This is like saying that nothing special happens to water at 100 degrees because if you look at the total thermal energy, it's a smooth increase.

[nicklecompte]

Transformer neural networks are not capable of true recursion, which is an excellent reason to think that the chance truly is 0%.

[CamperBob2]

That seems easy enough to fix

[pton_xd]

It's a static single-pass feed-forward network. How could that possibly result in AGI?! (Queue famous last words ...)

[Vecr]

Just feed the output through a quality system (with retry if the quality is too bad), scaffold it a bit, then run it back into the LLM. Should work(tm)

[Akronymus]

IMO it could become an AGI IFF it has an infinitely long context window. Otherwise I see absolutely no chance of it becoming a true agi

[jayveeone]

Non-zero chances don't deserve the hype AGI is receiving, is the issue.

And a lot of AI experts outside of the AGI grift have stated that it's zero.

[xiphias2]

Progress is definitely not inremental, it's exponential.

The same performance (training an LLM with a given perplexity) can be achieved 5x cheaper next year while the amount of money deep learning infrastructure gets increases exponentially right now.

If this method is able to get to AGI (which I believe but many people are debating), human intelligence will just be mostly ,,skipped'', and won't be a clear point.

[CatWChainsaw]

In nature, exponential curves reveal themselves to be sigmoidal on a long enough time scale. Since you're on HN you probably have a mathematical bent, and you should know that.

[blibble]

how long do you think the "exponential" (that looks very linear to me) growth in funding can continue?

until it's more than US GDP? world GDP? universe GDP?

either way you're close to the point it will have to go logistic