[knodi123]

That's remarkably short-sighted. First of all, no, millions of them don't walk the earth - the "A" stands for artificial. And secondly, most of us mere humans don't have the ability to design a next generation that is exponentially smarter and more powerful than us. Obviously the first generation of AGI isn't going to brutally conquer the world overnight. As if that's what we were worried about.

If you've got evidence proving that an AGI will never be able to design a more powerful and competent successor, then please share it- it would help me sleep better, and my ulcers might get smaller.

[seadan83]

Burden of proof is to show that AGI can do anything. Until then, the answer is "don't know."

FWIW, it's about 3 to 4 orders of magnitude difference between the human brain and the largest neural networks (as gauged by counting connections of synapses, the human brain is in the trillions while the largest neural networks are low billion)

So, what's the chance that all of the current technologies have a hard limit at less than one order of magnitude increase? What's the chance future technologies have a hard limit at two orders of magnitude increase?

Without knowing anything about those hard limits, it's like accelerating in a car from 0 to 60s in 5s. It does not imply that given 1000s you'll be going a million miles per hour. Faulty extrapolation.

It's currently just as irrational to believe that AGI will happen as it is to believe that AGI will never happen.

[knodi123]

> Burden of proof is to show that AGI can do anything.

Yeah, if this were a courtroom or a philosophy class or debate hall. But when a bunch of tech nerds are discussing AGI among themselves, claims that true AGI wouldn't be any more powerful than humans very very much have a burden of proof. That's a shocking claim that I've honestly never heard before, and seems to fly in the face of intuition.

[seadan83]

> claims that true AGI wouldn't be any more powerful than humans very very much have a burden of proof. That's a shocking claim that I've honestly never heard before, and seems to fly in the face of intuition.

The claim in question is really that AGI can even exist. The idea that it can exist, based on intuition, is a pre-science epistemology. In other words, without evidence, you have an irrational belief - the realm of faith.

Further, I've come to fully appreciate that without actually knowing the reasons or evidence for why certain beliefs are held, often we realize that our beliefs are not based on anything and could be (and possibly often are) wrong.

If we standing on just intuition there would be no quantum physics, no heliocentric galaxy, etc.. Intuition based truth is a barrier, not a gateway.

Which is all to say, the best known epistemology is science (assuming we agree that the level of advancement since the 1600s is largely down to the scientific method). Hopefully we can agree that 'science' is not applicable to just a courtroom or a philosophy class, it's general knowledge, truth.

Your framing also speaks to this. As if it is a binary. If you tell me AGI will exist, and I say "prove it". I'm not claiming that AGI will not exist. The third option is I don't know. I can _not_ believe that AGI will _not_ exist. I can at the same time _not_ believe that AGI will _exist_. The third answer is "I don't know, I have no knowledge or evidence" So, no shocking claim is being made on my part here AFAIK.

The internet for sure is a lot less entertaining when we demand evidence before accepting truth. Though, IMO it's a lot more interesting when we do so.

[Scrounger]

> That's remarkably short-sighted

I agree. Once these models get to a point of recursive self-improvement, advancement will only speed up even more exponentially than it already is...