[harshreality]

Before the era of quantum physics, the idea of a weapon like fission or fusion weapons would've been inconceivable without resorting to magical thinking.

Your argument, and Yann's, is that AGI, or what you call AGI, is a kind of quasi-intelligent golem that, despite being generally intelligent, doesn't have human-level intelligence. Your claim that it will never be human-equivalent, much less trans-human/ASI, is built into your worldview. It's not a conclusion. It's an assumption on your part.

People like you and Yann can believe that if you want, but you have no evidence, because nobody knows what's required for human-level intelligence. Nobody knows whether some kind of system involving neural nets could develop human-level intelligence or beyond. It could involve different architecture or training methods. There's no assumption by AI doomers that AGI will be achieved by a LLM with more parameters or more or better training data.

[bangkoksbest]

What is human-level intelligence is a normative and philosophical question, which means it will not be resolved in some objective manner. There's no requirements for us to know, there's only requirements for us to propose in the form of norms.

The only approach we will be able to claim objectively can produce systems with human-level intelligence is procreation.

[pixl97]

This kind of reminds me of the history heavier than air flight and those that thought it wasn't possible along with those that thought it would only be possible by flapping like a bird.

Of course over 100 years later we know it's not only possible but manned flight is far more capable than natural flight in almost every metric.

[worldsayshi]

>despite being generally intelligent, doesn't have human-level intelligence

I think this whole quote is riddled with assumptions in this debate.

What is human level? Is it really a "level" or is human level just a local variant in a space of possible intelligence varieties that maybe could be sorted along one dimension of levels of maybe multiple? Can it be super intelligent without being autonomous at all?

I'm not saying that you're wrong in just pointing out where people ought to have a myriad of different assumptions.

[mikeravkine]

This is not about intelligence, it's about agency. Text completion generators don't have agency no matter how big they get.

[DennisP]

Adding agency to LLMs is not that hard and people are already doing it.

https://promptengineering.org/what-are-large-language-model-...

[esafak]

Agency is simply something that business LLM products don't need. That does not mean it would be hard to add in say a robotic setting; all animals have it.

[zmgsabst]

They don’t share your assumption a super intelligent AGI is a problem.

[ruszki]

It’s kinda proven that it’s a problem: https://www.youtube.com/watch?v=3TYT1QfdfsM