[waynecochran]

Somewhere in all this it is crazy that the choice could be between a US company creating an AI that could doom civilization or letting China create the AI that dooms civilization. Do we want to be the first to "summon the demon" in our own fashion or let China manifest it first. Not saying this is the choice, but it would be a crazy dillema, albeit easy choice imo, if it was.

[worik]

> albeit easy choice imo

China, obviously.

[waynecochran]

Mao would have been great w AGI.

[fragmede]

Stick "what did Mao think of lazy people" into your closest LLM for a different take on that.

[anon291]

nothing is dooming civilization. These takes are so dumb. Civilization exists because humans want to reproduce.

[waynecochran]

I don't see it either, but there are folks smarter than both of us that disagree.

[AnimalMuppet]

"Folks smarter than us" sometimes start from really weird starting points. Their logic from their may be flawless, better than we can do, but if their starting point is wrong, who cares? They aren't going to get wrong answers anyway. They're just going to go further extrapolating the logical consequences of the wrong starting point, and that's not really any more useful than a stupid person doing it.

[ActorNightly]

Those folks are not "smarter", they are just louder. Just think independently for a few mins and its pretty easy to see why they are wrong.

For an all powerful AGI to exist, it has to basically beat the computationally irreducable processes within nature - i.e it has to simulate reality faster than reality, with a high degree of accuracy, which would imply that NP=P amongst other things.

And thats assuming that anyone has any idea to build an AI that can automatically build necessary simulations to make decisions in the first place. Such an AI is won't need data center with massive training data to be built. The "genesis" code will be something that is capable of figuring out how to go on the internet, and train itself. How do I know this? Because in order to figure out how to solve complex problems (like how to make humans give you control of the nuclear arsenal), is exactly equivalent to a problem of being able to write/read bytes to a file (assuming that file is a socket in Linux) and figuring out how to talk http to get a particular piece of data, without ever being trained on anything internet.

Even more so, there is a fundamental question of whether this genesis code is a P or NP problem in itself - i.e can we generate this code using a training data set, or can it only get created through simulated evolution, much like human brains and capacity for reasoning did IRL.

So as long as everyone keeps talking about number of parameters, transformers, attention, and benchmarks, I promise you we are safe against all powerfull AI.

[anon291]

This has nothing to do with p v np. An AI has no will. Even if superior to our abilities it just does what we tell it.