[dangus]

Of course, the issue is still that this is all science fiction.

In this present moment we only see the power consumption of AI systems rising dramatically.

The underlying silicon chips have slowed in progress dramatically. Moore’s Law is dead.

The AGI if it were to exist today exists on silicon that is crude and wildly energy inefficient compared to organic beings, but we are making a Sci-Fi assumption that it will be able to evolve faster design better despite this massive inferiority in its hardware. IMO this is like saying “Hey my dog learned to roll over, sit, and fetch today! At this rate he’s on track to design a better sports car than Enzo Ferrari!”

Even the assumption of an adversary is a major assumption. If the majority of humans on earth can be goaded into believing that some random dudes named Muhammad/Jesus are the most important prophet/literally god, how hard could it be to convince a computer program that humans are infallible gods that must be protected at all costs?

ChatGPT already won’t let you query illegal stuff as an basic built-in feature, and all the AGI proponents think that somehow the tech will somehow just lose that basic feature and build up a robot army to turn society into The Matrix. To me, that’s kind of like saying that Microsoft Word will lose its spell checker someday.

[asacrowflies]

I think you are misinformed on what the term AGI means. All the things you say it can't or shouldn't be able to do. Are the definition of AGI. and you seem incapable of addressing to the point the AGI will be smarter and more manipulative than us with the ability to self improve much more rapidly than human generation evolution and growth. This needs to be thought of more like micro organism containment .

[dangus]

I have addressed it: AGI hasn’t been proven to exist or be possible to exist. Like I said, science fiction.