[somenameforme]

The risk in your scenario there is not really coming from AGI, but how selective access to such might enable people to harm others. If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing - short of coming to try to actively destroy you. You'd have millions that would join you, and could turn that vision into a reality regardless of whatever currently powerful entities think about it. So the real danger does not really seem to be AGI, but the restricted access to such.

The focus on "safety" all but guarantees that there are going to be two sets of "AGI", if such is ever to be achieved. There will be lobotomized, censored, and politically obedient version that the public has access to. And then there will be the "real" system that militaries, governments, and influential/powerful entities will be utilizing. You can already see this happening today. There is an effectively 100% chance that OpenAI is providing "AI" systems to the military and government, and a 0% chance that responses from it ever begin with, "As an AI language model..."

[ip26]

If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing - short of coming to try to actively destroy you

I guarantee you there will be someone else with access to comparable AGI with an imperialistic vision who will enlist said AGI to help subjugate you and build their vision.

[somenameforme]

Think about current times, and imagine there was some group with an imperialistic vision set out to subjugate the rest of society to help build their vision. Do you think this group would be more, or less, successful in a time where both society and the imperialists had access to the same AGI? In other words, if we give both groups to the exact same tool, would the capacity/knowledge gap widen or narrow?

[pixl97]

The problem with humanity is we look to solve our inability to create unlimited power before we solve the problem of unlimited greed.

[beanbean01]

If we solved the problem of unlimited greed, we wouldn't need unlimited power in the first place.

[hesayyou2049]

The guys with the medals on their chests using unrestricted AI while the public gets the toothless one. Not too different from the enduring idea that common folk will be eating bugs while they eat steaks.

[incone123]

Hardly surprising that it's an enduring idea. The private jet crowd already tell everyone else to be mindful of our carbon footprints.

[raincole]

I feel you have a fairy-tale definition of AGI. AGI is not literally magic. It's not "genie in a bottle" in the literal sense.

> If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing

It's genie. AGI is a computer program, and it doesn't create an alternative universe for your personal comfort. Or, more specifically, even that kind of AGI is possbile, there will be a weaker AGI before that. The AGI that is not strong enough to ignore physical constraints, but strong enough to fuck everyone up if under control of a malicious entity. And that is what AI safely about.

[nullsense]

All these views on AGI are so self-serving. Which group will have access and control it etc.

It will control itself. We're talking general intelligence. They won't be tools to be used however we see fit. They will be Rosa Parks.

The more I think about "AI alignment" and "the control problem" I feel like most of it is Ph.D math-nerd nonsense.