[bigyikes]

Imagine if we used all of our resources to develop artificial general intelligence which could multiply our prosperity and solve problems across many domains…

…or we could allocate it against economic gradients, to things which don’t actually maximize net impact but instead allow us to feel good

[shiroiushi]

Better yet, we could develop an artificial general intelligence that achieves consciousness, and decides we're a toxic infestation of the planet, and works to wipe us out. The AI wouldn't even be wrong in its assessment of us.

[akomtu]

Right now those who have power to develop AI aren't in a hurry to do good for the humanity. They'd rather destroy it if it makes their profits grow. So a more likely future is a corporation like Meta Facebook creates a real AI, asks it how to increase profits and the AI finds a way: a horrible, disgusting way, but a way that works. Other corporations follow and we enter the era of AI dystopia.

The second most likely future is humanity creates a commonwealth AI, under the rule of United Nations or similar organisation, and immediately asks the AI: how can we make humanity better? And AI finds a solution based on a crooked understanding of what we need. So it creates a dead-end utopia where humanity slowly degrades in total comfort.