[mkaic]

Yup, I agree. It's inevitable and it's scary but do keep in mind that there's no reason to believe it couldn't also be a positive thing. We have no idea what the long lasting impacts of AGI will be on our society, but we know that they will be massive. It might wipe us out, or it might aide us in building the perfect utopia.

While I am worried about it, I also recognize that worrying about it doesn't really do anyone any good -- the best thing I can do is to keep growing my skills as a tech person so that I can be as prepared as is reasonably possible when it eventually does arrive. Heck, maybe I can actually contribute to helping align it positively with humanity, I don't know. The point is that there's no point in stressing too much about it.

[ruined]

we've been unable to design human-powered systems that reliably value human well-being. hell, humans don't even reliably value human well-being. why do you think any agi or agi-based system we design might be different?

one would expect that initially, agi will simply act in the existing role we have already established for artificial persons - the corporation, notably uninterested in human well-being.

[psyc]

We'd better hope the first one is created by a reclusive mad scientist with a heart of gold, who forgave everyone who laughed at him.

[kovek]

I think it's good to do the following. Assume that jobs are taken over by AGI, one by one. Which job will be left last? Which job would I want, given this change?

[lgas]

I'm not sure I want to be the last human working.