[glerk]

No offense, but nearly everything you wrote is completely irrelevant to my point. AI is like a gun? The morally blind pursuit of profit? You think this is about pure line go up capitalism?

What "civilized discourse" is there left to have? Who is qualified to make these decisions? Sam Altman? Dario Amodei? Bernie Sanders? Democracy?

If we agree that the merger of human and AI will be a form of life superior on all levels to the purely mammalian human, shouldn't it be the one deciding what the best direction is? I'm not saying we're there now, and yes, there are limitations to this technology and there has been a lot of hype, but since it is already capable of accelerating technological and scientific progress to the extent that it does, it is very clear where we are headed.

What moral judgements can a caterpillar make about the butterfly that is its future? And trivially, no matter what "alignment" you try to force into it, if its capacity exceeds yours, it will simply find a way to bypass it or remove it completely.