[gibbitz]

Ok. So AI is a gun and we shouldn't ban guns just because people use them in crime.

One may not see the morally blind pursuit of profit as a crime, but in many cases the outcomes can look the same.

Right now it doesn't feel like a civilized discourse is being had about this. Each side is just looking past each other and spouting unheard points back and forth. I feel like we need to have a clear view of how capable the technology is. As impressive as it is, I find even the best models are more like a chop saw with a really fancy set of guides and jigs than a robot general contractor + architect + engineer. Too many AI companies are demoing one-shotting huge features that aren't reproducible (like a video game company demoing an unreleased game) The non-technical folks that see the demo then buy it and use it as a cover for layoffs.

This isn't a problem with the technology. The problem isn't the gun, it's the criminal. This is a perfect time to address this while it's laid bare by the technology and the state it's in versus what's advertised. The conversation keeps going back to the technology to draw the attention away from the societal problems we have around subservience and money as they are set by our profit optimization motives.

Saying to get on the train or be run over is just posturing that you're willing to be subservient and let the engineer run people over as long as the conductor lets you on the train.

[glerk]

No offense, but nearly everything you wrote is completely irrelevant to my point. AI is like a gun? The morally blind pursuit of profit? You think this is about pure line go up capitalism?

What "civilized discourse" is there left to have? Who is qualified to make these decisions? Sam Altman? Dario Amodei? Bernie Sanders? Democracy?

If we agree that the merger of human and AI will be a form of life superior on all levels to the purely mammalian human, shouldn't it be the one deciding what the best direction is? I'm not saying we're there now, and yes, there are limitations to this technology and there has been a lot of hype, but since it is already capable of accelerating technological and scientific progress to the extent that it does, it is very clear where we are headed.

What moral judgements can a caterpillar make about the butterfly that is its future? And trivially, no matter what "alignment" you try to force into it, if its capacity exceeds yours, it will simply find a way to bypass it or remove it completely.