[JohnPrine]

I strongly disagree with you. We have no idea how to align a superintelligence to act for the benefit of humanity. Your plan would only cause faster and faster advances in AI tech without corresponding advances in AI safety research which would be catastrophic

[harpersealtako]

To me the chance of a future superintelligent AI being "catastrophic" is pretty much unknowable (we don't even have a concrete idea of how a superintelligent AI would even work yet!). It could be 99.999%, it could be 0.0001%.

Whereas the chance of a superintelligent AI created by a company being harnessed for personal profits, and that company attempting to maximize its profits by shutting down any competition, potentially by "raising awareness of AI safety concerns", is quite high simply based on our modern understanding of how large, powerful companies operate. And a single company with a monopoly on AI, in sole possession of AI (which you clearly agree can be dangerous) seems even more dangerous.

[focom]

I agree you pinpointed the real issue. People working on AI ethics are more often than not gate keeper to make sure AI is in the hand of the few. They also want AI to follow the leading moral of the day - Western liberal ideas