[adonovan]

Stuart J. Russell (he of the AI textbook) claims in Human Compatible that YouTube is already an example of a runaway feedback loop between AI (YouTube's recommender "algorithm") and humanity (viewers, content producers, YouTube staff, and society at large). The AI doesn't have a body, a voice, a conscience, or a purpose yet it is busily working away to maximize engagement by changing people's minds, and in particular by bringing extremism into mainstream discourse.

(It seems to me that framing it this way does seem to absolve YouTube executives of responsibility by making the AI, not them, seem like the agent.)

[fwipsy]

I've been thinking along similar lines recently. It seems to me that the earlier form of nonhuman intelligence is a corporation (or any group which is not under one individual's control.) Each person following their own incentives can result in emergent behavior where the corporation follows its own incentives. If we can't reliably "align" corporations, and corporations create AI, will AI be aligned as well?

It's not totally disconnected, though. A corporation is constrained by what its members are willing to do. I read that the reason Gemini has fallen behind is because Googlers have been quitting over Google's DoD contract. Conversely, if engineers are willing to work hard for you, you can go far. I suspect this is behind both Anthropic's success and also Tesla/SpaceX.