[slopinthebag]

I would argue that isn't really a moral argument though, it's rather utilitarian. If someone at OpenAI disagreed with that risk assessment, that's a difference of opinion not a reason to quit and write letters talking about ethics.

Also it's a rather American-centric view. If a Canadian is working at OpenAI, should they care? Or would they care more about possible anti-democratic interference by the American government on Canada?

[totally_human]

Utilitarianism is a moral system. If you disagree with the risk assessment and are utilitarian, you believe that OpenAI got it wrong and is thus doing bad stuff. If a company making bridges, say, or nuclear power plants, was doing risk assessments that appeared to ignore substantial risks in order to get a lucrative contract, I would fully expect engineers to quit and start writing letters talking about ethics.

Agreed on the America-centric view, to an extent. I will note that almost all countries have spied on each other since time immemorial, but serious efforts to spy on their own citizens tend to coincide with uniquely repressive and unpleasant regimes. I think having a norm against spying on your own citizens is good, even if it isn't a perfectly elegant principle. Also, countries can do more damage spying on their own citizens vs other citizens -- as a Canadian, I don't want the American government spying on me, but I'd probably be more worried if the Canadian government was spying on me.