[slopinthebag]

I can't help but to feel like this is an odd moral position to take. OP is apparently fine with building technology to spy on civilians in other countries, and I don't see a moral relevance to citizenship on this matter. If spying on civilians is fundamentally wrong, it doesn't become OK when the people live in a different region of the world. If spying on civilians is fundamentally OK, then why would there be a moral exception for civilians who live inside the geographical region in which the company is legally registered? Perhaps someone can enlighten me here.

The autonomous killing thing is more reasonable, but still, if you're OK building death technology, I'm not exactly sure what difference having a human in the loop makes. It's still death.

[totally_human]

Spying on your own citizens enables certain sorts of anti-democratic abuses (and has been used that way in the past), so I can understand the specific opposition to it. Put somewhat melodramatically, they're okay with spying but don't want to create self-coup tools.

I agree that the killbots red line is somewhat odd, but I guess you have to draw the line somewhere, and I prefer them having that principle to having no principle at all. (Also, it's possible that the AI insiders understand something I don't about why a human in the loop is important.)

[slopinthebag]

I would argue that isn't really a moral argument though, it's rather utilitarian. If someone at OpenAI disagreed with that risk assessment, that's a difference of opinion not a reason to quit and write letters talking about ethics.

Also it's a rather American-centric view. If a Canadian is working at OpenAI, should they care? Or would they care more about possible anti-democratic interference by the American government on Canada?

[totally_human]

Utilitarianism is a moral system. If you disagree with the risk assessment and are utilitarian, you believe that OpenAI got it wrong and is thus doing bad stuff. If a company making bridges, say, or nuclear power plants, was doing risk assessments that appeared to ignore substantial risks in order to get a lucrative contract, I would fully expect engineers to quit and start writing letters talking about ethics.

Agreed on the America-centric view, to an extent. I will note that almost all countries have spied on each other since time immemorial, but serious efforts to spy on their own citizens tend to coincide with uniquely repressive and unpleasant regimes. I think having a norm against spying on your own citizens is good, even if it isn't a perfectly elegant principle. Also, countries can do more damage spying on their own citizens vs other citizens -- as a Canadian, I don't want the American government spying on me, but I'd probably be more worried if the Canadian government was spying on me.

[insane_dreamer]

I agree mass surveillance is fundamentally wrong, but it's reasonable for people to feel greater responsibility towards the citizens of your own country, and how they are treated by your government.

[slopinthebag]

Maybe, but I still think it's an odd moral boundary to cross. You might feel as though it's fine to spy on Chinese citizens because of the relationship the US and China have, but what about Canadians or Australians or the Brits or any other NATO country? I get it might feel different, but is that really a hard moral line in the sand you refuse to crosss? Idk.

[dummydummy1234]

So the risks are different. If China does mass surveillance on us citizens, then what are the potential downsides? China can do targeted influence campaigns in the us, China can do targeted espionage in the us.

The harms that come from this are against us national security as a whole, the harms are not to individuals and civil liberties. Even if both China and US governments are bad actors, then the fact that China is spying on Americans will not affect Americans civil liberties.

On the other hand if the United States does mass surveillance on Americans, then that can be used by bad actor administrations to suppress dissent, throw people who disagree in prison, suppress speech. Essentially the government has the targeted ability to suppress civil liberties.

So it is very different, because the incentives and potential downsides are different. Similar with companies. Google does not have the ability to lock you up for your Google search, the federal government does (if you are American).

It's the same with Nato/allies, it's not about the country, it's about the spying governments ability and incentives to act on the information.

We don't want the stasi, but imagine a world where the stasi instead had millions of files on Scottish people. What is the worst the stasi could do? What is the worst they would be realistically incentivised to do?

[insane_dreamer]

It's not so much about morals as about the power that the governments have over the people they are spying on. I think it's wrong for the US gov to spy on Canadian citizens living outside the US. But the fact is the US gov has no power over those Canadian citizens outside the US. Whereas the US gov has a great deal of authority over US citizens, or foreign nationals living on US soil, and therefore the information gathered through mass surveillance becomes a much more dangerous weapon.