[stlHusker]

What are the limits of culpability and what are you responsible for? The article is pushing that definition beyond the first order effects of safety. And the proposed solution is that gate keeping is done by the press and the public. How well do you think that will go?

If you write software that replaces someone's job are you "culpable"? If you create a phone and people are killed when someone texts and drives are you "culpable"? If you create a better facial recognition program as part of building a secure access systems and someone uses that system to discriminate are you "culpable"?

I find this unending process of creating "victims" just so they can be "saved" as tiresome.

[jonnycomputer]

I find engineers and entrepreneurs who shirk their responsibilities as moral agents tiresome.

If a thief comes and steals money under the mattress meant to pay for your kid's college tuition, and your kid is dropped because she can no longer pay, your kid's loss is a second order effect of the thief's theft, but I still hold the thief responsible*

* this is a hypothetical story; please don't save that kind of money under your mattress.

[slededit]

Your argument has some basis in law and I do find it compelling. The nuance to my argument is that the original creator did not have nefarious aims. I believe that does make a difference.

[CJKinni]

I think that's fair. I like to consider three factors when considering an original creator:

1. Did they have nefarious aims?

2. Did they reasonably attempt to consider and solve issues surrounding first and second-order effects of their creation?

3. How did they react to previously unknown first and second-order harms when they found out about them?

The first two, I tend to give creators the benefit of the doubt about. But I've become quite dismayed by the responses of creators to the harms of their creations.

[slededit]

I assume the common response to #3 is that the benefits outweigh the costs. Do you not consider that a compelling argument?

[vkou]

Benefits to whom? The engineer and entrepreneur? Or the public at large?

[candiodari]

Benefits to management when the public actually believes the culpable will be punished when shit hits the fan.

Tell me, in the VW emissions scandal, did that happen ? No.

Germany literally changed the laws retroactively to avoid holding their own companies to account. This is technically illegal, but the judicial system REFUSED TO LOOK AT IT.

Given that both management of these firms, and government wants to commit fraud when culpability is assigned ... what point is there discussing the rules ? They won't be applied when it matters.

But of course there are large advantages to people believing they will be applied. Like there are advantages to people not responding to having their wallets taken from their pockets, their children kidnapped for sale into abuse, their houses robbed empty ...

This is the US government in action. Other governments, including European, are worse, not better. These are the people that make, and enforce the rules. Why are we discussing whether the right things will be happening ?

https://www.youtube.com/watch?v=eqBAOX6Qegk

[jonnycomputer]

And if her college boyfriend becomes depressed after she leaves and loses it, gets drunk, and hits someone crossing the road while driving, I still hold the thief to have some moral culpability. Moral culpability is shared and distributed across moral agents in time and space, as some function of causal distance and intent.

It is the second word, intent, that is the hang up. If a person hurts someone but it was not their intent to do so, that person is not typically to be held morally responsible. So, at least in the western concept of moral culpability, intent matters. But... people also have a responsibility, a responsibility to try to anticipate the consequences of their actions. One cannot shield oneself from culpability by intentionally remaining ignorant. I think that engineers avoid thinking about these things, not out of humility, but because its in their interest to do so.

[slededit]

> Moral culpability is shared and distributed across moral agents in time and space, as some function of causal distance and intent.

Absolutely agree.

> I think that engineers avoid thinking about these things, not out of humility, but because its in their interest to do so.

This is perhaps the reason we differ. My experience has been that the vast majority of technologists want to make the world a better place.

I believe you see the industry is filled more with people like Wernher von Braun who want to build something cool regardless of social cost. At the dawn of the social media revolution the dominate theory in tech circles was that information shall set you free. The people making it did not come into the industry for status, being a nerd was still a social negative. Today with the ever rising salaries things are changing.

Perhaps you are right that as the industry matures the sphere of liability should increase. I would still argue that the suggestions in the original article swing that much too far. We shouldn't prevent scientific discoveries merely because engineers may build something bad with the information. On the balance scientific discovery has improved the lives of everyone by many orders of magnitude - even if its distributed unevenly.

[glitchc]

That's what standards are for. The threshold is determined by harm. Software practices are still in their infancy, which is is why it is hard to take them seriously as mature disciplines. Now that software is having a direct impact on people's lives on a large scale, it's time to set some standards, starting with autonomous driving and mass surveillance. Then have people trained to understand those standards and adhere to them as best practices.

[2close4comfort]

If you do it knowingly yes...you have the ability to say no. Do you think you are not a victim, or are you just complacent enough that as long as you have X you are fine and I can going knowing I was a good person. You dont have to create victims you do have an obligation to prevent them.