[onceKnowable]

Yeah but that’s a different argument to what you’re replying to.

If that civil engineer sold or allowed (for any reason from negligence, insufficient security protocols, malice, ignorance etc) blueprints for his design to end up in enemy hands, he most certainly would be held liable.

The thrust of that point is the very idea that software engineers on projects involving user data, regardless of how and what they think they are building, are not simply building “bridges”. They are building systems that have the potential to be misused as mass-surveillance apparatuses. And they need to build their systems accordingly, with appropriate data-security and with acknowledgement that their work could leave them personally liable to action should their work be found to be used as a threat-vector towards their users.

The current attitude of “well people know what they’re signing up for and since they signed the User Agreement, we don’t have any liability here” towards users in the industry is sick.

[smoony]

I literally signed up for HN, something I've been uninterested in doing, just because of this comment.

Saying that engineers need to make responsible decisions in their software design is one thing. But you imply that misuse is a liability for the designer? So if someone takes an engineer's code and modifies it for malicious intent the engineer is liable? That's absurd.

That's like saying the person who designed the airplanes used in 9/11 are partially liable for 9/11.

[onceKnowable]

Absolutely, if that data is stolen due to the designers’ neglect in the security aspects of the systems they signed off as “adequately designed”.

[smoony]

Yup! It just concerns me since this topic has a lot of great discussion but many people are missing the distinction between different types of effects. Sorry if I misunderstood what you were saying! For others who are confused what I mean...

Consider:

The person who makes a car that someone uses to kill someone by running a bystander over. The engineer isn't responsible.

The person who messed up the Prius with the issues where the gas pedal or whatever didn't work. The engineer IS responsible.

The first example is analogous to a software engineer writing image recognition software for a safety app. Then someone maliciously taking the software and using it to discriminate via misuse. The engineer is NOT responsible. If someone takes your software (could even be open source) and applies it or modifies it in a malicious way they are responsible not the engineer.

The second is example is where if an engineer makes a widget app and has a vulnerability but signs off on it, and then widget purchaser credit card info is stolen. The engineer is responsible for negligence and should be liable for the misuse.

[onceKnowable]

No, yeah sorry, I’m obviously being super brief here to just get the point across without writing a thesis on my phone :)

Although, predictably enough, I followed up with more thesis length posts to clarify my point! Namedropping einstein too, who do I think I’m impressing? (But Einstein is relevant to this debate, although he made the crucial mistake of only informing the government and not informing the public. Obviously this was well-intentioned on his part and not malicious at all but it’s a lesson that every developer of new innovations needs to take onboard.)

This isn’t about specifically demonizing “engineers” or innovation. Or even to do with liability where “that engineer developed a car that murdered someone”. Liability laws are robust enough to figure out whether the gas pedal was functioning correctly, the driver is at fault, or if not, the engineer (or rather, the company he works for) is at fault.

This debate is about broader strokes: should developers of new technologies be ethically bound to inform both politicians & the public about their innovations, specifically so that, after a public debate on the potential negative consequences of that technology, the public can then demand their politicians to produce robust laws protecting the public from misuse of those technologies. (I say yes, I know it’s tough because I love just developing cool new shit too, but we developers have got to take a step back and realize the future implications of what we make. Data gathering, manipulation and retention is a ticking time bomb. And who knows what’s next.)

Another example is facial recognition, it’s depressingly funny to see experts who know so much about the technology proclaim it to be no big deal without understanding the ramifications of living in a society where your every movement is tracked and potentially monitored. That is a literal police state, and the US constitution has amendments made hundreds of years ago specifically to outlaw such actions to protect the public at large, both from private companies tracking their movements and to protect the public from the government itself. To have all of that progress in the name of “facial recognition will be awesome because you can login to your phone quicker” is not a good move because once that technology is in the wild, until laws catch up to robustly protect the public, who knows what will be done to the public at large with that technology. Ditto for “who cares about data retention and website tracking, it makes marketing so much easier!” and “don’t worry about your DNA being stored and sold by ancestry sites indefinitely because it’s so cool to know that you’ve got a 7th cousin living on a different continent”.