[yummypaint]

It's pretty wild that software is the only "engineering" discipline without any concept of professional ethics or loyalty to human safety that superceeds the whims of the employer.

Why do people think that is? Have there been any attempts to change this from the inside over the past decade? Where are professional associations like the ACM in all of this? It's a shameful state of affairs and reflects poorly on the whole discipline.

People who design bridges and vehicles have real responsibilities and standards they are held to, yet somehow the software that actually runs these things is exempt.

This is how Boeing negligently murdered hundreds of people with MCAS. By taking responsibility for safety away from actual engineers and misplacing it with people who write software.

[salawat]

My ethics curriculum while getting my CS degree basically highlights the last decade+ of big name tech companies as "things that should not be done for obvious reasons". So I avoided all of them, looking for some of the most boring, banal tech jobs/applications of computers to clerical work imaginable.

Imagine my surprise when even most of those jobs were founded on things/goals/ambitions/central risks that were also profound ethical lapses. I've gotten to the point where I'm honestly wondering whether humanity is even ready for ubiquitous computing, to which seemingly the answer lately trends toward "hell no". Great thing to only realize after 10 years in the industry.

Point being, ethics is something self enforced, and we've taken great pains to ensure there is no professional licensure body or anything else around software; in particular because of the fundamental asymmetry created by locking that fire away from mortals behind a bunch of barriers to entry. Those with tech will have an advantage over those without. This is a certainty. The price of that (intentionally opening access to expertise for anyone curious) is what we have now. The tech that gets implemented is a reflection of the collective soul of humanity. If Greed is God, and the Deadline and Sale take priority over Safety, Fitness for Use, Quality, and Characterization of the System-Under-Scrutiny; then previously mentioned lapses in the ethicality of implementations of computer systems are what we're gonna get, and the conscientious objector will just be walked out the door and the next practitioner less afflicted by scruples will be walked in instead.

I'm open, as I've always been to putting a thumb on the scales through greater organization of active practitioners to actually make a means to ensure some subset of systems don't get implemented, but I'm not sure that's the right answer. The right answer is to improve ourselves and our non-computerized ways of life so there is no damn incentive to make the Torment Nexus.

Easy to say right? But that execution... Oofta.

[Apreche]

When I was in college majoring in CS in the early ’00s I took several classes about ethics and technology. I also took plenty of liberal arts and literature classes throughout my entire education, as well as reading many sci-fi novels for pleasure. I learned the lesson of not building the torment nexus early and often. Thus far I have refused employment at any places doing so. I have an ethical and moral code that comes before anything an employer asks of me.

I think the problem is that clearly not everyone took the same path, and did not get the same moral education. I’m not religious at all, but religions do play a part in morally educating people. Not only are fewer people religious, but more of the religions that are popular are outright immoral in their teachings.

For profit companies do not act morally when not required to by law. Remember when Google hired some AI ethicists, and then fired them? Heck, they couldn’t even maintain their elementary school ethical code of “don’t be evil.” Companies who engineer bridges, medical devices, aircraft, etc. are regulated by law. They don’t follow ethical engineering practices of their own volition. They do it only because we force their hand.

We’re starting to see now what it looks like to regulate tech with some of the policies in the EU. While those regulations are flawed in several ways, they are also working properly. GDPR does have positive impact on privacy. Digital Markets Act is forcing Apple to allow side-loading.

We can defeat the torment nexus by simply outlawing it.

[GauntletWizard]

I took the same professional ethics course as you in school (Hi from another RIT anime nerd), and I found that even by mid 2000s, most people coming into the program openly mocked the lessons of Therac-25 and the Hyatt Regency Walkway Collapse as heavyhanded, but a disturbing few viewed these as lessons in how you could cut corners and avoid getting charged.

Google used to have ethicists ethicists and a culture of don't be evil. They were fired by the AI ethicists, who for the most part were among that latter group - Rationalists who were supremely good at rationalizing their choices rather than making good ones.

I don't think that any of the current efforts at regulation are going to work. The latter culture has taken over the institutions, and that's why you see so many baying for the institutions to be taken down.

[evantbyrne]

Hey, first of all sorry to post this so many days late, I just got here looking to see if this article had been posted already.

I think the piece you are missing is that guilds exist to protect workers, not consumers. US software engineers not feeling as though they need the protection of guilds means by extension there are no non-government bodies to enforce codes of conduct. It is also worth mentioning that, although limited to specific high-risk use cases, software engineering is regulated.

[musicale]

> Where are professional associations like the ACM in all of this?

https://www.acm.org/code-of-ethics

However, ACM continues to accept money from companies that routinely and systematically violate this code (for example privacy in section 1.6) and seems to have little interest in sanctioning them.

[musicale]

See also:

> We want you to actively contribute to the discussion of ethics and computing (through the comment section or our survey (https://tinyurl.com/CACM-ethics) because participation is part of the computing profession.

https://cacm.acm.org/opinion/the-future-of-professional-ethi...

[musicale]

> People who design bridges and vehicles have real responsibilities and standards they are held to, yet somehow the software that actually runs these things is exempt.

The most impressive accomplishment of the computing industry is avoidance of any kind of liability.

[shaunxcode]

Because there was a very conscious attempt to destroy cybernetics. Resulting in MIT etc becoming a school that builds human calculators for Harvard/biz grads to utilize as literal machines without souls. <cite>albertm

[blamestross]

I think we need our own variant of the "iron ring" at this point. I wear a similar steel ring on my pinky as an oath reminder.

Its a lot of work to grass-roots something like that, and I don't have the charisma for it.

[shortrounddev2]

I think other engineering disciplines grew up in an era where workers rights and public safety was becoming a big focus of government. Software engineering grew up in a time where we took government for granted; even demonized it

[mindslight]

I agree with the crux of your point, but commonly held engineering ethics are still compatible with building weapons that directly attack other humans. The ethics basically focus on doing right by the employer/society you're working for. And for the surveillance industry, that society is the obscenely rich elites and us plebs are all just the targets.

[musicale]

> commonly held engineering ethics are still compatible with building weapons that directly attack other humans

"Once the rockets are up, Who cares where they come down? That's not my department," Says Wernher von Braun.

https://news.ycombinator.com/item?id=44702782