[Waterluvian]

> what the engineers thought when doing design reviews for a "selective stand down" feature.

Possibly a version of, “I lack the freedom to operate with a moral code at work because I’m probably replaceable, the job market makes me anxious, my family’s well-being and healthcare are tied to having a job, and I don’t believe the government has my back.”

[Aurornis]

From my experience, it’s more likely that the engineers who got far enough in the company to be working on this code believed that their willingness to work on nefarious tasks that others might refuse or whistle-blow made them a trusted asset within the company.

In industries like this there’s also a mindset of “Who cares, it’s all going to corporations anyway, why not send some of that money to the corporation that writes my paychecks?”

[TheNewsIsHere]

I have noticed that in addition to this perspective there are scores of developers who espouse the idea that “we just create, what people do with our work isn’t our business.”

I understand the utilitarian qualities of the argument, but I submit that there’s a reason that capital-E-Engineering credentials typically require some kind of education in ethics-in-design.

[baby_souffle]

> but I submit that there’s a reason that capital-E-Engineering credentials typically require some kind of education in ethics-in-design.

Or said differently: there’s a reason why software engineering jobs pay so well; no mandatory ethics training required!

[triceratops]

No. It's supply and demand.

[code_biologist]

I agree that we're responsible for what we create. I would also submit that corporate culture has been under intense selective pressure over the past 10 years to get good at creating compliance with ethically problematic software projects. I'm curious how many people left Google because they dropped the "don't be evil" motto.

There's lots of carrots (compensation, high quality desk jobs) and sticks (promotion structures, threat of offshoring). The really annoying and egregious aspects of corporate speak are easy targets for ire and take the heat, while the subtle euphemisms make the actual questionable projects easier to live with day to day.

[pixl97]

capital E engineers have numerous other laws that protect their position.

Civil/mechanical/electrical have countless codes that must be followed with the force of law.

When we say we want engineering standards for software developers we are also asking for standards and codes to be applied to software and all that entails.

I'm not saying this is good or bad, just to consider the ramifications of this at all levels.

[_DeadFred_]

This has been 'being considered' my entire career, so since the 90s at least. I have finally determined all the libertarian style 'thinking' over action is just stalling. They have stalled to the point that tech now smells bad to the majority of people, I wonder what comes when OUTSIDE influences decide enough. I feel like tech's 'self determinism' runways is running out and I'm kinda happy for it. Couldn't happen to a more deserving industry.

[petterroea]

I suspect you are right. It reminds me of the whole "at the government you can hack legally" argument used by government intelligence agencies to recruit hackers.

I think a lot of skilled engineers want interesting challenges where they break boundaries, and being in an environment that wants you to break those boundaries allows them to legitimize why they are doing it. That is, "someone else is taking moral responsibility, so I can do my technical challenge in peace"

[zaphirplane]

Do you know of anyone declining to work on a project For ethical in their view ( non military non killing) ?

I’ve led a sheltered life and never met one, people have told me they wouldn’t apply for a role with a company for ethical reasons maybe they even believed they would get the job

[neilv]

I know a lot of people who won't work for some companies for ethical reasons.

Though, sometimes the exact reason is muddied, since companies that are perceived as unethical in how they behave externally are often also perceived as unethical in how they behave towards employees. So you might object on pragmatic grounds of how you'd be treated, before you ever get to, say, altruistic grounds.

Also, sometimes fashion is involved. For example, many people wouldn't work for company X, because of popular ethical objections to what they do being in the news, but some of those people would probably work for an unknown company doing the same things, without thinking much about it.

But often it's just "I don't like what company Y is doing to people, and I wouldn't work on that, even if they treated employees really well, and it was really fashionable to work there".

(See, for example, the people who refused to work for Google after the end of Don't Be Evil honeymoon phase, even though they generally treated employees pretty well, and it was still fashionable to work there.)

[zoltanse]

Sure: A couple of years ago I joined a company doing outsourced system administration. Then it was suggested I should take care of a new client: a manufacturer of weapons with a quite shady reputation. There were already other issues I had noticed. But this was the red flag for me and I left after four weeks. My then team lead was pissed and complained I should have told beforehand that I don’t want to go down that route. But it never occured to me before that to compile a blacklist of things I won’t do. And I had been in business for more than 20 years when that happened.

[chanchowancho]

Yes! I once met a highly paid contract tech lead who had walked out of a lucrative contract with a supermarket after he became aware the new credit card product he was working on was to be exclusively targeted at customers in poor areas.

The moral fortitude on that man!

I applaud his actions, but genuinely do not know if I would have the stones to leave my job if I was in a similar position!

[SpaceNoodled]

Half of it is stones; the other half is cash.

[tyre]

I worked at LivingSocial back in 2012. I was 21 and didn’t know anything about marketing. The pitch was that daily deals helped small businesses get new customers who would then become recurring, which was good. I liked helping small businesses.

Over time I realized that the company knew this wasn’t really true. Daily deal customers weren’t likely to return. They went where the deals were. The influx of cash from daily deals was a marketing expense, almost always at a loss (most deals were 50%+ off and half of the remaining revenue went to LivingSocial), and buyers rarely returned so SMBs would never recoup their loses.

Once I figured this out, I decided to leave even though I would miss my equity cliff by a month. I ended up joining ZenPayroll (now Gusto) early on because they were helping SMBs with a real problem (payroll was a fking nightmare back then.)

[sgerenser]

Even if you had got the LivingSocial equity, it probably would have ended up worthless anyway, right?

[BugsJustFindMe]

Hello. I have. The first time, I was offered a job working on missile guidance systems. I told them I would not work on weapons, so they offered me a job working on something else instead. Then they asked me to move to another project that would require getting government security clearance. I said I wouldn't do that either because I was not willing to make the required promises to my government, so they gave me other projects that didn't require it. It's really not that hard to have a penny's worth of a moral compass if your skill has any kind of value. I think maybe the problem is people who only have value to companies that only hire people without any morals.

[freeopinion]

Beyond that, I now accept that many employers screen candidates with questions like, "Have you ever been fired?". Answering the why with, "I refuse to do things I consider to be unethical" is typically enough to screen you out.

While this can be irritating, I have come to see it as a good thing. It helps me screen out candidate employers. It is taxing to work in an environment that constantly challenges your ethics. Imagine having access to all your customers' supposedly private emails and being tasked with mining them without your customers' knowledge. Imagine being tasked with adding an obscurely worded line item to the monthly bill of all customers that your logging indicates haven't accessed their billing statement in the last 12 months.

Now imagine working at a job where you are tasked to find all customers who haven't used an optional paid feature in the last 12 months and notifying them that there might be an opportunity to reduce the amount you bill them. Imagine working for an insurance coop that actively scours for ways to charge members less money without compromising their protection and without taking advantage of somebody else.

Imagine that your personal life choices automatically disqualify you from exploitative employers and lead you to more fulfilling employment. This is a real thing that many people don't have to imagine. They live it.

[lazide]

The issue of course is what if your personal life choices automatically disqualify you from (defacto) all employers and you end up not even being able to afford a van down by the River?

That is what anxiety based thinking produces.

[fernandotakai]

>Do you know of anyone declining to work on a project For ethical in their view ( non military non killing) ?

o/

i was offered a high paying job, with relocation to a 1st world country (at the time, i was living in a 3rd world country with high murder rates), to a industry that i consider quite shady (and it's not military and not around killing -- i have no issues with both of those). i politely refused.

most of my friends, at the time, told me that they would've have accepted without even thinking, but for me, it's just not worth it.

[mattclarkdotnet]

I know lots of people who had the offer to work in gambling but chose not to take it for moral reasons

[yetihehe]

I had an offer to work in gambling as a young inexperienced student, fortunately they didn't hire me because I was too inexperienced. I can imagine how my career would move if my first working experience was in such company. Some people might be like that.

[Marsymars]

Well kinda trivially, asides from secular ethics, you'll find that typical Muslims decline a number of jobs/projects for ethical reasons.

[_DeadFred_]

As do Christians. Jews. Hindus.

I was asked to help with creating what seemed like a human trafficking app to Christian me, but that to the Muslim founder was 'just an app to get the best payment for an arranged marriage' and just improving something that he said already happened all the time in his culture (he was from Pakistan I don't know if that is actually a thing there or he was just trying to justify his messed up app).

[Marsymars]

> As do Christians. Jews. Hindus.

Yes, absolutely. To elaborate a bit though, if you live in the West, Muslim ethics are more likely to stick out when applied to our regular practices. e.g. I know a Muslim programmer who declined to participate in a project involving billing interest to customers. (Which is decidedly non military and non killing, as posed by the post I was replying to.)

[moron4hire]

I quit a job on contract with a major insurance provider because they asked me to perform a truncate instead of a rounding operation in a formula without any mathematically sound reason for choosing the truncate over the rounding. I figured out they wanted truncating because it would lead to more people being denied flood coverage than rounding would.

[itsdesmond]

This is a real long look in the mirror moment.

[Tepix]

I think most people avoid this situation one step earlier by choosing the company they work for. I.e. do you accept a job in adtech, military, adult industry, etc.

I think pretty much everyone has an internal red line, of course they will vary a lot and may even move over time.

[square_usual]

I've often been contacted by recruiters for companies in the gambling (in India it's called "skill-based web gaming") or the crypto/web3 space, and I've always denied those for ethical reasons.

[Joeboy]

I've dodged multiple work opportunities on ethical grounds, although I can only think of one time where it was a big deal (I think we had to turn down a client because I declined to work on it).

[jbreckmckye]

Sure, it happens all the time. Speaking personally, for example, I walked out of an interview when I realised it was for The Sun's betting site (Sun Bet)

[immibis]

And they are right. It's not like anyone outside of the affiliate marketing "industry" was hurt by this - noting that some of your parasocials are likely to be in that "industry" and so you feel hurt on their behalf.

[hinkley]

Henchmen I think is the word you're looking for.

[furyg3]

I like the idea that what makes someone a 'professional' instead of just an employee is the wherewithal, agency, and expectation to say no to a particular task or assignment.

An architect or engineer is expected to signal and object to an unsafe design, and is expected by their profession (peers, clients, future employers) to refuse said work even if it costs them their job. This applies even to professions without a formalized license board.

If you don't have the guts and ability to act ethically (and your field will let you get away with it), you're just a code monkey and not a professional software developer.

[nosianu]

Maybe when the government and the shareholders start setting an example and hold the bosses and capital owners accountable, and reward instead of punish the whistleblowers, and when their are enough jobs so that losing the one you have is not a problem, moral behavior further down the hierarchy will improve.

[steve_adams_86]

In my experience, sometimes your employer blatantly lies to you about what you're making and how it'll be used. I was once recruited to work on a software installer which could build and sign dynamic collections of software which was meant to be used to conveniently install several packages at once. Like, here's a set of handy tools for X task, here are the default apps we install on machines for QA people, here is our suite of apps for whatever. It seemed to have genuine utility because it could pull data in real time to ensure it was all patched and current and so on. That could be great for getting new machines up and running quickly. Several options exist for this use case today, but didn't then as far as I recall. This was on Windows.

Ultimately it was only used to install malware in the form of browser extensions, typically disguised as an installer for some useful piece of software like Adobe Acrobat. It would guide you through installing some 500 year old version of Acrobat and sneakily unload the rest of the garbage for which we would be paid, I don't know, 25 cents to a couple dollars per install. Sneaking Chrome onto people's machines was great money for a while. At one point we were running numbers of around $150k CAD per day just dumping trash into unsuspecting people's computers.

At no point in the development of that technology were we told it was going to ruin countless thousands of people's browsers or internet experiences in general. For quite a while the CEO played a game with me where I'd find bad actors on the network and report them to him. He'd thank me and assure me they were on top of figuring out who was behind it. Eventually I figured out that the accounts were in fact his. They let me go shortly after that with generous severance.

I don't miss anything about ad tech. It was such a disheartening introduction to the software world. It's really the armpit and asshole of tech, all at once.

[bostik]

> Ultimately it was only used to install malware in the form of browser extensions, ...

Like any other MDM software.[0] Everyone who has been long enough in the infosec industry knows that MDM is fundamentally nothing more than a corporate-blessed malware and spyware package.

In the past 2-3 years the criminal gangs have realised that too. The modern form of socially engineered phishing quite often entices victims to install a legit MDM software package (eg. MS InTune) and hand over their device control for remote management. Why bother writing malware that has to fiddle with hooks to syscalls and screenshot capabilities when you have a vendor approved way of doing the same?

0: https://en.wikipedia.org/wiki/Mobile_device_management

[victorbjorklund]

Those poor guards working in the concentration camps in nazi germany just wanted job security. They can’t be blamed for their actions.

[dbtc]

Also likely, some version of "get dat money"

[autoexec]

I think you can only get away with that excuse so long as you're actively looking for a new job while also collecting data to turn whistleblower (anonymously if need be) once you have one. Ultimately it falls on the employee to do the right thing or get out because they risk being held accountable for what they do. A replaceable employee (which is pretty much all of them) will be especially vulnerable since they can be thrown under the bus with minimal inconvenience to the company.

[BiteCode_dev]

It's a very charitable explanation.

My experience with the people around me who are in this situation is rather either:

- They just don't care. Society and others are not on their radar.

- They don't think it's that bad.

- They think it's not great, but the benefit is too good so they ignore the voice at the back of their head. Or they have a lifestyle and that takes priority.

- They think it's bad, but the friction to live according to their own moral view of the world is higher than their desire to adhere to such a moral view.

When I was 20, I declined interview offers from Facebook and Google. Huge opportunity cost. My friends looked at me like I was dumb.

I have friends regularly coming to me with ideas that are about spamming, selling personal data or basically fraud. They don't see a problem with it.

When you talk to people and say "advertising is basically normalized lying at the scale of the entire society", people just give you a blank stare.

There is no need to look for coercion every time you see something bad to explain it. The human population is diverse and they all draw the line of what's acceptable in different places.

It's not rocket science.

[ocschwar]

This is why we need Professional Engineer licenses for software.

There are times when a product design needs to be reviewed and approved by someone who cares more about his license than about his job. It doesn't happen as often with software as it does with civil engineering, but often enough that it needs to become a thing.

[webstrand]

And what happens when the licensing board gets politically compromised? You cant fix broken incentives by papering over another layer of administration. If the underlying incentives are opposed, the administration layer will be adapted to fit.

Civil engineering licensing works because underneath it all the incentive structure is aligned with the goals of the license. Its not about imposing morals, its about ensuring that buildings and devices are constructed to not fail, and to not fail catastrophically. The motivations of the ones who hire engineers are mostly aligned, they don't want the devices to fail either, and expose them to liability.

Medical doctor licensing also works because the incentives are mostly for patients not to be dying. But in the pharmaceuticals industry the incentive structure is different, where some rate of fatality is considered an acceptable cost of doing business, we see examples of subversion.

Sure software engineering licenses could be a great addition. But alone it will fail unless the incentive structure for those employing software engineers is aligned with the licensing goals.

[pixl97]

This works great inside a country where said software must be written in the country for compliance reasons...

How does is work for a fungible product that can be written anywhere and shipped at the speed of light?

[ocschwar]

The firmware for a diesel engine goes into a diesel engine. The company can be required to get a PE's signify for putting the firmware in. After that, if it's copied elsewhere, that's not their problem.

[jbreckmckye]

It doesn't, but at some point a mature industry + society decides to make that tradeoff.

We can't have it both ways: be essential digital infrastructure, AND move at "the speed of light".

[PunchyHamster]

More like "well, they pay well and it's interesting problem so who gives a fuck"

[cowpig]

Ah yes let's be sure not to judge anyone for anything they do

[asimovfan]

People do not make choices in a vacuum.

[autoexec]

But they still make their choices and should face the consequences of them.

[Zetaphor]

What exactly do you propose?

[Kwpolska]

Death penalty for engineers, and a slap on the wrist for CEOs.

[Spivak]

You can still judge them evil even if the parent was accurate as to the motivations for their actions. Villains are more interesting when they're sympathetic.

You're in the planning meeting discussing this feature, you ask "Hey, are we allowed to do this? I thought stand downs were contractural." and your PM says yes, they got the okay from legal. Now what do you do?

[bryan_w]

> they got the okay from legal.

Now that I could definitely see happening. I would also want that in writing somewhere.

I guess discovery for the impending lawsuits should be very interesting

[rectang]

It’s easy, looking at the current state of affairs, to conclude that ethical behavior is incompatible with capitalist ambition. One might still choose to be ethical nonetheless, but with the understanding that you will be overtaken by those who have made a different choice.