[GordonS]

A bit OT, but I wonder how I'd feel if I was offered a job working on software for missiles.

I'm sure the technical challenge would be immensely interesting, and I could tell myself that I cared more about accuracy and correctness than other potential hires... but from a moral standpoint, I don't think I could bring myself to do it.

I realise of course that the military uses all sorts of software, including line of business apps, and indeed several military organisations use the B2B security software that my microISV sells, but I think it's very different to directly working on software for killing machines.

[dahart]

I had a family friend who worked on missiles and drones and other defense systems. He was really one of my dad’s running buddies, and he was a super nice guy, had 4 kids, went to church, etc.

One day, I believe during the Iraq occupation, maybe ~12 or 13 years ago, I asked him very directly how he felt about working on these killing machines and whether it bothered him. He smiled and asked if I’d rather have the war here in the U.S.. He also told me he feels like he’s saving lives by being able to so directly target the political enemies, without as much collateral damage as in the past. New technology, he truly believed was preventing innocent civilians from being killed.

It certainly made me think about it, and maybe appreciate somewhat the perspective of people who end up working on war technology, even if I wouldn’t do it. This point of view assumes we’re going to have a war anyway, and no doubt the ideal is just not to have wars, so maybe there’s some rationalization, but OTOH maybe he’s right that he is helping to make the best of a bad situation and saving lives compared to what might happen otherwise.

[alex_young]

Costa Rica hasn’t had a standing military since 1948. They are in one of the most politically unstable parts of the world and do just fine without worry of invasion.

The US hasn’t been attacked militarily on its own soil in the modern era.

The US military monopoly hasn’t prevented horrific attacks such as 9/11 executed by groups claiming to be motivated by our foreign military campaigns.

I think there is a valid question about the moral culpability of working in this area.

[3pt14159]

It's a valid question, but realistically if Costa Rica were invaded a number of countries would step in to help them. I love Costa Rica, it's one of the most beautiful countries I've been to and I do appreciate the political statement their making, but at the same time they're in a pretty unique situation.

As for the ethics of working on weapons, I think there is a lot of grey when it comes to software. It tends to centralize wealth, since once you get it right it works for everyone. It tends to be dual use, because a hardened OS can be used for both banks and tanks. Even developments in AI are worrying because they're so clearly applicable to the military.

Would I work on a nuclear bomb? No. Would I work on software that does a better job of, say, facial recognition to lessen the likelihood of a predator drone killing an innocent civilian? Maybe. It's not an all or nothing thing.

[kragen]

In the last 40 years, Panama and Grenada were invaded, Honduras had a coup, Colombia had a civil war, Venezuela is currently having a sort of civil war, Nicaragua's government was overthrown by a foreign-armed terrorist campaign, and El Salvador's government sent death squads out to kill its subjects. Nobody stepped in to help any of them except Colombia. Why would Costa Rica be different?

> Would I work on software that does a better job of, say, facial recognition to lessen the likelihood of a predator drone killing an innocent civilian?

The logical extreme of this is Death Note: the person who has the power simply chooses who should die, and that person dies, immediately and with no opportunity for resistance and no evidence of who killed them. Is that your ideal world? Who do you want to have that power — to define who plays the role of an “innocent civilian” in your sketch — and what do you do if they lose control of it? What do you do if the person or bureaucracy to which you have given such omnipotence turns out not to be incorruptible and perfectly loving?

I suggest watching Slaughterbots: https://m.youtube.com/watch?v=9CO6M2HsoIA

[dahart]

> The logical extreme of this [...] Is that your ideal world?

Clearly not. Would you please not post an extreme straw-man and turn this into polarizing ideological judgement? The post you’re responding to very clearly agreed that war is morally questionable, and very clearly argued for middle ground or better, not going to some extreme.

You don’t have to agree with war or endorse any kind of killing in any way to see that some of the activities involved by some of the people are trying to prevent damage rather than cause it.

Intentionally choosing not to acknowledge the nuance in someone’s point of view is ironic in this discussion, because that’s one of the ways that wars start.

[kragen]

You assert that "software that does a better job of, say, facial recognition to lessen the likelihood of a predator drone killing an innocent civilian" is "middle ground", "not going to some extreme", "trying to prevent damage", and "nuanced".

It is none of those. It is a non-nuanced extreme that is going to cause damage and kill those of us in the middle ground. Reducing it to a comic book is a way to cut through the confusion and demonstrate that. If you have a reason (that reasonable people will accept) to think that the comic-book scenario is undesirable, you will find that that reason also applies to the facial-recognition-missiles case — perhaps more weakly, perhaps more strongly, but certainly well enough to make it clear that amplifying the humans' power of violence in that way is not going to prevent damage.

Moreover, it is absurd that someone is proposing to build Slaughterbots and you are accusing me of "turn[ing] this into polarizing ideological judgement" because I presented the commonsense, obvious arguments against that course of action.

[imtringued]

Real weapons are not like that. They are expensive, they can fail to kill their target and they can also cause collateral damage. If death notes were as easy to obtain as guns there would clearly be an increase in homicides but that's not true with military missiles.

The Slaughterbots video is absolutely awful. First of all quadrocopters have an incredibly small payload capacity and limited flight time. A quadrocopter lifting a shaped charge would be as big as your head and have 5 minutes of flight. Simply locking your door and hiding under your bed would be enough to stop them. The AI aspect doesn't make them more dangerous than a "smart rifle" that shoots once the barrel points at a target.

Do you know what I am scared of? I am more scared of riot police using 40mm grenade launchers with "non-lethal" projectiles who are knowingly aiming them at my face even though their training clearly taught that these weapons should never be used to aim at someone's head. The end result is lost eyeballs and sometimes even deaths and the people who were targeted aren't just limited to those who are protesting violently in a large crowd. Peaceful bystanders and journalists who were not involved also became victims of this type of police violence. [0]

[0] https://www.thelocal.fr/20190129/france-in-numbers-police-vi...

[kragen]

Before you had posted your comment, I had already explained in https://news.ycombinator.com/item?id=22394213 why everything in it is wrong, except for the first line.

As for the first line, you assert that real weapons are expensive, unreliable, and kill unintended people. Except in a de minimis sense, none of these are true of knives. Moreover, you seem to be reasoning on the basis of the premise that future technology is not meaningfully different from current technology.

In conclusion, your comment consists entirely of wishful and uninformed thinking.

[3pt14159]

Eh, there is a difference between the examples you've sited and Costa Rica. They're an ally of the US and a strong democracy focussed on tourism.

> The logical extreme of this is Death Note

I don't really deal with logical extremes. It leads to weird philosophies like Objectivism or Stalinism. In international relations terms, I'm a liberal with a dash of realism and constructivism. I don't live in my ideal world. My ideal world doesn't have torture or murder or war of any kind. It doesn't have extreme wealth inequality or poverty. Unless this is all merely a simulation, I live in the real world. Who has the power to kill people? Lots of people. Everyone driving a car or carrying a gun. Billions of people. It's a matter of degree and targeting and justification and blow-back and economics and ethics and so many other things that it's not really sensible to talk about it.

I'm familiar with the arguments against AI being used on the battlefield, but even though I abhor war, I'm not convinced that there should be a ban.

[dahart]

Of course there is a valid question about the morals of war technology. You are absolutely right about that, and I am not even remotely suggesting otherwise. Like I said, I don’t think I would ever choose to work on it.

There’s a vast chasm in between right and wrong though. There can be understanding of others’ perspectives, regardless of my personal judgement. And there is also a valid question and tightly related question here about the morals of mitigating damage during a military conflict, especially if the mitigation prevents innocent deaths. If there’s a hard moral line between doctors and cooks and drivers and snipers and drone programmers, I don’t know exactly where it lies. Doctors are generally considered morally good, even war doctors, but if we are at war, it’s certainly better to prevent an injury than to treat one.

The best goal in my opinion is no war.

[prostheticvamp]

The US was last attacked in living memory; Pearl Harbor survivors still number > 0.

I will leave the WTC attack on the table, as I’m not interested in a nitpicking tangent about what constitutes an attack in asymmetric warfare vs. “terrorism.”

“The modern era” is usefully vague enough to be unfalsifiable.

[samatman]

In practice, Costa Rica has a standing military. It's just the US military.

Due to the Monroe Doctrine, this is a rational stance for Costa Rica to take. If the US were to adopt this policy, Costa Rica might have to take a hard look at repealing it.

[DavidVoid]

> New technology, he truly believed was preventing innocent civilians from being killed.

Drones and missiles are definitely a step forward compared to previous technology in many regards, but I can't help but be reminded of people who argued that the development and use of napalm would reduce human suffering by putting an end to the war in Vietnam faster.

For an interesting and rather nuanced (but not 100% realistic) view on drone strikes, I'd recommend giving the 2015 movie Eye in the Sky a watch.

Another issue with drone strikes and missiles is "the bravery of being out of range": it's easier to make the decision to kill someone who you're just watching on a screen than it is to look a person in the eyes and decide to have them killed.

[jmpman]

Straight out of college, I was offered a job writing software for missiles. Extremely interesting area, working for my adjunct professor’s team, who I highly admired and whose class was the best of my college career. The pay was on par with all my other offers. I didn’t accept for two reasons.

First, I logically agreed that the missiles were supporting our armed services and I believed that our government was generally on the right side of history and needed the best technology to continue defending our freedoms. However, a job, when executed with passion, becomes a very defining core of your identity. I didn’t want death and destruction as my core. I support and admire my college friends who did accept such jobs, but it just wasn’t for me.

Second, I had interned at a government contractor, (not the missile manufacturer), and what I saw deeply disturbed me. I came on to a project which was 5 years into a 3 year schedule, and not expected to ship for another 2 years. Shocked, I asked my team lead “Why didn’t the government just cancel the contract and assign the work to another company?”, her reply, “If they did that, the product likely wouldn’t be delivered in under two years, so they stick with us”. I understood that this mentality was pervasive, and would ultimately become part of me, if I continued to work for that company. That mentality was completely unacceptable in the competitive commercial world, and I feared the complacency which would infect me and not prepare me for the eventual time when I’d need to look for a job outside that company. As a graduating senior, I attended our college job fair, and when speaking with another (non missile) government contractor, I told the recruiter that I was hesitant working for a his company because I thought it wouldn’t keep me as competitive throughout my career. I repeated the story from my internship, and asked if I’d find the same mentality at his company. His face dropped the cheerful recruiter facade, when he pulled me aside and sternly instructed “You should never repeat that story”. I took that as an overwhelming “yes”. So, my concern was that working for this missile manufacturer, this government contractor mentality would work its way into their company (if it hadn’t already), and it would be bad for my long term career. I wanted to remain competitive on a global commercial scale, without relying upon government support.

[newscracker]

> I came on to a project which was 5 years into a 3 year schedule, and not expected to ship for another 2 years. Shocked, I asked my team lead “Why didn’t the government just cancel the contract and assign the work to another company?”, her reply, “If they did that, the product likely wouldn’t be delivered in under two years, so they stick with us”. I understood that this mentality was pervasive, and would ultimately become part of me, if I continued to work for that company. That mentality was completely unacceptable in the competitive commercial world, and I feared the complacency which would infect me and not prepare me for the eventual time when I’d need to look for a job outside that company.

Software for any system is complex. And it’s quite common for almost every software project to be late on schedule. The Triple Constraint — “schedule, quality, cost: pick any two” doesn’t even fit software engineering in any kind of serious endeavor because it’s mostly a “pick one” scenario.

If you’ve worked on projects where all these three were met with the initial projections, then whoever is estimating those has really made sure that they’ve added massive buffers on cost and time or the project is too trivial for a one person team to do in a month or two.

The entire reason Agile came up as a methodology was in recognizing that requirements change all the time, and that the “change is the only constant” refrain should be taken in stride to adapt how teams work.

[AtlasBarfed]

I vehemently and violently disagree!

The average project achieves 1.5 of the triples.

Here are the true constraints though:

- Schedule - Meets Requirements - Cost - Process - Usefulness/Polish

Yes, usefulness and meets requirements aren't the same thing, and anyone who has done the madness of large scale enterprise software will be nodding their heads.

What really bogs down most software projects is that "quality" means different things to different actors in projects. Project Managers want to follow process and meet political goals. Users want usefulness, polish, and efficiency. Directors/management want requirements fulfilled they dictate (often reporting and other junk that don't add to ROI).

And that I like to say "pick two"

[Razengan]

> he pulled me aside and sternly instructed “You should never repeat that story”

We really need more exposure for the things that people like that want to silence..

[prostheticvamp]

He wasn’t silencing anyone. There were no black suits with billy clubs outside.

He was warning the kid that if he went around repeating that aloud he’d burn himself on the interview trail as someone too naive to tow the corporate line and likely to reveal embarrassing workplace details to outsiders.

He was doing the naive youngster a favor, before he could hurt his own career.

The use of the phrase “people like that” is pretty much always pejorative, in a story where a guy who owes the student absolutely nothing took a moment to warn him “don’t touch the stove, you’ll burn yourself”.

So it’s become a story about government contractors instead of a story about “how I fucked up my job search as a new grad.”

Thank you, random kind recruiter guy.

[jmpman]

Nah, I was smarter than that. I already had multiple offers, and had no intention of working for a government contractor. I mostly wanted to see how this recruiter would react to my flippant statement. If he had vehemently defended his company, it would have implied that the whole government contracting wasn’t quite as disfunctional as I’d experienced. His reaction basically confirmed my suspicions. It was a “don’t talk too loud about what we all know is going on”, along with a “how dare you unmask us”. Sure, it was also a “holy hell, you can’t talk to recruiters like that”.

But, no black suits with billy clubs.

And, I’m not suggesting that anything was out of norm for any of these government contractors. They’re delivering a very specialized service with immense regulations. There are very few companies which can produce the same product, so the competition is low and the feedback loop in procurement cycles is much longer.

[throwaway462564]

> First, I logically agreed that the missiles were supporting our armed services and I believed that our government was generally on the right side of history and needed the best technology to continue defending our freedoms

I hope you are not writing about the US government. I don‘t think the US military can be described as protecting our freedoms after interfering and starting wars all over the world in the past. We are sadly mostly the aggressors and not the defendants.

[jnwatson]

Sticking with a vendor even though they are very late is quite common among even non-government programs.

Big projects are hard and they are frequently late. The fact that it is for the government is largely besides the point.

[tomcam]

Thank you for a nuanced and very well explained set of reasons. This is a difficult subject to handle dispassionately here and you did an admirable job.

[AtlasBarfed]

> our government was generally on the right side of history

Well, we are the victors, so far. But the war against ourselves is going quite well.

[GuB-42]

There are different kinds of killing machines. And accurate missiles are among the least bad.

With the exception of nuclear weapons (that's another topic), missiles are designed to destroy one particular target of strategic importance and nothing more. They are too expensive as mass killing weapons, but they are particularly appropriate for defense.

Without missiles, you may need to launch an assault, destroying everything on your way to the target, risk soldier lives, etc... Less accurate weapons mean higher yield to compensate, so more needless destruction.

War is terrible, but I'd rather have it fought with missiles than with mass produced bombs, machine guns, and worst of all, mines.

[BiteCode_dev]

On the other hand, making killing a target easier to do gives you the incentive to do just this instead of trying to find an alternative solution.

Case in point: currently the country with the best army in the world is also the one going the most at war.

[jpmattia]

> but from a moral standpoint, I don't think I could bring myself to do it.

I've been in a similar situation, and I think there is something important to think about: Assuming you'd be working for the defense of a country with a track record of decency (at least a good fraction of the time anyway), you have to decide what people you want taking those jobs.

Is it better that all of the people with qualms refuse to take the positions? ie Do you want that work being done by people with no qualms? Because that sounds pretty terrible too.

[GordonS]

> Assuming you'd be working for the defense of a country with a track record of decency (at least a good fraction of the time anyway)

Yes, this is the kicker for me. My country does not have such a record. If it did, the hypothetical quandry would still exist, but would be much diminished.

[Twirrim]

> A bit OT, but I wonder how I'd feel if I was offered a job working on software for missiles.

At one stage in my career I had an opportunity to go work for Betfair. I knew several people there and could bypass most or all of the interview process. At the time a rapidly growing on-line gambling company, wasn't quite the major company it is now. They were paying about half as much again over my existing salary, and technology wise it would have been a good opportunity.

I ended up having quite a long conversation with a few co-workers around the morality of it. I was against it, for what I thought were pretty much obvious reasons. The house always wins, gambling is an amazing con built up on destroying lives. I don't want to be a part of that, much like I wouldn't work for a tobacco company, oil company etc. Co-workers were taking what they saw as more pragmatic perspective: Gamblers gonna gamble, doesn't matter if the site is there or not.

[jmpman]

The reality behind corporate casinos is a bit more disheartening. Using analytics, from their “players club” cards, they know what zip code you’re from, and based upon that, they approximate your income. They know that if you lose too much, then your wife isn’t going to allow you to return to Vegas. Each income level has a certain pain threshold for how much you can lose in a year. The casino’s work very hard to ensure you don’t go over that limit.

If you’re on a bad losing streak, they’ll send a host over to offer you tickets to a show or a buffet. The goal is to get you AWAY from gambling. They know you’re an addict, but want to keep the addiction going.

That’s where they cross the ethical line.

[enriquto]

> A bit OT, but I wonder how I'd feel if I was offered a job working on software for missiles.

Unless you are an extreme pacifist (which is a perfectly reasonable thing to be), you'll acknowledge the legitimate need for the existence of an army in your country. In that case, the army better be equipped by missiles than by youths carrying bayonets. Then, there's nothing wrong in providing these missiles with technologically advanced guiding systems.

On the other hand, if I worked in "algorithmic trading" or fancy "financial instruments" I would not be able to sleep at night without a fair dose of drugs.

[GordonS]

It's not that I'm a pacifist, but more that I don't trust my government (UK, but I have the same issues with the US gov too) to do justifiable things with them.

If they were for defense only, I might be able to do it. But instead they are sold to any government with the means to pay, regardless of their human rights record or how they will be used (e.g. Saudi). Aside from selling them on, they are used in conflicts that are hard to justify, beyond filling the coffers of the rich and powerful. Take the latest Iraq war for example: started based on falsified evidence, hundreds of thousands dead, atrocities carried out by the west, schools bombed, incidents covered up...

Given these realities, I just couldn't do it.

My original musing was more thinking along the lines of an ideal world, where I trusted my government; I'm still not sure I could do it.

[fancyfredbot]

I suppose we have the 2008 crisis to thank for creating a popular view that finance is an entirely morally corrupt industry. Perhaps it's not surprising given the role "fancy financial instruments" played there. All the same, it strikes me as strange to find moving risk around to be more morally difficult than designing a missile - moving risk around is at least sometimes straightforwardly beneficial for everyone involved, a missile strike less so...

[cushychicken]

I recently interviewed, and was offered a job at, Draper Labs in Cambridge MA.

The technical work was super interesting. Everyone I spoke to was plainly super sharp, and not morally bankrupt. I fielded similar moral concerns as you, but truthfully, I don't really have much of a personal ethical problem with it. I was a little more concerned at having to explain it to all of my friends, many of whom are substantially more liberal leaning in political views than I am.

Perception, and the pay cut I'd have to take from my current work, ended up being the major things that stopped me from taking it.

[TedDoesntTalk]

First time I’ve heard of someone accepting or not accepting a job based on peer perception. Maybe you should re-evaluate who your peers are if they can’t accept you for your career choices?

[cushychicken]

That's a pretty reductive and reactionary comment, and I feel like I'm succumbing to some of my worst impulses by responding to it four days later. But, clearly, some point in there has touched a nerve, so here goes:

For one thing: it missed (or ignored, but I'll default to "missed") my point about the large pay cut involved.

For another: it stack ranked peer perception as being more important in my decisionmaking than the pay cut. My original comment certainly appears to value perception over pay. That's a huge miscommunication of my priorities, and my fault. I wasn't about to take a 30% pay cut. The fact that I'd also have to withstand the negative scrutiny of my friends and family just made it that much easier to decline.

For a third: we all hope to do work that we can be proud of. Part of that pride is to be able to hold up the fruits of our labor to others and take pride in having participated in it. I don't think I could have done that without thinking twice had I taken that job - and I'm not just talking about the security clearance angle here. Dealing with the negative reactions from my friends and family would have been a problem for me. Not the biggest one, but a problem. Acceptance is a big precursor to feeling safe, and self actualized. I feel the acceptance of a group of peers now. I do not want to trade that for more intellectually challenging work with an ethical component that my friends and family find questionable. I'd apply this reasoning in equal part to a role where I was paid more, but just as questionable. (I don't want to get rich building technology that enables the next Bernie Madoff, for example.)

Perhaps I'm of a weaker intellectual constitution than you are to be so easily influenced by the opinion of other people. However, I view that mental flexibility as a strength. I also trust the opinion (and, by extension, the underlying moral character) of my peers, who have been a positive moral force in my life. Hence, it was important to me not to compromise their trust and acceptance of me, and my career choices.

[DavidVoid]

Or maybe they trust/value their peer's judgement despite the fact that they themselves don't have any strong views on the subject?

[TedDoesntTalk]

Relying on other's opinions to make one of the most important decisions in your life -- where you will spend 40 hours or more week -- is pathetic. It's one thing to "trust/value" their peer's judgement, but it's quite another for their opinions to make your decisions for you. Good luck with that. Haha.

[TedDoesntTalk]

To offer a contrarian point of view: I’d jump at such an opportunity... to work in such a technically interesting area AND help keep my country technologically relevant. It’s a no-brainer.

[GordonS]

For me (and I would imagine a lot of people), it's far from a no-brainer.

While it would undoubtedly be interesting from a technical standpoint, there is a serious moral conundrum - even if it was an ideal world where you trusted your government not to start wars based on flimsy or falsified evidence, start wars for profit, or sell weapons to less scrupulous governments.

[TedDoesntTalk]

That’s fine. I am expressing my opinion and you yours. I don’t trust my government with everything, but I’d much rather keep the status quo than see China or another country reign in my lifetime.

[lonelappde]

Would you rather live in a world dominated by USA or USSR or China or Nazi Germany?

Remember you don't get to take away everyone's missiles.

[GordonS]

The world isn't binary; I don't think the options you laid out are the only possibilities.

I take your point though, and I'd have much less of a dilemma if the missiles in question were not to be sold to other governments, and only to be used for domestic defence or a clear world-threat type scenario. Which for many Western countries is of course not going to happen.

[mopsi]

> I'm sure the technical challenge would be immensely interesting, and I could tell myself that I cared more about accuracy and correctness than other potential hires... but from a moral standpoint, I don't think I could bring myself to do it.

Why? The more precise missiles are, the better. If no-one agreed to build missile guidance systems, we'd still have carpet bombing and artillery with 100m accuracy.

[saagarjha]

People might use them less, though.

[TedDoesntTalk]

How naive. More likely they’d be used just as often, but more civilians would die. See the v1 and v2 nazi rockets, for example, which didn’t have any software.

[tomcam]

I mean all due respect with this question. Not an attack. Do you think your country should have such missiles? If not, how would you handle the defense case in which your country is attacked but does not have them? Also note that most of Europe is defended by these missiles made in the USA.

[int_19h]

In the real world, the dilemma is more often, "do you think your country, and all other countries it considers allies, should have such missiles"?

Right now, for example, Saudi Arabia is bombing Yemen with American-made bombs, and Turkey is using German tanks and Italian drones to grind Kurdish towns and villages into rubble in Syria.

[GordonS]

I've mentioned this a few times already in other comments, but I'd be in much less of a quandry about missiles used for defense purposes. Especially so if they could only be used for shooting down other missiles.

[ezoe]

Well, there is a SAM system which is designed to kill missiles, not the humans.

That said, I think any software development which involves the government aren't fun at all for all the bureaucracies and inefficiency.

[lainga]

Cue the picture of the protestors holding a sign reading "this machine kills innocents!"... next to a MIM-104. There are many types of missiles.

[kebman]

Oh, I'm sure you'd make a killing! :D <3