Hacker News new | ask | show | jobs
by grellas 2998 days ago
Where does one draw a line these days among the personal, the moral, the legal, and the political?

The military application in question is legal and is approved by a duly elected government that supports it politically. In earlier days, employees generally would see this as just doing their jobs in developing technology that their employers wanted developed and would not concern themselves about ultimate uses and applications. In other words, doing your job is personal and, as long as you do it honestly and work hard, you should not be faulted for doing it as requested by your employer. That was always the standard. What then is the new element from which this sort of employee-driven demand arises? Is it morality? In other words, if I help develop A.I. that can be used for all sorts of things, one of which happens to be military-related, is the effort "evil" if the employer for whom I develop it agrees contractually to provide it to the government for a wartime/military use that can kill people? Do I really make a difference for the good if I convince my employer not to do this if all this means is that the company down the street gets the contract and the military gets the same results, albeit from a different vendor? If this is so, then I assume that you as an employee can make no practical difference in making the world better by insisting that your employer forego this particular form of contracting opportunity. If you succeed, your employer misses an opportunity but the evil you see being released into the world still gets released. It just means that you do not personally contribute to the development effort by which it is made possible.

Of course, it might theoretically be possible to persuade all persons working in the field of A.I. to ban further work that directly helps the military. But that would seem a practical impossibility. Many people in all countries believe that military technology of all kinds is proper, legal, and politically supportable for purposes of self-defense or for some other overriding purpose they deem proper. And certainly, there are bad people throughout the world who are eager to use any technology that comes their way for overtly evil purposes such as misuse of an atomic bomb. Unless and until human nature is fundamentally transformed, that will never change.

So, what is the answer in a country such as the United States where people and companies have the freedom to develop A.I. for any lawful purpose and where some inevitably will do so for a military purpose of which you disapprove?

You are then left only with a political solution: use political means to gain control of the government and the military and apply the force of law to ban the military use of which you disapprove.

So this is either a personal act of futility by the Google employees or it is a case by which they cannot separate the personal from the political and thereby insist that their employer sacrifice particular economic opportunities to ensure that your personal actions do not support a political outcome of which you disapprove.

Even then, does this mean that your employer should cease working on A.I. altogether? For, just as cash is fungible, so too is technology. Every improvement you make in A.I. might have an immediate use of x for your employer but, as humans collectively do this for all sorts of improvements, the results are there for the taking in the future for military applications of all kinds. In other words, you cannot put your improvement in a box or control it so as to limit its future uses (at least not in a free society). The computing technology of recent decades undoubtedly has bettered many aspects of life but it has also greatly magnified the lethality and utility of military applications so as to make the world far less safe. And this was inevitable unless a supervening agency were to have used forcible and totalitarian means to suppress such technological development from inception. Since no such supervening agency existed or even can exist in a free society, does this mean that all engineers and technical developers have blood on their hands because, ultimately, things they have done were used for applications of which they disapproved? Of course it does not. Nor would people today working on A.I. be held morally or legally responsible for ultimate downstream uses made of their work of which they would not morally approve today.

But this brings us back full circle. In the long run, you cannot stop such uses (or misuses) made from your technical development work. Nor can you be held responsible for them even though you contributed to them in some remote degree through your work efforts. Why then should it make a difference if your direct work efforts for a company like Google are applied to a military application of which you do not approve but which is legal, politically approved by the governing authorities, and will happen anyway regardless of whether Google is involved?

The puritans of old tried controlling the morality of others by shunning and shaming and doing it to an extreme degree. They failed miserably in their efforts because humanity is what it is and followed its own course without regard to external religious constraints.

This sort of effort by Google employees is obviously different in that it is not religiously driven but does it amount to anything more than a shunning-and-shaming method for trying to impose one's sense of morality on others by signaling that this way lies righteousness and everywhere else lies evil?

If this is what "don't be evil" now means, then Google will need lots of help going forward because every cause under the sun can be used in the same way to shun and shame. We then have management by a corporate board as may be swayed to and fro by any organized protest of the moment.

Whatever this is, and however it might be defensible in "sending a message" or whatever, it is a sure way to put a company at a competitive disadvantage while accomplishing nothing practically. It may further political goals but, if those are the goals, better just to try to advance them directly and not by attempting to shun and shame your employer (and your co-workers who may disagree with you) into submission. The personal need not be political. If it does become that way, a new form of puritanism will hold full sway to the detriment of all.

14 comments

This argument is not only extremely lazy but positively representative of that large quagmire of social and communicative zeitgeist that Alan Kay calls "the bell curve of normality".

"It's hard to define things, so why bother at all?" ... "Attempting to circumscribe the affects of your work is difficult, so why have a moral stance at all?" ... dancing through loaded assumptions like "duly elected" and "democracy", finally concluding with a tired crescendo of capitalist "competitive disadvantage".

I would merely counter: if we are the future, then we can't all be lazy sods, especially those of us empowering the greatest systems, information and power structures on the planet. Give a damn, it's your moral duty. Intelligent people recognize this.

> In earlier days, employees generally would see this as just doing their jobs in developing technology that their employers wanted developed and would not concern themselves about ultimate uses and applications.

This is a false history. In the 1960s the U.S. was full of young people questioning whether they should just do as they were told and be a good, dutiful employee.

Since then there has been a massive campaign to roll back what was called “Vietnam syndrome” — the idea that you should consider the morality of your actions and contributions to society, not mere legality. Hence all the passionate Hollywood WWII dramatizations, the Greatest Generation, etc., portraying war as a tough but noble effort in which we must all unquestioningly sacrifice for the greater good — de-emphasizing much of the horrendous atrocities that have been perpetrated by the U.S. military in Vietnam, Iraq, support for murderous dictatorships in Central America, Indonesia, and so on.

Noam Chomsky has written extensively on this. One essay in particular is called The Backroom Boys — a reference to the chemical engineers at Dow who developed napalm within the relative peace and equanimity of a laboratory.

> Whatever this is, and however it might be defensible in "sending a message" or whatever, it is a sure way to put a company at a competitive disadvantage while accomplishing nothing practically… If it does become that way, a new form of puritanism will hold full sway to the detriment of all.

Whatever your opinion of the anti-war activists of the 1960s, puritans they were not.

>there has been a massive campaign to roll back what was called “Vietnam syndrome” — the idea that you should consider the morality of your actions and contributions to society, not mere legality.

Is that what is typically meant by "Vietnam syndrome"?

https://en.m.wikipedia.org/wiki/Vietnam_Syndrome

Also, isn't it a stretch to say Hollywood movies about WWII propagandize the idea one should only consider the legality of one's actions rather than the morality? If anything I would think most films attempt (in a sappy and trite way) to defend the rightness of the Allied cause.

Didn't say that. WWII was a case where legality and morality overlap more conveniently which is why it is a favored topic. Still debatable in many aspects e.g. Nagasaki, firebombing cities, not bombing the concentration camps, etc., but in comparison to say Vietnam, you see the point. You don't see too many films these days celebrating the nobility of unquestioningly doing one's duty to support the righteous fight in Vietnam.
Ok, I understand a little better now. I still think this "legality" line of argument is a bit of a red herring and I'm not sure where the poster you responded to got his ideas about how things were "in earlier days" and what "was always the standard."

I think it's probably fair to say that Vietnam was an eye-opener that shook a lot of people's trust in the wisdom of our society's leadership in general. And that a segment of society nevertheless responded like Kissinger by doubling down and shaming the doubters.

Your discussion of "legality" here is U.S.-centric. Google has offices in Germany, France, Poland, Russia, Turkey, U.A.E., India, and China, to name a few. How would you feel if Google worked on military technology for those countries? Would you point out that it was inevitable that their militaries would seek ways to use A.I.? Would you point out that the Turkish armed forces' activities are permitted under Turkish law? Would you lament the inability of Google employees to separate work from politics?

Google has users in almost all countries, and even our friends in other liberal democracies do not see the U.S. military the same way we do. Perhaps some Google users' family members have even been killed by the U.S. military. This presents a perfectly reasonable business reason (one that has nothing to do with "the personal, the moral, the legal, and the political") for Google to turn down AI drone contracts.

I am from one of the country you listed above and it seems perfectly fine to me that Google comply with existing laws of respective countries even if they are in contradiction to US laws.
FWIW, I am American, and I would absolutely object if Google were building weapons for the Chinese military. I would stop using Microsoft products if it turned out they supplied weapons for the Russian takeover of Crimea. I would delete my Twitter account if they were found to be building special-purpose propaganda tools to aid Turkey's Erdogan. Etc.

Complying with laws in another country is one thing. Working with a military, which necessarily has implications beyond that country's borders, is another. And of course even here there are different degrees. You can build a general-purpose secure email client and sell it to a country's military, or you can design their bombs. Where the line is I'm not sure, but at some point your activity is inherently violent, inherently adversarial to some fraction of people in the world.

I am more conflicted about at least one person I knew committed suicide because IT automation provided by Google eliminated his job. Or in general IT/industrial automation wreaking havoc on my highly populated but poor nation. Now is it just the price of progress as people here would say or should Google employees be held morally responsible for causing destruction of livelihood for many a people.
I bet neither China, nor Russia, nor the US would allow foreign citizens in foreign countries develop any serious military technology for their armies. Even their own citizens would be checked in detail before being allowed to work on it.

If a technology is not under that kind of scrutiny, chances are its military applications are... far-fetched.

Lots of military technology is traded between countries all the time, tanks, planes, guns, boats, missiles, etc. China just bought a bunch of Su35 jet fighters from Russia.

http://nationalinterest.org/blog/the-buzz/chinas-air-force-m...

We used to have a moral military, we championed the banning of weapons such as gas, bio, nuclear, mines, etc and we established international institutions and protocols to stop their spread. At some point we turned evil and have been destroying the foundations of international cooperation that we built and are building and spreading new categories of weapons without concern or moral debate. We are the bad guys now.

We are starting to see the shape of the AI driven future and it is not pretty. Autonomous drones and other robots and surveillance patrolling systems establishing strict inescapable authoritarian control.

The elite employees at the global AI leader are best suited to see the coming dangers and are sounding the alarm bells. The outlook is bleak but moral engineers are going to be one line of defense in this fight. And I hope they keep doing it.

The US has never signed the Ottawa treaty, banning the use of land mines. The US do however claim to abide by it, outside of the Korean peninsula. The US never signed the treaty on cluster munitions either, which have many of the same humanitarian issues that mines possess.
>if I help develop A.I. that can be used for all sorts of things, one of which happens to be military-related, is the effort "evil"

There's a famous quote for this:

It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.

https://en.wikiquote.org/wiki/Nathaniel_Borenstein

Of which I believe the meaning is yes, it's evil. It's handing a toddler a loaded gun sort of evil. If you DestroyBaghdad, you've limited the harm your program can do to what is specifically required by the situation. DestroyCity is easily misused in the wrong hands and should be carefully considered by ethical programmers.

Doctors solve this by disallowing unethical members of their profession to legally practice. Programmers should consider becoming an ethical profession, because depending on others in the field to do the right thing and police themselves hasn't been working out.

It's relatively easy to prevent the unlicensed practice of medicine. But anyone can buy a computer and start programming. There's no practical way to require that all programmers adhere to a code of professional ethics.
I completely agree with the loaded-gun metaphor, but doctors are a very different kettle of fish.

Doctors are healers. The Hippocratic oath - "do no harm" - is the logical conclusion of the practice of medicine. Medicine heals, which is the opposite of causing harm. Avoiding harm is the only consistent metric of success, which explains the oath's persistence for millennia.

Can you think of a consistent, concrete set of ethics that would draw unanimous support among programmers?

I think healing has less to do with it than liability. Snake oil salesmen used to be a thing.

What currently sets programmers apart is the lack of liability. Programmers write their own get out of jail free cards. We call them EULAs.

If a doctor screws up and leaves a clamp inside you after surgery, he is sued. If a programmer screws up and leaves a debugging backdoor in a shipped product, nothing.

>Can you think of a consistent, concrete set of ethics that would draw unanimous support among programmers?

I think if programmers can't come to a consensus on that answer, then legislators will do it for them.

If you look around, we're actually witnessing this happening right now. Populist anger has erupted after Equifax, Cambridge Analytica, and Uber. NYT opinion pieces call for changes in liability law around programming.

https://www.nytimes.com/2017/09/11/opinion/equifax-accountab...

And it's not just talk. Changes have already started. Section 230 was recently modified to make small changes in liability of web hosts. In response, Craigslist went full nuclear option in protest and dropped their Personals section. Almost nobody noticed, which means in the next round, law makers will be much more bold in applying more liability to the businesses of programmers.

Google's "Do no evil" was the closest thing I think we've witnessed to a Hippocratic oath for programmers. That's long gone now. Now it's all jerk tech, exploit your users for content and then demonetize them with no recourse or redress.

I don't think the west can get any wilder, so the pendulum is going to go against us from here on out. Programmers should be getting ahead of this, but like all dumb humans, we will sit stupidly. We will only react to immediately obvious consequences instead of preparing for the storm on the horizon.

There's a lot to unpack here, but it seems that the gist of it is that things that happen in our society are because humanity is some kind of untamable animal, and that we should all just resign to letting it run wild as it does.

Do you not believe that society is only the sum of its parts? Do you not believe that the mathematics of society can be changed, the more parts of the equation object to letting their talents be used for unscrupulous goals?

I would point you towards any cultural shift in modern society, and how it began—usually, as the imbalance of classes further divides, until one class can't tolerate it any further, and uses what power they have to reset the scales. That is what is happening today, and it isn't a fluke of the short attention span of the beast of humanity. It is a conscious, concerted effort of people in this country who are tired of existing in a system devoid of morals. And to frame it as something like embracing the status quo, or becoming a puritanical society, is simply a false dichotomy.

In earlier days, employees generally would see this as just doing their jobs in developing technology that their employers wanted developed and would not concern themselves about ultimate uses and applications.

You might want to read some history. E.g. history of the development of nuclear weapons.

>In other words, doing your job is personal and, as long as you do it honestly and work hard, you should not be faulted for doing it as requested by your employer.

This sounds remarkibly similar to the "superior orders" defense given at Nuremberg [1]

(Those who gave it were hung from the neck until dead.)

[1] https://en.wikipedia.org/wiki/Superior_orders

I really appreciate your thoughtful comment!

But I also wonder, if your employer asked you to provide some legal cover for something you found unconscionable -- like maybe draw up incorporation documents for organizing sex tourism to a place where it's both legal and likely to involve slavery -- would you subscribe to the same argument about how you should just do your job and not ask questions, how someone else will do it if you don't, and dutifully provide the legal services?

And if you think that hypothetical scenario is meaningfully different from this one, could you describe how? (I don't mean to try to back you into a rhetorical corner -- I'm genuinely interested in your response.)

> The military application in question is legal and is approved by a duly elected government that supports it politically. In earlier days, employees generally would see this as just doing their jobs in developing technology that their employers wanted developed and would not concern themselves about ultimate uses and applications. In other words, doing your job is personal and, as long as you do it honestly and work hard, you should not be faulted for doing it as requested by your employer.

This is basically the "just following orders" defense. Your argument rationalizes doing nothing.

As a moral person I think you basically have two choices: either don't work on things you think can be used for evil or if you do work on those things step up and make sure they are used responsibly.

It can be seen as a message (as you say) to a wider populace that this is all getting in a wrong direction. If government is hiding from its own populace what its doing with drones, who should raise this issue? Why not people working on the drone program (regardless of where exactly). It's much easier for newcommers than incumbents I'd say, so it make sense some Google employees would be ones to protest.

You never know what will be the initial trigger for change, who will be inspired or whatever. Take this guy: http://www.ecns.cn/2018/01-02/286632.shtml Why should some basketball celebrity and his campaign have so much effect?

Don't forget that Google is a multinational corporation, whereas there is no multinational government with political control by the people.

Google supporting American military may have negative effects on Google the corporation, especially considering that the rest of the world is a bigger economy than the US.

> Don't forget that Google is a multinational corporation

Google (and it's parent, Alphabet) is a US corporation, predominantly controlled by a pair of American individuals, with overseas operations and subsidiaries.

It's “multinational” in much the same way that the CIA is.

There are still people living who, in the middle of the last century condemned other people to death for only doing their jobs. Of course, a future AI may decide to simulate our reality and torture you for a subjective eternity if you do not do everything in your power in this reality to bring that AI into existence. You plays your cards and you takes your chances, but in general its simpler if you try to do what you honestly believe is morally right. And if you honestly believe that the right thing to do is pummel our society with shitty advertising then please do your very best.
Thank god, some sense! It was driving me crazy with how many irrational positions that people in this thread are holding without considering their orientation to the product (AI) and their relationship to the employer.

Something I would add is that a lot of people don't understand how fundamental military R&D is to the collective progression of knowledge and technology. Take almost any common technology that we use today (computers, gps, rockets, airplanes, cell phones, radios, the internet, etc) and you will find it came from military R&D and use in war.

Since AI and all its related parts are the new technological hotness, to put it mildly, it only makes sense that Google, one of the companies on the forefront of this technology, would work with the government/military to do research and find ways to apply it with their scope.

Part of the reason so much stuff comes out of military research is simply that the military has such a huge budget it can spend it on R&D.

There are definitely useful dual use technologies. But there's also a lot of military research that is almost strictly for military. Nuclear weapons research goes beyond the stuff needed for power generation, for example. Money that could have went into research in more stable, safer power generation instead went into how to make nukes small enough to be used by infantry on the battlefield

The military helps a lot by being a huge customer for a lot of this tech, but we can also cut out the middle man and just spend on r&d directly in some cases!

By discussing the morality of weapons research in a world where we already have civilization-ending technology ,we can maybe reorient ourselves to spending directly on progress in more cases. Without needing to have a military application to justify it.

This happened in the past with the neutron bomb, perhaps it can happen again with tech that could be used to help solidify a police state

There is no logical connective in your implied argument. Just because A is trying to buy from B does not mean that B should sell things to A, even if A's spending would increase the quality of B's product.
Since we live in a market driven economy, the government (A) will always find a way to buy what they want. So if Google (B) doesn't supply the demand, the customer will just go somewhere else. So it is completely illogical for Goolge to move away. They are an extremely powerful business and are subject to the rules of business.