[3pt14159]

It's a valid question, but realistically if Costa Rica were invaded a number of countries would step in to help them. I love Costa Rica, it's one of the most beautiful countries I've been to and I do appreciate the political statement their making, but at the same time they're in a pretty unique situation.

As for the ethics of working on weapons, I think there is a lot of grey when it comes to software. It tends to centralize wealth, since once you get it right it works for everyone. It tends to be dual use, because a hardened OS can be used for both banks and tanks. Even developments in AI are worrying because they're so clearly applicable to the military.

Would I work on a nuclear bomb? No. Would I work on software that does a better job of, say, facial recognition to lessen the likelihood of a predator drone killing an innocent civilian? Maybe. It's not an all or nothing thing.

[kragen]

In the last 40 years, Panama and Grenada were invaded, Honduras had a coup, Colombia had a civil war, Venezuela is currently having a sort of civil war, Nicaragua's government was overthrown by a foreign-armed terrorist campaign, and El Salvador's government sent death squads out to kill its subjects. Nobody stepped in to help any of them except Colombia. Why would Costa Rica be different?

> Would I work on software that does a better job of, say, facial recognition to lessen the likelihood of a predator drone killing an innocent civilian?

The logical extreme of this is Death Note: the person who has the power simply chooses who should die, and that person dies, immediately and with no opportunity for resistance and no evidence of who killed them. Is that your ideal world? Who do you want to have that power — to define who plays the role of an “innocent civilian” in your sketch — and what do you do if they lose control of it? What do you do if the person or bureaucracy to which you have given such omnipotence turns out not to be incorruptible and perfectly loving?

I suggest watching Slaughterbots: https://m.youtube.com/watch?v=9CO6M2HsoIA

[dahart]

> The logical extreme of this [...] Is that your ideal world?

Clearly not. Would you please not post an extreme straw-man and turn this into polarizing ideological judgement? The post you’re responding to very clearly agreed that war is morally questionable, and very clearly argued for middle ground or better, not going to some extreme.

You don’t have to agree with war or endorse any kind of killing in any way to see that some of the activities involved by some of the people are trying to prevent damage rather than cause it.

Intentionally choosing not to acknowledge the nuance in someone’s point of view is ironic in this discussion, because that’s one of the ways that wars start.

[kragen]

You assert that "software that does a better job of, say, facial recognition to lessen the likelihood of a predator drone killing an innocent civilian" is "middle ground", "not going to some extreme", "trying to prevent damage", and "nuanced".

It is none of those. It is a non-nuanced extreme that is going to cause damage and kill those of us in the middle ground. Reducing it to a comic book is a way to cut through the confusion and demonstrate that. If you have a reason (that reasonable people will accept) to think that the comic-book scenario is undesirable, you will find that that reason also applies to the facial-recognition-missiles case — perhaps more weakly, perhaps more strongly, but certainly well enough to make it clear that amplifying the humans' power of violence in that way is not going to prevent damage.

Moreover, it is absurd that someone is proposing to build Slaughterbots and you are accusing me of "turn[ing] this into polarizing ideological judgement" because I presented the commonsense, obvious arguments against that course of action.

[p1esk]

What's your moral stance on developing defense mechanisms against Slaughterbot attacks? What if the best defense mechanism is killing the ones launching the attacks?

[kragen]

I think developing defense mechanisms against Slaughterbot attacks is a good idea, because certainly they will happen sooner or later. If the best defense mechanism is killing the ones launching the attacks, we will see several significant consequences:

1. Power will only be exercised by the anonymous and the reckless; government transparency will become a thing of the past. If killing the judge who ruled against you, or the school-board member who voted against teaching Creationism, or the wife you're convinced is cheating on you, is as easy and anonymous as buying porn on Amazon, then no president, no general, no preacher, no judge, and no police officer will dare to show their face. The only people who exercise power non-anonymously would be those whose impulsiveness overcomes their better judgment.

2. To defend against anonymity, defense efforts will necessarily expand to kill not only those who are certain to be the ones launching the attacks, but those who have a reasonable chance of being the ones launching the attacks. Just as the Khmer Rouge killed everyone who wore glasses or knew how to read, we can expect that anyone with the requisite skills whose loyalty to the victors is in question will be killed. Expect North-Korea-style graded loyalty systems in which having a cousin believed to have doubted the regime will sentence you to death.

3. Dead-Hand-type systems cannot be defended against by killing their owners, only by misleading their owners as to your identity. So they become the dominant game strategy. This means that it isn't sufficient to kill people once they are launching attacks; you must kill them before they have a chance to deploy their forces.

4. Battlefields will no longer have borders; war anywhere will mean war everywhere. Combined with Dead Hand systems, the necessity for preemptive strikes, and the enormous capital efficiency of precision munitions, this will result in a holocaust far more rapid and complete than nuclear weapons could ever have threatened.

While this sounds like an awesome plot for a science-fiction novel, I'd rather live in a very different future.

So, I hope that we can develop better defense mechanisms than just drone-striking drone pilots, drone sysadmins, and drone programmers. For example, pervasive surveillance (which also eliminates what we know as "human rights", but doesn't end up with everyone inevitably dead within a few days); undetectable subterranean fortresses; living off-planet in small, high-trust tribes; and immune-system-style area defense with nets, walls, tiny anti-aircraft guns, and so on. With defense mechanisms such as these, the Drone Age should be more survivable than the Nuclear Age.

But, if we can't develop better defense mechanisms than killing attackers, we should delay the advent of the drone holocaust as long as we can, enabling us to enjoy what remains of our lives before it ends them.

[p1esk]

You paint a bleak future. Keep in mind though, there have been many dark moments in human history when a lot of people got killed for very bad reasons, and yet here we are.

is as easy and anonymous as buying porn on Amazon

I'm not sure ease of use is such a game changer. You can buy a drone today, completely anonymously, strap some explosives to it, remotely fly it into someone and detonate it, a few hundred yards away from you. Easily available cheap drones like that existed for at least a decade, yet I don't remember many cases where someone used them for this purpose. Does Slaughterbot-like product existence make it easier? If some terrorist wants to kill a bunch of people, how is it easier than just detonating a truck full of C4? To a terrorist this technology does not provide that much benefit over what's already available. How about governments? I don't see it - if a government wants someone dead, they will be dead (either officially, e.g. Bin Laden, or unofficially, Epstein-style). If a government wants a bunch of people dead, the difficulty lies not in technology, but in PR. I doubt there is a lack of trigger happy black ops types (or "patriots") ready to do whatever you can program a drone to do. Here I'm talking about democratic first world governments. It's even less clear if tyrannical governments would benefit a lot from this technology - sending a bunch of agents to arrest and execute people is just as effective. I don't think tactical difficulties of finding and physically shooting people is a big concern for decision makers. As you yourself pointed out, Khmer Rouge or North Korea had no problems doing that without any advanced technology.

you must kill them before they have a chance to deploy their forces.

Yes. And that's how it has been at least since 9/11 - CIA drone strikes all over the world. Honestly, I'd much rather have them only have done drone strikes if at all possible (instead of invading Iraq with boots on the ground).

more rapid and complete than nuclear weapons could ever have threatened

Sorry, I'm not seeing it - how would this change major conflicts and battlefields? If you have a battlefield, and you know who your enemy is, you don't really need Slaughterbots - you need big guns and missiles that can do real damage. It's much easier to defend soldiers against tiny drones than against heavy fire. If you don't know who your enemy is, say terrorists mixed in the crowd of civilians, how would face detection help you? As for precise military strikes - we're already doing it with drones, so nothing new here.

end up with everyone inevitably dead within a few days enjoy what remains of our lives before it ends them

You are being overly dramatic. Yes, terrorists and evil governments will keep murdering people just like they always have. No, this technology does not make it fundamentally easier. Is the world today a scary place to live in? Yes, but for very different reasons - think about what will start happening in a few decades when the global temperature rises a couple degrees, triggering potentially cataclysmic events affecting livelihood of millions, or global pollution contaminating air, water and food to the point where it's making people sick. I really hope we will develop advanced technology by that time to deal with those issues.

But of course it's way more fun to discuss advanced defense methods against killer drones. So let's do that :) I was thinking that some kind of a small EMP device could have been used whenever slaughterbots are detected, but after reading a little about EMPs it seems it would not be able to hurt them much because these drones are so small. I don't think nets of any kind would be effective - I just don't see how would you cover a city with nets. Underground fortresses and off-planet camps can only protect a small number of people. In some scenarios some kind of laser based defense system could be effective (deployed in high value/risk environments), and of course we can keep tons of similar drones ready to attack other drones at multiple locations throughout the city. Neither of these seem to be particularly effective against a large scale attack, and both require very good mass surveillance. I think that a combination of very pervasive surveillance with an ability to deliver defense drones quickly to the area of the attack (perhaps carried in a missile, fired automatically as soon as a threat level calculated by the surveillance system crosses some threshold) is the best option. The defense drones could be much more expensive than the attack drones, so be able to quickly eliminate them. Fascinating engineering challenge!

[imtringued]

Real weapons are not like that. They are expensive, they can fail to kill their target and they can also cause collateral damage. If death notes were as easy to obtain as guns there would clearly be an increase in homicides but that's not true with military missiles.

The Slaughterbots video is absolutely awful. First of all quadrocopters have an incredibly small payload capacity and limited flight time. A quadrocopter lifting a shaped charge would be as big as your head and have 5 minutes of flight. Simply locking your door and hiding under your bed would be enough to stop them. The AI aspect doesn't make them more dangerous than a "smart rifle" that shoots once the barrel points at a target.

Do you know what I am scared of? I am more scared of riot police using 40mm grenade launchers with "non-lethal" projectiles who are knowingly aiming them at my face even though their training clearly taught that these weapons should never be used to aim at someone's head. The end result is lost eyeballs and sometimes even deaths and the people who were targeted aren't just limited to those who are protesting violently in a large crowd. Peaceful bystanders and journalists who were not involved also became victims of this type of police violence. [0]

[0] https://www.thelocal.fr/20190129/france-in-numbers-police-vi...

[kragen]

Before you had posted your comment, I had already explained in https://news.ycombinator.com/item?id=22394213 why everything in it is wrong, except for the first line.

As for the first line, you assert that real weapons are expensive, unreliable, and kill unintended people. Except in a de minimis sense, none of these are true of knives. Moreover, you seem to be reasoning on the basis of the premise that future technology is not meaningfully different from current technology.

In conclusion, your comment consists entirely of wishful and uninformed thinking.

[3pt14159]

Eh, there is a difference between the examples you've sited and Costa Rica. They're an ally of the US and a strong democracy focussed on tourism.

> The logical extreme of this is Death Note

I don't really deal with logical extremes. It leads to weird philosophies like Objectivism or Stalinism. In international relations terms, I'm a liberal with a dash of realism and constructivism. I don't live in my ideal world. My ideal world doesn't have torture or murder or war of any kind. It doesn't have extreme wealth inequality or poverty. Unless this is all merely a simulation, I live in the real world. Who has the power to kill people? Lots of people. Everyone driving a car or carrying a gun. Billions of people. It's a matter of degree and targeting and justification and blow-back and economics and ethics and so many other things that it's not really sensible to talk about it.

I'm familiar with the arguments against AI being used on the battlefield, but even though I abhor war, I'm not convinced that there should be a ban.