[vslira]

Killer bots will be treated just like nukes: world powers will justify their investment in the technology by saying that the other side is also doing it, and when it becomes clear that there are only two or three powerful countries that can wield the technology, the "international treaties" will come barring everybody else from developing it.

I'm not saying that everyone should have nukes, I'm saying the EU, US, China and Russia should be making those treaties right now instead of later

[3pt14159]

The problem is enforcement, not will.

With nukes we can have inspectors and we can lock down uranium. Killbots are too diffuse and too multi-use. I'm all for arms control / non-proliferation, but I really don't see how it works for these things. Also, we already have weaponized AI. We have for decades. We're now talking about a matter of degree.

I'm far, far more worried about the mass weaponization of civilian systems (self-driving car, etc) via cyber attack than I am about killbots, but they're both symptoms of a different problem:

As technological growth continues the space of potential combinations of methods accelerates. This ever expanding search space results in unpredictable threats and increasingly asymmetric attack economies.

I've been meaning to write an article on it, but I'm having trouble pulling together the math because it's so abstract. But that's the general idea.

[Nasrudith]

Sounds a bit like the Fermi Paradox and Drake Equations - while the math technically works out the missing terms make it useless on a practical level.

The possibility space is limited by sanity checks of the actors - technically a terror group could start putting live rattlesnakes into snake in a can prank devices on people's doorsteps but it would be logistically difficult and incredibly stupid.

Beyond that one interesting thing about attackers are how memetic they are over what is appealing to their personal image as opposed to effectiveness. Like the symbolism matters more.

Terrorists in the US could cause lasting damage and disruption by infrastructure attacks like going around and emptying a few AR-15 clips into unguarded transformers and substations or train track sabotage around oil freight trains yet they don't at all. Similarly looking at the historic trends between bombings vs mass shooting vs vehicular murder plots seems to be more cultural than availability related.

[3pt14159]

I wouldn't say useless, but I agree with some aspects of your general thrust.

I agree that symbolism seemingly matters more. I was actually once on a podcast years ago arguing that terrorism was essentially a solved or non-existent problem because armed with nothing more than a rented truck terrorist could plow through crowds. And in the US it was even easier! They could buy weapons at gun shows! Why did they need to hijack planes?

Ultimately, though, my arguments didn't hold. The terrorists did exactly what I thought they would do if they really wanted to cause damage.

Just because X could Y (but doesn't) doesn't mean X won't ever Y.

I'm not worried about ISIL style attacks where a couple hundred people die. Frankly cutting out one can of cola a week would probably save more lives in the West.

I'm worried about mass attack. We can skate around it all we like, but we're one bad Windows / Dropbox / Tesla / Linux / Cisco / QNX update from hundreds of thousands or millions of people dying. The Windows worm that preceded the electrical grid blackout in the mid 2000s was the inadvertent cause. If an accidental worm can take out that much we should be much more worried about the scale of the threat.

We've also never been able to secure the boarder. With drones this gets even harder because capture is no longer a real deterrent.

Security is an arms race that favours the offense over time. A theatre that strongly favours the offense requires deterrence and intelligence for dominance, but the deranged can't be deterred and I don't think intelligence will work forever. Ultimately the arms race needs to end.

I don't know the solution, but my intuition points to GATACA and UBI. But even with an optimistic estimate I still consider that a stop-gap.

[dpflan]

I'm intrigued. What do you need to in order to write your article?

[3pt14159]

I'm waiting on some academics that study multi-lateral trade agreements to send me the details of their mathematical models, since I think they'll share many of the same aspects.

1. Hard to quantify incentive structures.

2. Chaotic / cascading decisions and adaptations.

3. Growing search space that's influenced by information restriction.

4. Uncertainty of future arrangements and options.

5. Uncertainty of deterrence efficacy.

6. Even some aspects of agency problems.

I've gone through some of the papers out there but, frankly, nothing I'd consider a serious mathematical investigation. I don't need the input data so I thought this would be easy. I only need the math or computer code.

[thrower123]

Startup and capital costs for a nuke program are considerably higher, and it is a lot harder to hide that you are building the massive facilities needed to process the raw materials and manufacture nuclear weapons. Not to mention needing to come up with a viable delivery system.

AI is nothing like that. It's just software. Once it is out there, it is going to be copied, and the resources involved are minuscule.

International arms control treaties haven't historically been worth the paper they are printed on, so there are probably a great many things that would be more worthwhile to direct energies towards.

[beginningguava]

Unfortunately I think there's a good chance humanity wipes itself out, or at least destroys civilization in the next several centuries. We are still effectively monkeys but have over time developed weapons that are capable of incredible destruction.

The issue with regulating AI kill bots is that much of the technology that could be used for good like self-driving cars can also be weaponized. There's really no way to progress without risk. The only other option would be to ban AI research.

Might be the Amish and Unabomber were right about technology

[ForHackernews]

Why several centuries? We're already on track for +4°C by the end of this century.

No need to speculate about killer AI bots when you've got plain old famine, drought and resource wars.

[fsloth]

I don't share your pessimism - entirely.

While the human body and psychological context is preprogrammed up to a point from our genetic makeup, we are capable at learning.

The same mind that was good enough to craft better stone tools sufficed to send people to other worlds.

The same societies that practiced slavery and genocide reached a point where these became non-accetable.

While as a species we are inherently capable of great horror we are also capable at great acts of compassion and love.

The fact that we are constantly teetering on the edge of animal chaos makes it only more important to try to be a decent human being. And not to listen to your amygdala so much.