[metalsiliconYT]

Respect for sticking to your beliefs. Just out of curiosity though why do people not want smart AI weapons? I would much rather have an onboard AI that can discriminate between unarmed civilians and military assets, seems irresponsible to not... Is a dumb sea mine that blows up everything somehow better than a smart sea mine that knows to not blow up sometimes?

[jodacola]

"You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"

I realize that's not a great argument and was definitely tongue-in-cheek, but given there's still a lot of debate about the accuracy of AI for far more mundane tasks, my personal perspective is that until we have LLMs and such that are truly, demonstrably far more accurate than humans, with true reasoning and judgement capabilities, they don't belong where lives are at stake.

I wouldn't want an LLM-underpinned machine running anesthesia during a surgery; why would I want an LLM-underpinned military apparatus that is deciding the lives of far more? I wouldn't, not in their current state.

In a hypothetical future where we truly trust incredibly smart AIs or LLMs or whatever "smart" technology it is for driving weaponry, okay - if it's truly necessary; I abhor war and the death and destruction wrought by it.

In my mind, though - even if we get to that future where there's some vastly superior technology to the LLMs we have today, which can judge and reason, then I'll have a bunch of other questions, like understanding the motivations of said technology, because I suspect it'll be something much closer to AGI, and that opens a whole separate can of philosophical worms.

[tartoran]

Not only that but an AI could follow any orders it is given (whether it hallucinates that's a different story) whereas humans may oppose. The friction of orders having to go through a chain of command through its completion is a feature not a bug and that adds responsibility. AI can take no responsibility by itself and with 'moving fast break things smart weapons' that responsibility can be further shoved under the rug.

[metalsiliconYT]

So you would prefer weapons that cannot reason at all?

> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"

Most likely this event happened due to a bad targeting system that wasn't smart enough, if it had a better llm underpinning it (assuming it had any) maybe those lives would have been saved. More reason for more smart people like the author to work on these systems.

[nvme0n1p1]

> So you would prefer weapons that cannot reason at all?

Um, yes? It's bad enough humans are murdering each other. At least a human can in theory be held accountable for pulling the trigger. The last thing we need is an unaccountable ralph loop reasoning about which schools and churches to bomb every time it wakes up.

> if it had a better llm underpinning it

Ah yes, the "LLMs are intelligent, you're just not using the newest fanciest model" fallacy. This time with innocent lives on the line. If only we used ChatGPT-8.9 instead of 8.8, those poor kids would still be alive today.

[butlike]

The bottom line is a human should have to fire the gun and answer for it

[Retz4o4]

President Trump reveals the US Space Force has CAMERAS that can read the NAME TAG "Mohammad" trying to reach any nuclear dust in Iran

"We say: 'Mohammad something is there with shovels.'"

"We have cameras that can read the badge of the person. 'Mohammed Something.'"

"WE'RE WATCHING. If anybody goes there, THEY GET BLOWN UP."

"Eventually, we'll take it."

They have also admitted they saw the children’s flower chalk drawings too. And they double tapped.

[Barrin92]

>Just out of curiosity though why do people not want smart AI weapons?

When the decision to kill another human being is made that should be in the hands of a directly accountable other human being, not an unaccountable machine developed in the basement of a private corporation.

And mines, both dumb and smart, in particular anti-personell mines are banned by the Ottawa treaty ratified by 162 countries. It's exactly the autonomous and fundamentally uncontrollable nature of mines, not just that they're dumb, that has produced countless of casualties long after wars were over. Can you tell me that millions of autonomous loitering munitions are not going to end up exactly like those mines still blowing legs off people decades after conflicts are over? And who is responsible then?

[metalsiliconYT]

The mines got banned because they were dumb...

[Barrin92]

no they were banned because they're persistent. That's also the US position despite not being signee to the treaty. Making autonomous weapons intelligent only makes the issue worse because at least expertise was a limiting factor for most rogue actors. Do you know what will happen to millions of autonomous drones when the Russia Ukraine war draws down? Which hands they will end up in? You've created the capacity to kill people 2k kilometers away for a thousand bucks on a chip.

And when those inevitably backfire and start blowing innocent people up, are the CEOs going to prison, the politicians who ordered their production originally? No, nobody is going to claim they're responsible, and that's why it's insane to build a weapon that does not have a name, time, place and direct order associated with it.

Even the US military is already incapable of taking responsibility of blowing a school with 150 kids up. You want to hand these people systems that are unaccountable by definition, that are smarter than they are, when they already struggle to understand the consequences of their own decisions?

[cogman10]

Mainly I don't want them because I don't want a machine instantly making life or death judgements. It might be fine if it mostly informs a human who ultimately pulls the trigger, but having that failsafe is important in keeping machines from going on killing sprees.

I also want people to be held accountable when they do unjustified killings. AI weapons make it FAR too easy to simply pass off a killing as a "woopsie doodle." It's just not acceptable to say "The algorithm made a mistake, version 23 will do better".

I don't have a problem with the AI providing additional information to it's user, but when that's incorporated into a weapon it's a short distance from that to completely automating the killing.

That's why I'm completely against AI weapons.

[metalsiliconYT]

Responsibility will always have to lie with the user, but if the weapons are more advanced and safer then why is that bad? I like that autonomous vehicles like Waymo are 10x safer then human drivers, even if a 'Machine' is making decisions.

[scarecrowbob]

Because they aren't "safer" for the folks getting killed.

Historically, the folks doing the murdering just don't care about the folks they kill, so "safer for the killers" isn't a win for most of us.

"More precision" isn't about killing less folks, it's about making it safer and easier for folks who kill folks to do that work.

So those of us who dislike killing don't like these tools because we consider them to be immoral.

[butlike]

That is a great point and one I never considered. The folks doing the murdering shouldn't have their jobs be easier. They aren't safer for the person getting killed.

[cogman10]

> Responsibility will always have to lie with the user

Being fully autonomous makes it hard to identify exactly who that user is and is easy to dilute responsibility. Perhaps someone was added to a kill list by mistake. Maybe some internet hi-jinks tricked an AI into falsely identifying someone as a kill target. Perhaps it's the case that someone was in the 5% of a 95% confidence of identification. I'm not a fan of putting killing into the hands of something known to get things wrong 1 in 20 (95%), or 1 in 100 (99%), or 1 in 1000 (99.9%).

> but if the weapons are more advanced and safer then why is that bad?

It's yet to be proven that they are "safer" as they become more advanced. There's also a question of "safer to who". It's technically "safer" for a soldier to shoot first and ask questions later, it's obviously not safer for the villagers.

False identification, which is an absolute part of AI, doesn't make these weapons safer for anyone.

> I like that autonomous vehicles like Waymo are 10x safer then human drivers, even if a 'Machine' is making decisions.

Waymo has the reverse bias. If anyone dies as a result of waymo it's gone horribly wrong.

AI weapons are designed exactly to kill, if they don't kill when they should something has gone wrong.

[anon-3988]

A good thought experiment for this is to replace guards at jewellery stores or banks with a robot. Which one would you feel more safe? Do you trust that the robot won't randomly shoot you if you act funny on the street?

[aiono]

First issue it makes killing people easier. We already see it in IDF soldiers feeling less guilty because in the end decision is made by AI-assisted targeting systems. Secondly it automates killing people. Now a soldier doesn't need to use the weapon himself, he just oversees them and the killing itself is automated. Imagine how much more deaths it will lead to.

[UncleMeat]

I do not believe that AI weapons will reduce civilian casualties. Maybe per strike it will but we also know that when we give people new capabilities they will use them more. Suddenly instead of taking time to select targets you can ask an AI and boom you've got a thousand targets and you are off to the races.

[unknownfuture]

> Just out of curiosity though why do people not want smart AI weapons?

Because the minute you defer decision making to a machine you eliminate human accountability.

Any time the robot is used to do something illegal or immoral, you can just say "it was a bug" or "that's not what I really meant". It enables plausible deniability.

We see this every time AI is allowed to make decisions. That robot promoted violent extremism in your feed? It misidentified a black man? It disproportionately chose men in hiring decisions? shrug Its the algorithm. Who can argue with the algorithm?

Likewise any attempt to codify liability is typically avoided at all costs because the purpose of those systems is what they do.

[righthand]

They aren’t training smart AI weapons to discriminate between civilians and soldiers. Even so, what happens when the soldiers are disguised as civilians? Or when the enemy forces civilians to serve or be a decoy?

[fooo1882992]

Jevons paradox, but for violence.

Right now there's a lot of killing that's only limited by the risk of collateral damage.

Flipped around, the best deterrent against major wars is mutually assured destruction.

[sc68cal]

> I would much rather have an onboard AI that can discriminate between unarmed civilians and military assets,

Has this been proven to actually occur? There have been lots of reports that similar systems in Israel that use AI, struck hundreds of targets that were civilian. They did not discriminate.

https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G...

[moribvndvs]

It’s true that humans are unreliable, even if we could guarantee a consistent set of moral and ethical standards as well as the appropriate tools and authority to enforce them at all times. We do, however, have to deal with 100% of the implications, one way or another. Machines don’t. Vasily Arkhipov and Stanislav Petrov had to consider the world they would condemn their children to if they pushed the button, the missiles didn’t. These are imperfect devices made by imperfect beings, whose rules and goals can be changed more or less at will by whomever wields them. Offloading the responsibility of pulling a trigger or trampling someone’s legal or human rights to such a machine while insulating those in command from consequences threatens to make things worse at a speed and scale I don’t think we’ve seen before in all of our dark history. Meanwhile, it further concentrates immense and unchecked power in the hands of a relative scant handful of elites, all of whom exhibit the lack of ethical and moral fiber we were talking about to begin with.

Teaching something or someone to pull a trigger is trivial compared to teaching them not to. I have no reason to believe that automated weapons and surveillance will be more reliable than our world leaders or that the world will be safer with them in it.

[ncruces]

> Is a dumb sea mine that blows up everything somehow better than a smart sea mine that knows to not blow up sometimes?

Yes. If smart mines lead to belligerents laying more of them because they're "safer."

[pibaker]

A regime can only be as oppressive as the people following its orders. There are only so many protestors you can order your army to kill until your troops realize they are killing their own friends and family, at that point they either desert or turn their guns against you. This puts a check on what even the most evil tyrants can do. Not a perfect one, for sure, but still better than nothing.

Technology changes this by reducing the human involvement needed in killing. The Nazi gas chambers were a historical example — it turned out mowing down innocent civilians with machine guns was too mentally taxing for even devoted Nazis, so they invented a bloodless industrialized process where they can kill with greater efficiency while also sheltering the killer from the immediate effects of his actions. A win win for the Nazis, not so much for its victims.

Autonomous weapons are dangerous for this exact reason. They bypass individual moral decisions entirely. They are the perfect "just following orders" machine. They allow a small concentration of powerful people to impose force upon the populace without ever having to fear their own soldiers. It tips the balance of power massively in favor of those already powerful.

[tgv]

Because it's going to be used in any possible way, and certainly not for the good of the people, as imply in your "naive" false dichotomy.