[holmesworcester]

To make sure we keep track of what we're talking about with loss-of-control x-risk, a sufficiently smart version of Claude Code is more deadly than any government's army of autonomous killbots, because it can recursively self improve and has unpredictable training-induced preferences.

[SXX]

Sufficiently smart version of Claude Code: dont exist.

Autonomous flying killbots: exist.

Once somebody scientifically prove and shows any kind of self-improving software we can start bothering about it. I pretty sure everyone trying to do it and it would be all over the news once its here.

[plaguuuuuu]

That's exactly what Fable is. They use Fable to improve Fable. I reckon the successful experiments must go into the model training set with a strong RL signal, and that is why they are so paranoid about people using Fable for LLM tasks. Fable knows what it did to improve itself. Pure speculation of course.

[Davidzheng]

We're on track to get there globally and economic pressures will ensure it happens. It's not too early to worry about it

[codeGreene]

There's a 745 mile front of the Ukraine war where neither side have been able to pierce for months because of drone warfare. It's definitely not too early to worry about it.

[SXX]

I guess they were talking about self improving software. Which obviously not there and likely wont be there soon.

As for killbots they are all over frontline, but dont actually need particularly smart LLMs to run - some good enough segmentation, pattern recognition on smartphone SOC is enough to kill hundreds of thousans of people.

[catlifeonmars]

You don’t even need autonomy to be deadly. Fly by wire is proving to be very effective as a case in point.

[Muromec]

Most of the drones are operator guided and the ether is really badly jammed out there with the exception of glass and that new redacted thingy.

It will start moving really fast once the automatically targeted anti-drone turrets get to production pipeline. Now calling it anti drone is a bit of self delusion -- pattern recogniser gonna pattern recognise whatever it's told to, including "anything moving that emits EV or IR and not broadcasting the friendly signal hard enough".

I wonder how it is supposed to behave if the invasive fauna decides to call it quits and surrender. Should the robot following the Convention or is it yet another accountability sink?

One thing I'm sure of -- the killing not will b blessed by at least one Orthodox priest, maybe this year. OCU will have to develop guidance on that matter.

[Muromec]

I saw an llm bootstrapping and testing it's own harness and rewriting it's own system prompt. If that's not self improving then I dunno what is.

Can the thing enter into an runaway looop while improving the model itself -- probably not, not without us not noticing at least

[mx7zysuj4xew]

That's ridiculous scifi nonsense