[stcredzero]

One step at a time! Next it'll be defensive technology only, then defensive weapons, then the best defence is a good offence.

LLM chatbots pose no existential risk, because they are not autonomous. However, give a model an OODA loop and planning, and it could become an existential threat.

https://en.wikipedia.org/wiki/OODA_loop

The problem with military applications, is that having an OODA loop which is potent enough to defeat a human being is basically required. Once multiple factions have such AIs, then those AIs will be in a mutual arms race and become ever more potent. Planning capabilities will also improve in such an arms race.

Before you know it, it'll be "wipe out the humans to achieve peace in our time."

If the AI arms race is very rapid and the improvement curve is steep enough, then humans might become as irrelevant as black powder arms on the modern battlefield. (And everywhere else)