[cube00]

> servers hosting LLMs, which I wouldn't even consider a huge security concern

The new problem is if the LLMs are connected to tooling.

There's been plenty of examples showing that with subtle changes to the prompt you can jailbreak the LLM to execute tooling in wildly different ways from what was intended.

They're trying to paper over this by having the LLM call regular code just so they can sure all steps of the workflow are actually executed reliably every time.

Even the same prompt can give different results depending on the temperate used. How security teams are able to sign these things off is beyond me.

[_flux]

The tools are client side operations in Ollama, so I don't see a way an attacker could use that to their benefit, except to leverage the actual computing power the server provides.