[Terr_]

> On top of all the other things that add uncertainty to our software (flaky servers, third-party APIs going down, scaling issues),

Plus the elephant in the room, that prompt injection and hallucinations are a fundamental way of life for the algorithm, even when it's not getting updated in the background.

[aickin]

Yep, that's also a huge issue with LLMs in production. Our product has some automatic detection of jailbreak attempts so that you can see when people are trying to jailbreak & prompt inject, but hallucinations are the biggest unsolved problem imo.