[cowboyscott]

How on earth do you coordinate incident response for this? Imagine an agent for customer service or first line therapy going "off the rails." I suppose you can identify all sessions and API calls that might have been impacted and ship the transcripts over to customers to review according to their application and domain, I guess? That, and pray no serious damage was done.

[callalex]

It would be extremely irresponsible to use these current tools as a real customer service agent, and it might even be criminally negligent to have these programs dispense medical care.

[Gazoche]

For customer service, that ship has already sailed. And it's as disastrous as you may expect: https://arstechnica.com/tech-policy/2024/02/air-canada-must-...

[carom]

Ideally they would be logging the prompts and the random seeds for each request. They probably also have some entropy calculation on the response. Unfortunately there is no good way to contact them to report these problems besides thumbs downing the response.

[barryrandall]

It'll probably require AI. Being on-call for explicitly programmed systems is hard enough without the addition of emergent behaviors.