[bryant]

Companies are rushing or skipping a lot of required underlying security controls in a quest to be first or quick to market with what they think is transformative applications of AI. And so far, probably very few have gotten it right and generally only with serious spend.

For instance, how many companies do you think have played with dedicated identities for each instance of their agents? Let alone hard-restricting those identities (not via system prompts but with good old fashioned access controls) to only the data and functions they're supposed to be entitled to for just that session?

It's a pretty slim number. Only reason I'm not guessing zero is because it wouldn't surprise me if maybe one company got it right. But if there was a way to prove that nobody's doing this right, I'd bet money on it for laughs. These are things that in theory we should've been doing before AI happened, and yet it's all technical debt alongside every "low" or "medium" risk for most companies because up until now, no one could rationalize the spend.

[buu700]

The sad thing is it's not even difficult to get right. I've got something launching soon with a couple different chatbots that I'll share with you later, and it would never even have occurred to me to rely on system prompts for security. A chatbot in my mind is just a CLI with extra steps; if the bot is given access to something, the user is presumed to have equal access.