[oxcabe]

It'll get better over time. Or, at least, it should.

The biggest concern to me is that most public-facing LLM integrations follow product roadmaps that often focus in shipping more capable, more usable versions of the tool, instead of limiting the product scope based on the perceived maturity of the underlying technology.

There's a worrying amount of LLM-based services and agents in development by engineering teams that haven't still considered the massive threat surface they're exposing, mainly because a lot of them aren't even aware of how LLM security/safety testing even looks like.

[Mizza]

Until there's a paradigm shift and we get data and instructions in different bands, I don't see how it can get better over time.

It's like we've decided to build the foundation of the next ten years of technology in unescaped PHP. There are ways to make it work, but it's not the easiest path, and since the whole purpose of the AI initiative seems to be to promote developer laziness, I think there are bigger fuck-ups yet to come.

[iainctduncan]

Why do you think this? the general state of security has gotten significantly worse over time. More attacks succeed, more attacks happen, ransoms are bigger, damage is bigger.

The historical evidence should give us zero confidence that new tech will get more secure.

[oxcabe]

> Why do you think this?

From an uncertainty point of view, AI security is an _unknown unknown_, or a non-consideration to most product engineering teams. Everyone is rushing to roll the AI features out, as they fear missing out and start running behind any potential AI-native solutions from competitors. This is a hype phase, and it's a matter of time that it ends.

Best case scenario? the hype train runs out of fuel and those companies will start allocating some resources to improving robustness in AI integrations. What else could happen? AI-targeted attacks create such profound consequences and damage to the market that everyone will stop pushing out of (rational) fear of running the same fate.

Either way, AI security awareness will eventually increase.

> the general state of security has gotten significantly worse over time. More attacks succeed, more attacks happen, ransoms are bigger, damage is bigger

Yeah, that's right. And there's also more online businesses, services, users each year. It's just not that easy to state that things are going for the better or worse unless we (both of us) put the effort to properly contextualize the circumstances and statistically reason through it.