[torginus]

I had a geniunely surreal conversation with the security team the past week, it went like:

'Hi, we are reaching out to you because our tool flagged a large data transfer between such and such services'

'Wait, the source endpoint is an internal service, the target endpoint is an internal S3 bucket (I was doing a routine DB backup) Neither are reachable from the internet. How is it a security issue?'

'Our tool has flagged it'

[chillfox]

Almost all the corporate security professionals I have dealt with have been tool runners with no more than Helpdesk level skills.

[void-star]

As someone with over 30 years experience in computer security, both in corporate as well as boutique security and startup shops, who has been consistently fighting this trend, and recently bearing witness to and engaging in the current AI surge: I can say with absolute confidence that it is only getting and going to get even worse yet.

People like me who know there is a better way are getting pushed harder to lean on AI tooling even though we know that it is making things worse. This isn’t just because our founder/funding overlords are pressing us to do it. The sheer volume of new mission critical code being pumped out enabled by vibe coding is also leaving us little choice but to lean in too just to try and keep up.

We can all see it as clear as day: The tech isn’t ready for any of this. But nobody wants to hear that and everyone is marching off the cliff together anyway. We’re all going to land in the same waste pit together. Raise a glass and whimper.

[jatora]

AI is far better at security than the majority of security professionals. It is a net positive.

People constantly compare AI to this very rare expert human rather than the reality of who is already employed. Experts like you are a major culprit of this. And it puts you at odds with yourself to both admit the industry is full of subpar workers and then lament that they will be replaced with workers that are better, but still worse than you.

What is wrong with someone to make them think in this manner? Is it just a kneejerk response with little thought? Is it ego? Is it a coping mechanism? I find it very strange and interesting and annoying.

[nullpoint420]

I also don’t like your framing, here.

We need experts to know when AI is wrong, which it is all the time.

Earlier this week someone commented here that we shouldn’t expect a language model to know that you need to drive a car to a car wash, to wash a car.

So then, what do we expect it to know? Who’s responsible for when it’s wrong?

Also, why can’t Mythos just fix all these issues itself if it’s so smart. And test them to make sure they work?

[void-star]

I actually agree somewhat with jatora. However a large segment of the top ~20% of security folks are being forced to become reverse centaurs, as opposed to centaurs (disempowered vs empowered) due to the factors I mentioned. I genuinely see value in the tech, but it is currently being deployed recklessly and stupidly.

[scrollaway]

> why can’t Mythos just fix all these issues itself if it’s so smart. And test them to make sure they work?

“Why”: because you didn’t ask it. It’s not its job in this case.

You don’t hire an accountant and tell them “why can’t you fix my cash-flow problems and make me money if you’re so smart”

[nullpoint420]

Ah ok, sure. The difference being the model should know how to do both based on what I’ve been told.

So why didn’t Anthropic ask it for me?

[void-star]

You are leaping to the assumption that I don’t actually believe in the tech. This is incorrect. I am griping with the way it is being recklessly and stupidly deployed by poeople who really don’t know what they’re doing.

[ikiris]

That means you aren't high enough up to deal with the non helpdesk level security people.

[torginus]

True. It is a well-known fact that braincells per capita, and technical competence and understanding rapidly increase the higher you are on the management ladder.