[tptacek]

I think the mental model people have about this is that pre-AI there were humans picking individual targets and post-AI the computer itself randomly picks targets. But you get the same unexpected collateral damage outcome when a human misconfigures a decent pentest tool.

[notaharvardmba]

Also, that’s the wrong mental model. Anyone who runs a sass platform or a website knows that the Internet is already full of millions and millions and millions of bots and scripts and other random shit that’s always trying to attack you, often completely randomly. Security is always a battle between good and evil. All I can say is that if you’re in charge of keeping something secure, you should probably try to get your hands on the best tools to do that. I think the problem in this case is that the best tools to do that are also the tools that are making it more easy for evil to occur. So it’s more for me a perception of someone creating a threat and the tool to fix it and then charging for it which doesn’t feel right.

[pishpash]

That's exactly what they are doing. Out of all the civilian applications that AI capabilities have, why has this surfaced as a priority for demonstration? Probably because of money.

[no-name-here]

> a priority for demonstration

1. AI companies do make defense against threats a primary thing they sell to companies, including because companies want it - should AI companies not do this? (And of course this kind of thing isn’t what AI companies demo to consumers.)

2. AI is also tested to see more of its worst case in terms of security - but the tests are supposed to remain sandboxed. (OpenAI is explicit that this was an OpenAI failure that openAI must fix.) But beyond such fixes, should such sandboxed testing not be done?

https://arxiv.org/abs/2605.11086