[notus]

> Security is always reactionary, you can't defend against an attack you've never seen before

Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.

[danShumway]

Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail.

Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?

[notus]

I think it was implied that I meant heuristics that humans have added themselves. The point of it all is to allow models to make generalizations about things it hasn't seen before. This can be done with a combination of supervised and unsupervised techniques.

[danShumway]

> heuristics that humans have added themselves

I don't think this is a meaningful distinction. Who cares whether the new heuristic is being added by a machine or a human?

You still need to keep feeding the neural network data to learn from, and it will still choke when it sees novel data that doesn't align with the heuristics it developed.

That's the entire reason adversarial AI works. The reason the Trippy T-shirt makes you invisible to some current AI systems is because it exploits the heuristics they've built using data that these systems are unfamiliar with and haven't learned to process yet. If it was possible to build an AI system that could defend against novel attacks, the Trippy T-Shirt wouldn't be able to fool them.

[jadell]

If you train your security on parallel lines, and I come in with circles, I've just defeated your security. Machine learning only learns how to categorize things into predetermined categories. If I come in with a novel category it's never seen before, the best it can do is guess, and most likely, it will be worse at guessing than random chance.

[notus]

Except nobody would train just on parallel lines. They use a wide array of different types of data so the model can make generalizations about things it hasn't seen before.

> Machine learning only learns how to categorize things into predetermined categories.

This is just one type of machine learning called classification, there are others like regression and clustering which can be combined to create more robust models. Look at the technology behind Cylance's product which identifies files as malicious or not pre-execution. They are not just using classification.