[danShumway]

Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail.

Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?

[notus]

I think it was implied that I meant heuristics that humans have added themselves. The point of it all is to allow models to make generalizations about things it hasn't seen before. This can be done with a combination of supervised and unsupervised techniques.

[danShumway]

> heuristics that humans have added themselves

I don't think this is a meaningful distinction. Who cares whether the new heuristic is being added by a machine or a human?

You still need to keep feeding the neural network data to learn from, and it will still choke when it sees novel data that doesn't align with the heuristics it developed.

That's the entire reason adversarial AI works. The reason the Trippy T-shirt makes you invisible to some current AI systems is because it exploits the heuristics they've built using data that these systems are unfamiliar with and haven't learned to process yet. If it was possible to build an AI system that could defend against novel attacks, the Trippy T-Shirt wouldn't be able to fool them.