[yorwba]

But we don't really care about the cat-masquerading-as-desktop category in itself, so an adversarial example that makes a cat look like a cat-masquerading-as-desktop, or masquerades a cat-masquerading-as-desktop as a cat, isn't really relevant.

By adding enough adversarial examples to the training set, you can absolutely immunize a model against adversarial perturbations of the training data.

The problem is that the volume of "not very different" data points surrounding an example grows exponentially with the input dimension, so you need to train for much longer, and your "adversarial protection" will likely overfit to the neighborhood of training examples, which doesn't help with unseen data.

[mannykannot]

We care about the existence of a nontrivial set of images that demonstrate a troubling lack of robustness in image classifiers, at least until we we have good reason to say with confidence that such failures will not be a problem in practice.