[sillysaurusx]

An invariant isn’t the same thing as a purple giraffe. One is an image manipulation applied at training time to make the classifier robust against transformations. The other is a thing that might someday exist in nature. (The most straightforward way is to dump a barrel of wine over the giraffe and take a photo.)

[svara]

You're thinking simple image augmentations. These nets learn much more complex invariants. Basically to isolate concepts from irrelevant context. The point is you can't remove that image from the training data (not practically) and the experiment is pointless if it's in there.

[sillysaurusx]

Sure you can. Have it generate a new photo. Or dump a barrel of wine over the giraffe.