[acallahan]

Obscurity seems like useful security here. IIUC it shouldn't be possible to e.g. trick self-driving cars with noisy signs, unless you have a copy of the classifier to train against. Thinking about ATMs, you could train against it as a black box, repeatedly inserting different patterns of noise? But it seems probably infeasible if you need to do a lot of iterations.

It also suggests that people concerned about adversarial attacks shouldn't use off-the-shelf pretrained classifiers, where attacks can be trained offline in advance. Similar to hashing algorithms and rainbow tables, maybe a practice of "salting" an off-the-shelf classifier could be effective in dodging attacks.

[3pt14159]

True. Also, just having unconnected systems that use different types of features / heuristics should be enough to at least pull the car over when they wildly disagree over what to do.