[dontreact]

I don't think adversarial examples give any evidence of relevant problems with these models because they occur on a very specific subset of images that can only be discovered using detailed knowledge of how these networks process images.

For all we know, humans have similar problems on some obscure subset of images, but we can't find human's adversarial examples because we don't have detailed knowledge of how the brain processes images.

[mannykannot]

I think we do actually know enough about how the human mind does process images to have some idea of what is different. It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it, such as making hypotheses as to what's going on and checking them out, and/or seeking to get a better view (or other evidence) in a way that is specifically designed to resolve the uncertainty. It is this higher-level semantic analysis that is missing from current image processing software.

In these discussions, someone always mentions optical illusions, but only humans (so far) understand the concept of 'optical illusion', and recognize that they are experiencing them.

[thaumasiotes]

> It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it

This is true, but step one is "move your head" (or in your words, "get a better view" -- but you get more value from just the fact that your head is in a different place than from the possibility of a better angle on whatever you're looking at).

That strategy doesn't work at all when you're trying to classify static images rather than physical objects.

[mannykannot]

That raises the interesting question of how object recognition in streams of images is progressing, beyond being just object recognition within the individual frames. Humans are capable of extracting a lot of additional information in such situations, and are actually helped when the perspective on a given object changes. One cannot give current machine vision a pass if, through lacking this capability, it is under-performing.

And moving one's head to get a a better view is only one thing that a human might do. Firstly, of course, we must recognize that we are having a difficulty, and current machine vision seems to be somewhat deficient in this regard. Then, even without being able to get a different perspective, we will do things like make guesses as to what might be there (using our extensive semantic models of the world) and figure out if they might be a good fit to what we see, and/or we might try to extract specific features of the problematic area and search our memories for objects that might plausibly match, bearing in mind that it might be from a different perspective than we are accustomed to. We are also quite good at estimating whether an object might be a problem for us, even if we have not positively identified it. There is a lot more to it than just moving one's head.

[TeMPOraL]

GP's statement applies as much to observing objects in 3D space as it does to looking at photos, where just moving your head ain't gonna help you much. Optical illusions are great to study this process, because most of them are delivered in form of flat images on paper or computer screen.

[thaumasiotes]

Optical illusions are delivered as flat images because moving your head doesn't affect those.

[TheOtherHobbes]

Humans are rarely aware of optical illusions unless they're extreme images they don't see in real life - crawling dots, impossible geometry - or they're explicitly labelled as optical illusions.

Some more subtle examples:

http://www.terrycolon.com/1features/optical-illusions.html

In fact human perceptual processes are only kind of reliable some of the time. Low and/or unusual light, suggestibility, and unusual contexts all have a very negative effect on reliability, but humans are often unaware of this.

Cognitive and semantic illusions are even more persistent. People literally believe all kinds of nonsense, and will carry on believing it even when offered robust evidence that they're wrong.

The point being that human perception and cognition are not some kind of gold standard. They have plenty of issues of their own. But there's a kind of assumption/requirement of perfection with machine intelligence that doesn't apply to human cognition. So bugs in our own evolutionary firmware tend to be overlooked, while equivalent-level bugs in ML are seen as terrible failures which undermine the entire premise of AI.

[ben_w]

Recent update — we do know, and humans are vulnerable to perturbations created to fool a multitude of existing AI: https://arxiv.org/pdf/1802.08195.pdf

[adrianN]

I think adversarial examples for humans are called "optical illusions".

[ccvannorman]

There is a categorical difference between "a [specifically designed] image that can be construed as a duck or a rabbit" and "a human can regularly mis-categorize random pictures of ducks as rabbits if a weird filter is overlayed". The first is well-known and fun and trite -- the second is unheard of and probably impossible for humans, yet provably possible for trained computers.

[cameldrv]

It's called camoflauge. The natural world is full of adversarial examples.

[fenomas]

I'd imagine GP was referring to "humans perceive straight lines to be curved when certain shapes are overlayed", or "humans perceive shapes of the same color to be different colors when filters are applied" sorts of optical illusions.

There are plenty of those, and I personally I think they're probably analogous to how adversarial filters fool AI classifiers.

[adrianN]

It's really easy to cause humans to misclassify all kinds of images as containing faces ;). Humans also regularly misclassify random noise as words. You can even suggest which words we hear by telling us what the noise is supposed to be.

[fons]

The point outlined is that we don't know enough about how we identify objects to discard a simple adversarial attack; probably not a filter-based but maybe something else.

[dontreact]

"probably impossible for humans"

Based on what?

[ars]

The difference is that humans are aware that there is an illusion happening, they just can't help seeing it.

[adaml_623]

Or sometimes not aware

https://www.dw.com/en/man-falls-into-black-hole-art-exhibit-...

[mmirate]

The important difference here is that most adversarial examples for the human perception:

(a) do not occur frequently in nature,

(b) are not frequently - if at all - produced in man-made architecture or transit-constructions,

(c) often contain repetitive and regular geometric and chromatic patterns which further make them stand out from everything else, and

(d) practically cannot be produced by digital (ergo noisy/less-than-perfect) images of any common real-world scenario.

In short: optical illusions don't accidentally occur in places where they can be seen by meatbag drivers.

[dontreact]

I don’t see how you can make any claim about “most human adversarial examples”. There is a huge space of images and we have explored a negligible part of it.

Also a) and b) empirically seem to be true of the test sets people have collected thus far of the natural world for these models.

In short, we have no evidence that adversarial examples of the type being studied occur commonly in images collected by self driving cars.

[mannykannot]

The issue with regard to self-driving cars is that these cases demonstrate a disturbing level of fragility: we don't have a good handle on where the boundary between acceptable and chaotic responses lies.

You hypothesize that there are comparable examples for humans somewhere out there in the domain of all possible images, but the fact that, for all the countless cases of people looking at things that have occurred in humanity's existence, no-one has found a good example, suggests that, from the pragmatic point of view that you propose, image-recognition software has some catching-up to do.

Maybe a system that seeks consensus among several differently-trained models would be more robust.

[dontreact]

https://arxiv.org/pdf/1802.08195.pdf

Looks like we are starting to find examples.

I think your intuition is wrong because humans are adapted to what exists naturally so of course there are no naturally occurring adversarial examples. It seems like the same is true for models trained on large natural image sets though.

My point is not wow let’s stop developing neural networks they are perfect. It’s more let’s go collect real world test sets to find and then fix gaps. Adversarial examples actually help very little in making nets more robust in the ways that matter.

[de_watcher]

The difference is that you can calculate an adversarial example for our classifiers, but it's too slow to calculate on a human.

Even if you could, the result would be specific to that particular person, so it won't work as good on others. And these bastards learn while you're constructing the example (which isn't fair at all to a helpless classifier that's just sitting there and doesn't change).

[mannykannot]

Fairness doesn't come into it - machine vision has to be up to the task it is given, period. If humans depend on their more general intelligence to deal with problem cases, machine vision either has to do something similar, or compensate adequately in some other way.

[de_watcher]

That was a joke.

[thaumasiotes]

> The important difference here is that most adversarial examples for the human perception:

> (a) do not occur frequently in nature

You've never heard of walking sticks? Ever seen one of those leaf moths?

If you had seen one, would you realize you had?

Is a deer visible on that stretch of hillside, or is that just dead grass?

[LoSboccacc]

Well yes adversarial images cannot appear naturally because by definition are created out of the network itself, but they highlight the same issue that caused misclassification of the lady in the Uber incident.

[UncleMeat]

Not true. There are black and grey box adversarial techniques as well.