[njoubert]

This is a common misconception. It is not a small amount of noise that causes misclassification of images. It is a carefully designed and quite unique pattern that causes misclassification. It only looks like noise to the human eye, but it really isn't.

Yes, neural networks are susceptible to adversarial attacks. No, just adding noise to an image doesn't break neural networks.

[pmoriarty]

Adding small amounts of noise is actually sometimes used to improve the performance of various AI techniques. It helps prevent overfitting.

In fact, if your technique or model is seriously affected by a little noise this is usually enough to brand it brittle and maybe even a failure, as it's a sign of overfitting. Anyone working in this field knows to look for this and will try to make what they create more robust.

The design of visual captchas is one obvious indication of just how successful AI techniques have been at image recognition in the presence of noise. It's no longer enough to make them a little noisy. In order to resist being solved by mechanical means, visual captchas have to include so much noise that even humans have problems recognizing them.

[ipsa]

Read it as: A small amount of carefully constructed noise. Then you are correct to literature and pop-science. No misconception needed. There are 1-pixel attacks now. Randomly shuffling a small amount of pixels around can cause predictions to shift.

The issue is that there is no scene understanding. No common sense. No 3D modeling. Just 10x10 pattern matching on a very large fuzzy database of natural images (which works really really well in most cases).

The hype of ML is driven by 3 things: Big companies vying for AI dominance, militaries that want to finally use neural nets that work, and international competition between the West and the East to be the first to largely automate their economies (or AGI if you want to call it that). Catalysts were big data hoarding, GPU training on ImageNet, and then AlphaGo.

[SubiculumCode]

I am not a machine learning expert, but could not these adversarial example issues be resolved by solving an image classification problem by (1)producing multiple non-equivalent classification solutions with adequate accuracy, then (2)fusion (e.g. voting) to produce a consensus classification? (3) Maybe random shuffling of which X of Z solutions get to vote in each classification attempt.

What might fool one solution might not fool another, and adversarial examples seem to depend on idiosyncrasies of a particular solution.

[freeone3000]

When we do that (and we usually do it for accuracy improvements, not resistance to atrack, that happens at the same time for the same reasons - adversarial examples are a misclassification problem), we change the exact attack that breaks the system, but there is no 100% accurate system - and since it's completely foreign, the examples that a machine would misclassify would likely not confuse a human.

The issue is classification currently relies on a very small embedding of the data which is pattern-matched, with no semantics. It has no way of telling that the difference between a dog and an elephant ISN'T that noise gradient, at least some of the time!

[ipsa]

Some of them yeah. There is active research on this. But it is also possible to create adversarial images for soft voting ensembles of the 6 most popular architectures. Those strong adversarial images that beat the consensus, also have a large chance to fool new neural network architectures that the adversarial image creator never had access to.

[hahajk]

Or just adding a small amount of random noise to the input, which would wipe out the carefully constructed attack.

[ipsa]

You can try out this technique at https://github.com/google/unrestricted-adversarial-examples My guess is it would have the same result as adding noise to the normal images too (resulting in a slightly worse performance overall).

[apatters]

Can I have a tool to add this noise please? So that Facebook et al. can't find me and build a profile on me based on random images that I didn't even know existed?

[ipsa]

https://cvdazzle.com/

[apatters]

This is the most cyberpunk thing I've ever seen

[Izkata]

This style made an appearance on Elementary a few years ago: https://www.youtube.com/watch?v=A1_9aHo0S30

[Animats]

Nice. I just sent that to a friend who has a hair salon in SF.

[jakecopp]

That is a really cool research/art project!

[ericd]

Sounds like those networks need an adversarial network or two to improve their performance and make them less susceptible to attack.