[mysterypie]

> altering an image’s pixels in a way that wreaks havoc to computer vision but leaves the image unaltered to a human’s eyes

Can someone who understands the original paper give an ELI5 on how that's possible?

I understand how labeling an image of a cat as "dog" could poison a dataset.

I also understand how adding images of a toaster (which sort of looks like a handbag) to a dataset of handbags could poison a dataset.

But I don't understand what's happening in Figure 6 in the original paper. The pairs of pictures of the dogs, cars, etc. look absolutely identical. What exactly is happening there?

[taylorius]

Computer vision systems are not as robust as human vision systems. There are subtle changes to pixel values that can utterly flummox a computer vision system, but seem undetectable to a human looking at the image.

[hackernewds]

I've read about this

https://www.nytimes.com/2023/02/13/technology/ai-art-generat...

[mysterypie]

How do subtle pixel changes cause the image of a dog to classified as a cat rather than as gibberish?

[amenhotep]

This is a guess (I have read some of the paper but to me it does not seem to explain), but presumably you work backwards from the result you want. There is some vector that comes out of the classifier model that would represent cat, and another that would represent dog; you get the difference between these two models, and work back through the layers of matrix multiplication finding places where small perturbations in the input data force the classifier to make the exact errors you need.

[dragonwriter]

They claim that the poisoning is transferrable across different models, which would not be the case if that was true.

OTOH, given this group's track record with their earlier poisoning effort (Glaze) vs. the adulatory press that they managed to arrange, I don't expect much from Nightshade despite the similarly adulatory press. Great media relations game, though.

[gooob]

what is the nature of these pixel changes?

[taylorius]

It varies. Specially crafted noise added to the pixel values. Looks like random noise, but obviously isn't. TBH I'm not an expert, but as I understand it, It is "trained" using the vision network, with a loss function that is some combination of being low amplitude, and reducing the strength of the correct image identification.