[antpls]

But the model will eventually be updated to detect and process the new cloaking images. So, to stay ahead, you decide to create a model that automatically generates different cloaking images, and... The whole system is now just a GAN : https://en.wikipedia.org/wiki/Generative_adversarial_network

[bigiain]

I think there's a (hopefully strongly privacy preserving) combinatorial explosion here though. If current models can be trained to accurately-enough recognise me with, say, 100 training images - this tool might produce unique enough perturbations to require 100 images for each of the possible perturbations, potentially requiring you to train your new model using tens of thousands or millions of cloaked versions of the 100 images for each of the targets in your training set.

(If I were these researchers I'd totally be reaching out to AWS/Azure/GCE for additional research funding... <smirk>)

[frandroid]

Not necessarily, because the changes are destructive. They can't restore what was there before, and they can't necessarily infer which image was cloaked and which was not.