[derivt]

We need to define the derivative of a deep model. I mean a way to measure how a model change when we change one pixel in the training data. Since pixel -> feature -> margin, we need to define the derivative with respect to a natural parameter, the natural parameter of the model has to defined ad hoc for every application. Perhaps the natural parameter encodes an uninformative prior. The intuition is to use information theory to see how the discriminative power of the model change when the training data is perturbed. So we need to measure the derivative of the added information. Fisher information seems to be related to this.

[derivt]

this paper https://openreview.net/forum?id=HJC2SzZCW suggest that sensivity is related to poor generalization power. To define derivative we need to use a natural parameter in such a way that it measures sensivity and also allow us to use methods from calculus and manifolds, such as parallel transport of features. How a DNN label a cat when is catching a rat.

[derivt]

If in a DNN for label a cat we explore the group of movements of the animal cat (realistic movements available for a cat) we could relate the discriminative power of the DNN to the energy of the cat. The energy of the cat is related to the volume of the group of movements. A cat with zero energy has the identity group of movements (no movement), a hulk cat is able to alter many of her features, so a very power model is needed to identify a hulk cat. A person full of rage is able to change the color of her face, again energy alter training space. Sorry for using HN for thinking.

[derivt]

Given that DNN are deep, what a one pixel attack means is that one pixel change propagates through the map of features: one pixel => 0-level-feature change -> one 1-level feature change. So this attack relies in weak features that can easily propagate to next level of features. Hence, to defend against this attack the model should put a threshold on the ratio (sensitivity of features)/(number of pixels) and avoid features with high sensivity to easily propagate to the next level of the DNN. If features are not linearly related to input set, then correlation is not a measure of feature sensitivity and has nothing to say about the full DNN effect of such change in a pixel.