[gradys]

Does anyone have a good sense of what exactly they mean here:

>Instead of exactly prescribing which feature we want the network to amplify, we can also let the network make that decision. In this case we simply feed the network an arbitrary image or photo and let the network analyze the picture. We then pick a layer and ask the network to enhance whatever it detected. Each layer of the network deals with features at a different level of abstraction, so the complexity of features we generate depends on which layer we choose to enhance. For example, lower layers tend to produce strokes or simple ornament-like patterns, because those layers are sensitive to basic features such as edges and their orientations.

Specifically, what does "we then pick a layer and ask the network to enhance whatever it detected" mean?

I understand that different layers deal with features at different levels of abstraction and how that corresponds with the different kinds of hallucinations shown, but how does it actually work? You choose the output of one layer, but what does it mean to ask the network to enhance it?

[murbard2]

The detection layer will detect very faint random signals. For example, if you have a unit that's supposed to detect dogs, it might be very faintly activated if by random chance there is a doggish quality to some part of the image. What they do is pick up that faint, random, signal and amplify it.

They say: oh you think that cloud is a tiny bit dog-like? Ok, well then find me a small modification to the image that would make it a little more dog like, then a little more, and so on.

Think of it as semantic contrast enhancement

[gradys]

So in concrete terms, does this mean that we show the network an image, choose one layer's output vector, and then back-propagate gradients to the image such that the direction of that vector stays the same, but the magnitude increases?

[murbard2]

That is my understanding of the blog post, yes.

That plus a prior on the input pixels to keep it image-like.

[rndn]

They've written though that they have chosen a particular layer in the network, which reads like "independent of the output layer". Features in such a layer correlate with certain classes, but I don't think they have dealt with classes at all. If that's the case, then the question is how they've amplified the detected features.

[murbard2]

Yes they play with various layers. Layers closer to the input act more like edge enhancers, while higher layers emphasize whole objects ("animal" enhancers). You get increasingly less syntactical and increasingly more semantic as you go deeper in the network.

[nitrogen]

So "lower" layers are closer to the input, while "higher" layers are closer to the output?

[discardorama]

My understanding: when you're doing standard gradient descent, you push the error down through the layers, modifying the weights at each layer. Now, in "normal" NN training you stop at the input layer; it makes no sense to tweak the error at the input layer, right?

But what if you did the following: flow the error down from the outputs to the layer you're interested in, but don't modify the weights of any of the layers above it; just modify the values of this layer in accordance with the error gradient.

Added later: I think we should wait till @akarpathy comes along and ELI5's it to us.

[ebetica]

My thought is that they basically run gradient descent on the image where the loss is the magnitude of one output plane in one of the layers of the neural network. Probably using gradient descent to push up the magnitude of one of the output plane layers or something like that.