[arnioxux]

There was another submission recently that kind of does what you described. Click on the visualizations to see the iterations: https://dmitryulyanov.github.io/deep_image_prior. The ELI5 is that prior knowledge of what images look like can be used to reconstruct images w/o corruption.

For non-images, this idea is pretty old and sounds a lot like compression algorithms that share a pre-defined dictionary. For example: https://en.wikipedia.org/wiki/Brotli, "improved the compression ratio by using a pre-defined dictionary of frequently-used words and phrases." Of course words/phrases are a lot easier to predefine.

I wonder how large the predefined weights of the NN has to be to effectively compress real world images?