[aab0]

> Neural networks are good at adaptation, but useless at forming concepts about how the data is structured. For example: in video we do motion compensation, because we know video captures motion since objects move in physical reality. A neural network would have to do the same in order to get the same compression levels.

Neural networks know that objects move. See for example https://arxiv.org/abs/1511.05440 http://web.mit.edu/vondrick/tinyvideo/

[andybak]

I can't find any images in that second link other than pointlessly small thumbnails. Am I missing something?

[aab0]

Note the moving objects in those generated/predicted thumbnails.

[andybak]

Yes but it's almost impossible to see what's going on.

[aab0]

You're not supposed to. Image generation and modeling scene dynamics is a hard task, and thumbnail scale is what we're at at the moment. Nevertheless, those and other papers do demonstrate that NNs are perfectly capable of learning 'objectness' from video footage (to which we can add GAN papers showing that GANS learn, based on static 2D images, 3D movement). More directly connected to image codecs, there's work on NN prediction of optical flow and inpainting.