>As a practical example, figuring out where a given pixel moves from one video frame to the next one, when working on real-world videos, the best known algorithms get about 50% of the pixels correct. With clever filtering, you can maybe bump that to 60 or 70%, but in any case you will be left with a 30%+ error rate.
what do you mean by this? optical flow isn't really a learning problem? it's a classical problem with very good classical algorithms
https://arxiv.org/abs/1612.01925
https://arxiv.org/abs/1709.02371
https://arxiv.org/abs/1904.09117
BTW, also the classical algorithms deal very badly with noise and repetitive textures, e.g. a video of a forest in the afternoon.