[zenir]

I actually hacked something together like that and it surprisingly looks much better in a video than in real life. I guess the main reason being that we have two eyes and the brain still realizes it is flat (which is weird because it doesn't look flat in a video).

[vanderZwan]

It works well in the video because in that context we are trying to estimate depth relatively to the camera.

In real life, we are trying to estimate depth relative to us. In order to fool our brains we need both a very low latency and a high frame-rate. That was one of the major hurdles to solve for VR as well, leading to John Carmack's famous complaint that he can ping across the Atlantic and back faster than he can send a pixel from his desktop to his screen[0][1].

Anyway, back to the video: basically, when comparing 3D movement relative to the camera our brains seem to be more "forgiving".

[0] https://twitter.com/id_aa_carmack/status/193480622533120001

[1] https://danluu.com/latency-mitigation/