[kakali]

That is a valid problem, but with MR pass through there is also a compute issue. The cameras are not in the same location as the users eyes. They have a wider parallax than the users eyes and are forward several inches. There is no simple transformation to correct the video to have the same perspective as the user. Instead a 3d representation of the environment must be created, camera imagery projected on to it, and then rendered to the position of the user’s eye. All of this done as fast as possible and highest rate possible. This explodes in compute with increasing resolutions. MQP is doing this with 3 cameras, color compositing, and without the aide of a depth camera.