[SammoJ]

This would work for 2D games, but 3D projection doesn't allow this. Furthermore in a 3D game there are more than two degrees of freedom.

[leni536]

You can render the images on the sides of a cube with 90° fov, put your viewpoint in the center and that's it. Once I programmed something like this and it works pretty well. I suppose that Google street view works in a similar way but I can't back this up.

[readerrrr]

Of course it does. Oculus is already using it; it's called timewarp.