Really guys? It's so easy. Stream to earth local server with x minute delay, and Oculus users interact with local feed. No more latency. Really not difficult.
That works only if the system is always capturing 360 degree video, and Rift users are using their own head-tracking to select which portion of the spherical view they're seeing. If they're actually using the head-tracking to send movement commands to the lunar camera, then the latency is unavoidable.