[skratlo]

Can't you already do that? With DISPLAY variable. Although there could be less overhead in running local (guest) X server and stream only GL commands.