[Sephiroth87]

Yeah, for the 3d side of things it gets a bit trickier, vs 2d (compositing, grading, etc) where the interface itself could trivially run on anything, and you just need rendered video frames.

But even for 3d I would thing a "streaming based" 3d protocol or something would be much better than just an RDP connection

[Const-me]

RDP connection already does all that. Essentially, it connects client PC's GPU to server's CPU. The software running on the server ain't aware, it calls regular D3D11 APIs and they just work, except for rare edge cases when it doesn't.