[thelamer]

Yes video codecs are great but in almost every case they need a GPU. This functions with no special hardware. The only lossless option would be AV1 and that needs an Intel arc card or 4000 series Nvidia card.

You also have an inherent latency issue as you have to buffer 2-5 frames at 16ms a pop server side to encode the data.

[vlovich123]

> You also have an inherent latency issue as you have to buffer 2-5 frames at 16ms a pop server side to encode the data.

Consider that oculus link uses video compression and doesn’t introduce the extra latency you’re describing. You only need to buffer frames if you want the best compression ratio. But you can always configure the encoder to not do look ahead. It’s also better to choose cbr over vbr to avoid the second pass of a frame at the cost of reducing quality/bitrate a bit. I’m practice it can work really well because even 20mbit/s is sufficient to send high res text.

I wish Remote Desktop applications would copy the oculus link architecture. You can easily get only a few frames of latency (sub 100ms) provided you composite on the GPU, use hardware encode/decode, and slice the video stream (which ie send 1/4 of the screen while encoding the next 1/4) which cuts down on decode latency and ensures you smear the expensive work across the entire refresh cycle time instead of having to do it all at once in a non-pipeline fashion (which introduces bubbles into scheduling).

[Veliladon]

When doing streaming for realtime interaction you don't use B-frames. You typically just have P-frames and slices of I-frames. Max one frame of latency and pretty consistent data rates.

And just about every phone, tablet, laptop, desktop made in the past decade has some form of H.264 fixed-function hardware decoding.