[jungler]

With the gaming scenario some distinction between display latency and network latency is worth reviewing. Network latency is something that software can cover for in some degree(to create online experiences there's always some kind of synchronization of different wallclock timelines going on). Display latency is universal, though. A faster display will let the game give you feedback faster, so long as we are talking about feedback in the local experience(so: aiming, movement are commonly predicted on the client, while hit registration often uses a server roundtrip). This separation of responsibilities allowed people to play Counter-Strike 1.x on 200ms dial-up connections back in the 2000's.

When it's a pure video feed you get the impact of all the latency all the time, and that means the problem has to be solved with brute force reduction of bottlenecks everywhere.

So how much latency is enough for responsiveness? This is pretty easy to derive from the common target framerates: for 30hz display, you need 33.3ms. For 60hz, 16.6ms, for 120hz, 8.3ms. Since perceptual latency is known to keep improving up through 144hz it's reasonable to say that we should be looking for only 5-6ms at most, while most broadband connections are still achieving pings in the 20-100ms range, depending on the game and the specific connection. On the very best connections, that is, we can assume a "30hz transparent" video stream, which is fine for casual experiences but severely impacts competitive ability when tested. In many current popular titles this manifests in mechanics that are both latency sensitive and require a server roundtrip, e.g. building in Fortnite.