[Carpetsmoker]

What would a "non-pathetic" latency look like? What improvements in "the stack" would be needed?

The 130ms is "the time from light hitting the camera to when it appears on-screen on the other side". The article doesn't really specify where the latency happens, but with just the encoding/decoding of video alone I think you'll very quickly get latencies in roughly the ~100ms ballpark no matter what.

[Kuinox]

130ms is considered a lot in competitives FPS. But for videoconference, I don't know if it's a lot, In the fps you get feedback of your own actions instantly, in videconference, you don't get this feedback. When you talk to someone in person, there is a latency, the time the brain process what happened in front of him. It add a latency of at least 250/350ms. I don't think adding 130ms on top of that is pathetic.

[snvzz]

>It add a latency of at least 250/350ms. I don't think adding 130ms on top of that is pathetic.

Adding latency is always bad. You're seldom in control of the full stack. Latency will come from hardware, firmware, your OS, the network, everywhere. The responsibilities are split. But if even a single node of the chain doesn't take latency seriously, it screws up the whole, because every link of the chain adds latency.

That said, 130ms is a lot for a single link of the chain other than the network (physical distance will of course incur latency).

[ponker]

Not “no matter what,” but perhaps “running on a multitasking operating system.” With a hardware encoder and 1-frame buffer you could get down to 50ms.

[snvzz]

As anyone that ever used AmigaOS would tell you, multitasking is not an excuse for poor latency behavior.

seL4[0] will guarantee (with formal proof) a low maximum latency despite multitasking.

[0]: https://sel4.systems/About/seL4-whitepaper.pdf

[vertex-four]

Across the internet, at all times?

[ponker]

No, just exclusive of transport. So 50ms in-the-building and another 50ms for SF-NY and another 50ms for SF->Moscow or similar.

[snvzz]

>What would a "non-pathetic" latency look like?

Negligible, compared to that of the network.

>What improvements in "the stack" would be needed?

It would have to take latency seriously, thoroughly.

>The article doesn't really specify where the latency happens,

It really happens everywhere. The rust code likely accounts for very little of the overall latency anymore, but the overall latency is still bad.

[Carpetsmoker]

I'm sorry, but this is such a non-answer. What am I supposed to do with this? Clearly things aren't that simple to achieve negligible latency, otherwise we would have it. I bet there are a lot of trade-offs involved here (there usually are), and simply stating "take latency serious" isn't helpful in having any meaningful conversation about this.

[snvzz]

I'd start by doing a PoC of replacing the OS with an RTOS. This would allow for pointing and shaming; Linux/OSX/Windows are to blame.

I would pick seL4 because of their WCET proof.

[Carpetsmoker]

But there are also a lot of drawbacks of RTOS systems, no? Just searching "Real-time operating system downsides" shows a lot of drawbacks that makes it seem like a poor fit for a general-purpose OS: multi-tasking is limited, it's less efficient, and much more complex. There are certain use-cases where this is a good trade-off, but for my desktop OS? Probably not.

[snvzz]

>tradeoffs

It depends on the implementation. Most protected RTOSs are vastly inefficient but it doesn't have to be so.

For an example, many multitasking RTOSs do reserve CPU time for the "important" tasks, whether they use it or not. That's the industry standard for "mixed criticality", where less and more critical tasks share the same CPU.

But there's better ways to do this. Refer to the seL4 whitepaper[0].

>desktop OS

I'd rather not WaitForever™ for the computer to react to my input due to unbounded latency on my Desktop OS. We might be used to trash responsiveness, but interacting with a computer does not have to be this bad. We should stop pretending non-RTOS systems are okay to build a desktop around.

[0]: https://sel4.systems/About/seL4-whitepaper.pdf

[Carpetsmoker]

So what is the latency on seL4 then for a similar program? And how well is the performance under real-world desktop conditions compared to Linux, macOS, Windows, etc? I read through that PDF and it doesn't really contain concrete answers to that.