The bottleneck is the video encoding/decoding/rendering, which is done on GPUs to begin with. Of course if it were done on CPU instead, then it would be significantly worse, but that's not where we're starting from. Improving stuff on the host side by, say, rewriting WebRTC in Rust won't improve the latency of your video by much or at all.