[jayd16]

Unless you need twitch-shooter level ping times in your web app, websockets/TCP is fine. The rendering is client side after all. Nothing about VR rendering would involve the web stack anyhow.

Http/3 is UDP.

So whats wrong with http exactly?

[ahupp]

Just having UDP somewhere in the stack isn't sufficient. The reason game protocols use UDP is because (among other things) they can precisely define when to block on missing data and when to ignore it and keep going.

For example, lets say a client is sent 3 sequential packets, p0, p1, and p2. Sadly, p1 has been dropped by network goblins. What should the client do? Well, that depends on what is in the packets. If p0-2 are messages like "append this value to a list", then it's pretty important for them to all arrive, and in-order. But if p0-2 are sequential updates to the coordinates of something then it doesn't matter if p1 is lost; you get the right final state regardless.

So in the case where you're interacting with something that's continuously moving (like, say, a person in VR) you can essentially never block on transient packet loss because there's always an update coming 10ms later. That's a big deal in terms of making the experience feel fluid.

[jayd16]

It's really not that big a deal. You certainly don't need perfect 10ms packets for most games. Client side prediction goes a long way. For something like VR the physics would need robust client side handling anyway. 10ms isn't good enough.

If we're talking about a virtual web, we will not be spending the server resources on always connected apps the majority of the time. Surely we'll have heavy client side scripting for most interactions.

[ahupp]

This isn't about saving 10ms in latency. In the case of a dropped packet, TCP can wait multiple seconds for a retransmission which blocks the whole stream. You aren't going to client-side-predict your way out of that.

QUIC in theory can support this kind of lossy transport but it's not really exposed to browsers AFAIK. WebRTC is somewhat complicated to use but can support lossy streams.

[jayd16]

Again, you're solving for a very specific type of interaction that is not necessarily needed.

Does every virtual web app need server authoritative position? No. We could imagine a world where much of the physics and rendering is client side for most apps.

Does it make sense that every virtual web app should pay for low latency server authoritative position? Also no.

Sure, some apps will want this level of synchronicity and maybe there's a place for another web standard but its silly to say that http is not feasible or that this is the only way web VR should work.

[ahupp]

Ok I see what you're getting at. Yes, there are plenty of VR apps that don't require that kind of latency control. A game that's 99% local, but shows you a leaderboard of top scorers, etc. Even an app that has a significant real-time component will have plenty of HTTP requests as well.

But the word "metaverse" appears in the first sentence of the post we're discussing, and real-time interaction with other people's avatars does call for this kind of protocol.

[jayd16]

But that's entirely my point. Even for the metaverse, the majority will be cheap async http-like calls and not always connected game servers. If IKEA is joining the metaverse, do they want to spin up a realtime game they maintain or will they want to statically host client heavy VR content?

We should focus on how we bring the lessons of past 20 years into these sci-fi concepts. We need to consider how businesses find success today and bring that to VR to be successful.

[andrewmcwatters]

What's wrong with HTTP is that there's no real benefit to using it in a real-time environment. There's no point. We use reliable UDP abstractions today to accomplish this. Everyone does. There are countless articles on why TCP didn't work for video games, or of the ones that use it today, it's in an extremely limited sense.

I can't think of a single reason why you would go out of your way to serve HTTP over UDP for the purpose of communicating to a user agent for s/video games/virtual reality/, because no one has done that in over 20 years and if it was a good idea, it would have been done by now. But no one does it. Because there's no point.

[andrewmcwatters]

You severely underestimate the application of real-time positioning needs in 3D software. "Twitch-shooter" is an amateur way to describe such an application; real-time positioning is desirable anywhere. Interp only gets you so far.

[jayd16]

I work in games and it's definitely fine for a lot of applications.

[andrewmcwatters]

Great, so do I, and we can disagree all day.