[derefr]

> Pre-rendering the scenes on the server and then sending those is just not really possible today due to latency -- even google can't get it right with Stadia so far.

Ah, but Stadia (and Playstation Now, and Geforce Now) are trying to solve a different problem: moving all the compute to the cloud, so that the game console can be wimpy (or of a different/incompatible architecture, in PSNow's case.)

If you remove that requirement—or rather, impose the requirement that the client hardware must be just as powerful as if it were running a full game—then you can do something clever by splitting the game not into "renderer on the server, frame buffer on the client" but rather into "game and projection matrix on the server; display-list renderer on the client."

In other words, the server (or per-client server-side agent) would be responsible for doing all the evaluation of local physics (e.g. soft-body dynamics, constraint-resolution during rigging, etc.), lighting calculation, and all the other stuff that currently requires the client to "know stuff" about the world; and then it'd output to the client, on each tick, just a display-list—a bunch of loaded-model references, their positions in space and the positions of their rigged limbs, and the reflected-light amount on each of their polys; any temporary particles that exist for that frame, and their positions; etc. This would all be culled down to a virtual viewport.

The job of the client would then just be to receive this display-list, each frame, and draw it. It would also perhaps be responsible for doing client-side visual lerps between server ticks. (Probably each rigged limb would be annotated with linear+angular velocity properties.)

In other words, this would be a recapitulation of the X11 protocol, but where the primitives are 3D objects rather than 2D ones. (It'd kind of look like the protocol between games and OS display drivers, but at the same time higher-level in some ways and lower-level in others.)

This would be highly bandwidth-efficient (and jitter-tolerant!) compared to what Stadia is doing. You'd probably be able to play a decent game through this on 3G, even. But you'd need a computer that could do the final rendering passes.

[AndrewStephens]

Interesting idea that might work well for simpler games but what you call display lists on today's games can be very large. Modern video codecs are also very good, so the trade off is not as obvious as you make it seem.

And even then it wouldn't stop cheating. Bots would just look for triangles drawn with enemy uniform textures (very easy if you have the display lists) and aim at them.

[derefr]

> Bots would just look for triangles drawn with enemy uniform textures (very easy if you have the display lists) and aim at them.

I think you missed the "viewport culling" step in the above description. The server would only be sending the client enough information to draw what's going to be on the client's screen (since that's the only information that's naturally left after that step of the rendering pipeline!) So, on any frame where another player is obscured by a wall, data about that player wouldn't be in the display-list sent to the client, any more than it would be in the image Stadia sends.

[AndrewStephens]

And the during the first frame where the tiniest sliver of the enemy is visible around the edge of the wall, the bot running on the local computer will move the crosshair exactly over the visible area and shot.

This is ignoring things like player shadows which are rendered separately and need polygons which are not in the visible fulcrum.

Also we are both ignoring audio. Sounds like gunshots come from a specific place in the world.

[derefr]

> And the during the first frame where the tiniest sliver of the enemy is visible around the edge of the wall, the bot running on the local computer will move the crosshair exactly over the visible area and shot.

Well, yes, but I'd categorize that as "acts indistinguishable from theoretically-optimal human performance", rather than cheating per se. Or would you ban The Flash for cheating?

> This is ignoring things like player shadows which are rendered separately and need polygons which are not in the visible fulcrum.

No, not really; a shadow mesh is generated on the server, as part of the lighting step. The client then receives the shadow mesh, potentially disconnected from whatever's casting the shadow if the thing casting the shadow was culled. Just like what happens inside your GPU.

> Sounds like gunshots come from a specific place in the world.

In a paradigm like this, audio-cue triggers would essentially be "scripted particles" in the scene. If you can't see them, they're not rendered, so you can't hear them, either.

I'm not trying to describe something here that's a lossless approximation of how the game would work if run locally. I'm describing something with real effects on gameplay balance, but potentially good ones.

Mind you, I'm less picturing FPS games as the best use of this, and more picturing RTS games. Take the "fog of war" of a game like Starcraft, and implement it server-side, such that it's not the client not-rendering unexplored stuff, but the server.

[AndrewStephens]

Counterstrike aimbots have done what I describe for decades. They can aim and shoot in one frame, then aim and shoot at someone else as soon as the gun animation allows, exactly like playing against The Flash. The Flash might not be cheating per se, but he would be unsatisfying to play against as he decimates your entire team in a second.

Shadow meshes are often generated by rendering the scenes from the viewpoint of the light sources. This would have to be done on the server, so the server is already doing some rendering.

Sounds certainly need to be rendered, even for non-visible players. Otherwise you couldn't hear someone shooting behind you.

You ideas would work much better with a RTS. But so do solutions like Stadia.

[jimmydorry]

Dota2 (more accurately the source2 engine) already does this. The cheats are then limited to acting on things at bot speeds instead of at human reaction speeds.

[derefr]

That's cool! Is there a deep dive somewhere into the architecture of Source2 (or any game built with it) that you'd recommend?