[slalomskiing]

I wonder since this runs at real time framerate if it would be possible for someone to composite a regular rasterized frame on top of something like this (with correct depth testing) to make a game

For example a 3rd person game where the character you control and the NPCs/enemies is raster but the environment is all radiance fields

[jasonwatkinspdx]

If you go to one of the demos the space bar will cycle through some debug modes. One shows a surface reconstruction. It comes from the usual structure from motion techniques I presume, so it's coarse and noisy, but I think the fundamental idea is viable.

[duckworthd]

This should absolutely be possible! The hard part is making it look natural: NeRF models (including SMERF) have no explicit materials or lighting. That means that any character inserted into the game will look out of place.

[vanderZwan]

Why bother to make it look natural when you can have a really awkward greenscreen-like effect for nostalgic and "artistic" purposes?