[_0ffh]

The problem for gaming is, these scenes are completely static.

[spyder]

Lot of game assets are static, and people are working on animated NERFs too. Also another necessary thing will be collisions and physics.

example for character animation:

https://zju3dv.github.io/animatable_nerf/

example for the static world already working in Unreal engine real-time:

https://www.youtube.com/watch?v=GjpzMDur7UY

diffusion models for NERF asset generation:

https://sirwyver.github.io/DiffRF/

https://dreamfusion3d.github.io/

Physics:

https://www.youtube.com/watch?v=Md0PM-wv_Xg

https://www.youtube.com/watch?v=Eklh1pIAri0

and there are a lot more research being done on this, it's hard to keep up with all the things happening in AI

[alphabetting]

True but even for static areas and backgrounds I feel like this detail and fidelity blows away what is used in current AAA games.

[NoOneNearMe]

I think a bigger problem is how much computational resources NERF models need. From the paper,

"Our model, Mip-NeRF 360, and our 'mip-NeRF 360 + iNGP' baseline were all trained on 8 NVIDIA Tesla V100-SXM2-16GB GPUs."

Even with all that, it takes almost an hour to train. A developer would need to do this for every scene, which may make it unfeasible for indies, those who would benefit the most from NERF in gaming.

[piyh]

The better question is how many person hours would it take to model a scene like that with alternative approaches, and can a NeRF be used to create outputs that could be used in a game engine?

[mrguyorama]

That's basically just FMV games all over again.