[phire]

This technique isn't actually speeding up the NeRF rendering algorithm.

It bakes the NeRF back to a semi-discrete representation (Octree of Spherical Harmonics voxels) which can render near-identical results at interactive speeds.

The baked data is much larger than the original NeRF model (2gb vs 5mb), but they can be downsampled to 30-100mb with little loss in quality.

[arduinomancer]

So if I understand right, for the real-time version rather than querying the NeRF to compute the frame pixels on the fly, they instead use the NeRF to pre-generate 3D Voxel data representing the scene which can then be rendered in real time using more traditional voxel rendering?

[phire]

Yes and No.

This preserves the exact lighting equation that the NeRF learned, while traditional voxel rendering is limited to traditional lighting equations.

You would have a hard time voxelizing a NeRF, because you can't extract a traditional lighting equation out of it.

[pmayrgundter]

I think this is sensitive to Hinton's work on capsules, which I believe is a more reprojectible primitive. Maybe you can coax a voxel

[pmayrgundter]

The point about spherical harmonics hits home. You could sample the different harmonics with a probabilistic scattering to construct a probability distribution for a signed distance function render, and use half the very solid render pipeline