[yread]

I think it sounds a lot like ray tracing:

The Unlimited Detail engine works out which direction the camera is facing and then searches the data to find only the points it needs to put on the screen it doesn’t touch any unneeded points, all it wants is 1024x768 (if that is our resolution) points, one for each pixel of the screen.

[elblanco]

Maybe just first bounce? Cast rays out perpendicular from the lense (to handle FOV) and return back any point at the first bounce.