[jsheard]

> We need to incorporate a few sample rays at least so that it 'knows' what's actually off-screen, and then we just give it lots of training data of partially rendered images and fully rendered images so that it learns how to fill in the gaps.

That's already a thing, there's ML-driven denoisers which take a rough raytraced image and do their best to infer what the fully converged image would look like based on their training data. For example in the offline rendering world there's Nvidia's OptiX denoiser and Intel's OIDN, and in the realtime world there's Nvidia's DLSS Ray Reconstruction which uses an ML model to do both upscaling and denoising at the same time.

https://developer.nvidia.com/optix-denoiser

https://www.openimagedenoise.org