[toshinoriyagi]

While some very large models may need beefy hardware, there are multiple forms of deep learning used for similar purposes:

Nvidia's DLSS is a neural network that upscales images so that games may be rendered quickly at lower resolutions, and than upscaled to the display resolution in less total time than rendering natively at the display resolution.

Nvidia's DLDSR downscales a greater-than-native resolution image faster than typical downscaling algorithms used in DSR.

Nvidia's RTX HDR is a post-processing filter that takes an sRGB image and converts it to HDR.

So, it is very likely that a model that converts rasterized images to raytraced versions is possible, and fast. The most likely road block is the lack of a quality dataset for training such a model. Not all games have ray tracing, and even fewer have quality implementations.

[jsheard]

To be clear DLSS is a very different beast than your typical AI upscaler, it uses the principle of temporal reuse where real samples from previous frames are combined with samples from the current frame in order to converge towards a higher resolution over time. It's not guessing new samples out of thin air, just guessing whether old samples are still usable, which is why DLSS is so fast and accurate compared to general purpose AI upscalers and why you can't use DLSS on images or videos.

[jms55]

To add to this, DLSS 2 functions exactly the same as a non-ML temporal upscaler does: it blends pixels from the previous frame with pixels from the current frame.

The ML part of DLSS is that the blend weights are determined by a neural net, rather than handwritten heuristics.

DLSS 1 _did_ try and and use neural networks to predict the new (upscaled) pixels outright, which went really poorly for a variety of reasons I don't feel like getting into, hence why they abandoned that approach.

[mywittyname]

> So, it is very likely that a model that converts rasterized images to raytraced versions is possible, and fast.

How would this even work and not just be a DLSS derivative?

The magic of ray tracing is the ability to render light sources and reflections that are not in the scene. So where is the information coming from that the algorithm would use to place and draw the lights, shadows, reflections, etc?

I'm not asking to be snarky. I can usually "get there from here" when it comes to theoretical technology, but I can't work out how a raster image would contain enough data to allow for accurate ray tracing to be applied for objects whose effects are only included due to ray tracing.