| It's for deep learning, not that much "for graphics". The problem is "how do you do 3D deep learning 3D scene reconstruction" aka "how to make 3d equivalent of stable diffusion". Voxels are bad as they take too much memory and are uniform (each voxel is eg 5x5x5 cm). Gausians are kinda like "variable sized voxels" that also have orientation, color based on angle viewing and can stretch. Imagine 3D blobs basicaly (3D capsules or more like 3D density bubbles with transparency). So the scene can be represented in 3D much more efficiently using gausians splats, which is why they are used for "3d stable diffusion". So now that this has come into use we also need efficient way or rendering them. |
> It's for deep learning, not that much "for graphics".
No, while it is true that there is some overlap between the techniques and concepts used, gaussian splatting isn't necessarily "for deep learning". The library provides a differentiable rasterizer for gaussian splats. This basically means that you can ask it "if I want my output image to change in this and this direction, in what direction should I change the position / orientation / color / ... of my splats?". This enables users to plug it into other software (that is also commonly used for deep learning) and to optimize the parameters of the splats to represent a particular scene.
Since it's primarily a differentiable rasterizer for splats I think it's fair to say that it is "for graphics".
> The problem is "how do you do 3D deep learning 3D scene reconstruction" aka "how to make 3d equivalent of stable diffusion".
That it uses gradient descent doesn't mean that it is "deep learning". There are no neural networks or layers here.
It's not an "equivalent of stable diffusion". The way it's used now is to learn a representation of a single scene, not unlike photogrammetry. Sure, there may be other use cases for this library, but this is primarily what gaussian splatting is about.