[andybak]

> Gaussian splatting transform images to a cloud points.

Not exactly. The "splats" are both spread out in space (big ellipsoids), partially transparent (what you end up seeing is the composite of all the splats you can see in a given direction) AND view dependent (they render differently depending on the direction you are looking.

Also - there's not a simple spatial relationship between splats and solid objects. The resulting surfaces are a kind of optical illusion based on all the splats you're seeing in a specific direction. (some methods have attempted to lock splats more closely to the surfaces they are meant to represent but I don't know what the tradeoffs are).

Generating a mesh from splats is possible but then you've thrown away everything that makes a splat special. You're back to shitty photogrammetry. All the clever stuff (which is a kind of radiance capture) is gone.

Splats are a lot faster to render than NeRFs - which is their appeal. But heavier than triangles due to having to sort them every frame (because transparent objects don't composite correctly without depth sorting)

[vessenes]

Minor nit — in what way do splats render differently depending on direction of looking? To my mind these are probabilistic ellipsoids in 3D (or 4D for motion splats) space, and so while any novel view will see a slightly different shape, that’s an artifact of the view changing, not the splat. Do I understand it (or you) correctly?

[refibrillator]

In 3DGS, spherical harmonics are used to model view-dependent changes in color.

https://en.m.wikipedia.org/wiki/Spherical_harmonics

Basically for each Gaussian there is a set of coefficients and those are used to calculate what color should be rendered depending on the viewing angle of the camera. And the SH coeffs are optimized through gradient descent just like the other parameters including position and shape.

[vessenes]

Ah, thank you. Taking into account say reflection/refraction.