[slalomskiing]

Can someone explain why there’s so much interest in these recently?

How is it related to NeRFs?

And how is it different/better than rendering triangles/voxels/SDF?

Is it just ray-marching these 3D gaussian shapes?

Why is a gaussian used specifically?

[corysama]

Splatting various kinds of blobs is actually an old idea. It’s a branch of point cloud rendering that researchers have been talking about for a couple decades now. What’s new is the technique to create the splats from photos.

NeRFs are made of a dense grid of tiny AIs that each can answer what the inside of that grid cube looks like from any angle. You march through the grid to see how a whole scene looks from a particular angle.

Gaussian Splats target the same goal with a different approach. A splat is just a fuzzy oriented blob. You render them by sorting them front to back then marching through them per pixel until it gets opaque enough.

The Gaussian aspect is important because Gaussians are differentiable. With that you can start with a bad estimation of the splats, render the bad estimation, compare it to the photos and back-propagate the difference between actual vs desired to iteratively improve the bad estimate to a better one.

Triangles are great because you can make solid, closed surfaces from them. But, you have to maintain the connectivity between the vertices. That makes adding and removing triangles from a mesh a major challenge.

SDFs are great because you can easily make crazy changes to the topology without ever thinking about connectivity. But, you have to be able to describe the shape of the whole scene with math. For realistically complicated shapes, that adds up to a lot of math per marching step.

Splats are loose, independent blobs. They are easy to individually manipulate. Adding and removing them is trivial. But, it’s difficult to get the appearance of a solid surface from them.

[enriquto]

> > Why is a gaussian used specifically?

> The Gaussian aspect is important because Gaussians are differentiable.

Sure, but why Gaussians and not any of the many other well-known smooth functions? Is it because of lack of imagination (since the Gaussian is a sort of "default" choice)? Or is there some reason to pick gaussians, specifically?

[mediumfi]

One additional reason is that the projections of a Gaussian are also Gaussian (more generally any linear transform of a Gaussian is Gaussian). So if you have the parameters of a 3D Gaussian ellipsoid and a viewing angle, you can quite easily get the parameters of the corresponding 2D ellipse. So the ray marching algorithm turns into just blending a stack of 2D ellipses. For many other non Gaussian functions projection is trickier.

[enriquto]

Ah, that is a very good reason indeed! At least for projecting from 2D to 1D the gaussian is the only distribution with this property. The functional equation g(x)g(y)=g(√(x^2+y^2))g(0) determines the gaussian up to a constant.

[T-A]

Also, their derivative is just a multiplication (if normalized) or two (if not).

https://hannibunny.github.io/orbook/preprocessing/04gaussian...

[amelius]

To me it sounds silly to name a technique after a minor implementation detail of the technique.

[lostdog]

NeRFs were cool, but it turns out you get just-as-good results by fitting these Gaussian blobs. Plus it's faster. Plus the representation of gaussians is much easier to edit than the opaque neural representation.

It's better than other representations because...no one has been able to reconstruct scenes with other representations this well.

Gaussian splatting is not popular because it's a good rendering method. It's popular because there's now a good technique for generating great 3d gaussian splats from 2d images. And now that you have gaussian splats, it's nice to render them quickly.

[orbital-decay]

Inspired by NeRFs (renders radiance fields as well) but is faster. It's a point cloud with fuzzy points that have size, shape, and direction, not just color/alpha. It has the same niche and up/downsides as point clouds, and the recent wave of interest is due to the easy reconstruction and rendering of photorealistic scenes from a few photos.

[nox100]

I'm probably wrong and I honestly hope I'm wrong but AFAICT this is a dead end. even a small scene takes 30-50meg and is only viewable withim a short distance of the center of the data.

If you wanted the inside of an apartment or house like matterport it would easily be 500mb to 1gig of data and forget it if you try an outdoor scene

[bradleyishungry]

It’s not related to NeRF’s at all. It does use CUDA but it’s just rendering a scene with particles as gaussians in screen space

[slalomskiing]

I don’t think that’s correct, the abstract page of the original paper mentions NeRFs

https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

[bradleyishungry]

You’re correct in that, i could have picked different wording. it can be used to create radiance fields, but in a way that is completely different than NeRFs.