[baxuz]

I'm trying to understand this but I'm not sure I'm following.

As far as I understand as a complete beginner in 3D rendering:

Regular 3d models/meshes are just a collection of points/triangles in a 3d plane, along with some additional data like normal orientations. For rendering, the mesh is constructed and then rendered with some perspective/distance/occlusion settings etc. for each pixel painted on screen.

This format isn't a discrete collection of points but some data set where every point queried during 2d painting returns a distance (if it exists) based on some function or ML algorithm, basically returning a point cloud?

[GreenHeuristics]

Think of it like so:

float distance_to_surface_of_model(x, y, z);

For every point in 3d space you can get the distance to the surface. So, if you have a camera you can shoot a ray into the scene and pick some points on it to ask if it is inside the model. if so you paint that pixel with the surface material of the model. Then it's just a matter of picking a good way of sampling points along the camera ray. usually this distance function is some basic math (distance to sphere or so). But here they take a 3d mesh and create a neural network that can answer the question of distance to it (since meshes are very slow at that).

[dartmoose]

Meshes are actually extremely fast for querying distance to a surface. Even in scenes with lots of geometry, using an acceleration structure like a BVH can be extremely fast. You can even do ray tracing in real time on a GPU now (at least for direct lighting, certainly not for global illumination).

By contrast, SDFs require you to do lots of expensive lookups as you slowly march along a ray until you happen to bump into a surface.

[skye-adaire]

This is a bit confused. Current RT implementations are faster at intersecting the mesh along the ray direction than sphere tracing (which require the expensive lookups or evaluations).

The distance to the surface is independent of direction, it is only a function of position (as all SDFs are).

The depth map is the distance from ray origin to intersection only, not how close that ray came to the surface at any point along it's segment.

[dartmoose]

That's fair, my point is that if you only care about rendering something quickly then a mesh is your best bet.

Are there applications you have in mind where the distance to the nearest surface is actually useful, regardless of the surface representation? As in, when would knowing the distance to the nearest surface be helpful when you're working with meshes? It seems to me you only need to know the closest distance to a surface when you have an SDF and are trying to compute an intersection along a ray.

The nearest sort of application of this kind that I can think of is Walk on Spheres (cool recent paper https://www.cs.cmu.edu/~kmcrane/Projects/MonteCarloGeometryP...), but in that case the authors do a closest point query, not a distance to the nearest surface. There's a subtle difference here because you actually need the closest point (not just the distance) so that you can look up a boundary condition. It's not clear to me how you could make use of an SDF in this setting, although maybe there's an interesting research direction there.

[skye-adaire]

>Are there applications you have in mind where the distance to the nearest surface is actually useful, regardless of the surface representation?

Distance fields are a subset of implicit functions (see wikipedia). The use of level curves (which can be interpreted as "distance" for SDFs) is broad. You can dynamically and trivially thicken or shrink objects by adding or subtracting to the distance. Functions defined on the level curves output of an SDF can be used to map a volume to another implicit function (see nTopology). With implicit functions, the non-zero level sets are really just as important, but most focus on the zero level set in rendering.

>There's a subtle difference here because you actually need the closest point (not just the distance) so that you can look up a boundary condition.

Vector to Closest Point (VCP) is subtle but different in important ways. It is unsigned, and cannot indicate whether a position is inside or outside the object. Implicit modeling is not generally concerned with parameterizing the surface, since we can use volumetric, 3D procedural textures.

[dartmoose]

Thanks for the insights! I come from a mostly rendering background, so my understanding is definitely biased from always thinking of SDFs in the context of surface rendering or reconstruction :-)

[GreenHeuristics]

You are not querying the mesh you are querying the accel structure.

To show why it is wrong: Change the mesh every frame.