[diggan]

Show. Us. The. Wireframes!

Every single time a new "Generate 3D" thing appears, they never show the wireframes of the objects/scenes up front, always you need to download and inspect things yourself. How is this not standard practice already?

Not displaying the wireframes at all, or even offer sample files so we could at least see it ourselves, just makes it look like you already know that the generated results are unusable...

[jsheard]

They usually don't show the material channels either, which I assume is because there aren't any, and instead the lighting is statically baked into the asset. That works for a demo where you just wiggle the camera in a circle, but it'll immediately fall apart if the lighting environment changes or anything in the scene moves.

[Legend2440]

Think of it more like a 3D picture than an animation model.

There are no materials channels or wireframe. It’s a volumetric 3D representation, like a picture made up of color blobs.

[kookamamie]

And thus unusable for most things 3D models and scenes would be used for today.

[hwillis]

Apples, not oranges. This isn't for 3d models and scenes- think of it as a fancy version of streetview or an apartment walkthrough. Those project pictures onto a sphere around you. Gaussian splats are an improvement that uses multiple images to interpolate viewpoints- take two pictures from different angles, and guassian splats let you take a look from between those views, or above or below them.

This method uses ai to generate even more unseen structure, so with relatively few images you can still represent a real scene with some level of fidelity. It will never need dynamic lights or animation because the point is just to look as close as possible to a still image. Splats do that FAR better and more efficiently than you ever could with dynamic lighting, triangulated models, and visual effects.

[ForTheKidz]

I see your point, but also consider that interactivity comes to mind in part because 3d models are so expensive to describe compared to 2d shapes that they're largely worth it for interactive stuff. We might see more innovation on that front with a low-cost barrier to entry.

Plus, scaling dynamic lighting up has always been the Big Bad of computer graphics, and precomputation will always give us an amazing heuristic to use against it. Everything else basically tends towards not mattering: we can only absorb a finite number of details, but we live in a world with virtually infinite lights.

[teamonkey]

Honestly, I thought this was the most practical and usable example of AI generation I've seen to date. I actually found it refreshing after all the guff we usually see.

I bet in a couple of years it'll be standard for estate agents to show 3D views like this on their web sites, architects converting quick paintovers of existing sites to 3D models, improvements to Street View, and so on. Anywhere where you want a quick 3D view of a space based on a few photos taken on a smartphone and where accuracy isn't 100% important.

For things like games, it still follows the existing photogrammetry workflow (with all of those problems), but it might reduce the number of photos needed to create a point cloud.

[ajwin]

My understanding is that it is not mesh, it’s Gaussian Splatting. There are tools to convert Splats into mesh though.

[diggan]

Yeah, but isn't still the expected outcome to end up with actual 3D objects, not point clouds? Or did people start integrating point clouds into their 3D workflows already? Besides for stuff like volumes and alike, I think most of us are still stuck working with polygons for 3D.

[tracerbulletx]

No geometry in the conventional sense. I did a demo of rendering a Gaussian splat in React Three Fiber here, you can open the linked splat file (its hosted on hugging face) if you want to see the data format. https://codesandbox.io/p/sandbox/3d-gaussian-splat-in-react-... I also have this youtube video about creating that demo https://www.youtube.com/watch?v=6tVcCTazmzo

[lmpdev]

I use pointclouds all the time in Rhino/Lastools/Meshlab

I much prefer pointclouds and nurbs over meshes

Not everything is gamedev

[diggan]

> Not everything is gamedev

Agree, I'm not sure why you'd think that's the only use case for 3D, unless I misunderstand your argument here.

How would you handle visual effects with point-clouds for example? There are so many use cases for proper 3D, and all I can think of as use-case for point clouds are environments with static lightning, which seems like a really small part of what people generally consider "3D scenes".

[lmpdev]

> Visual effects

Maybe I missed the mark on “gamedev”, but 3D is larger than just “aesthetically pleasing 3D VFX” for its own sake

Often I’m trying to use something as a reference for a design where a 3D model isn’t the actual end goal, or I’m performing analytics on a 3D object (say in my case for a lot of GIS and simulation work)

The whole “mesh is the be all and end all of 3D modelling” irks me as while yes it’s a really important way of representing an object (especially with real time constraints), it doesn’t do justice to the full landscape of techniques and uses for 3D

It would be like 2D sprite artists from the gamedev world saying “what’s the point of all this vector art you illustrators are doing” or “what’s the point of all these wireframe designs you graphic designers are doing” - “these aren’t raster images!”

I suppose my snipe was trying to communicate the idea that 3D is larger than just a vehicle for entertainment production. It intersects many industries that may eschew polygons because real time rendering is irrelevant

3D tooling has uses beyond producing 3D scenes, just as Photoshop is used for more than touching up photographs

Edit: for anyone stuck in a rut with meshes come join the dark side with nurbs - it makes you think about modelling in a radically different way (unfortunate side effect is it makes working with meshes feel so so “dirty”)

[CyberDildonics]

The whole “mesh is the be all and end all of 3D modelling”

No one said this, it seems like you are making up fake questions and not dealing with the actual questions that the person you replied to asked.

You can view point clouds and you can warp them around, but working with them and tracing rays becomes a different story.

Once you need something as a jumping off point to start working with, point clouds are not going to work out anymore. People use polygons for a reason. They have flexible UVs, they can be traced easily, they can be worked with easily, their data is direct, standard and minimal.

[MITSardine]

Games are the least of it, the vast majority of scientific applications to do with physics use meshes rather than point clouds.

This is because a point cloud does not represent a surface or a volume until the points are connected to form, well, a surface or a volume.

And physical problems are most often defined over surfaces or volumes. For instance, waves don't propagate over sparse sets of points, but within continuous domains.

However, for applications where geometric accuracy is needed, I think you wouldn't want to use a method based on a minimal number of photographs anyways. For instance, the Lascaux cavern was mapped in 3D a decade ago based on "good old" algorithms (not machine learning) and instruments (more sophisticated than a phone camera). So these critiques are missing the point, in my opinion. These Gaussian Splatting methods are very impressive for the constraints they operate under!

[Legend2440]

You can convert splats into meshes using a simple marching cubes algorithm.

But the meshes produced are not easy to edit.

[dheera]

Generating good meshes sounds like a problem for a completely different machine learning algorithm to me.

[MITSardine]

Meshing has been around long before machine learning came to prominence, there's plenty of methods to improve surface meshes already.

[Legend2440]

None of which work.

There is no good method to take a 3D scan and make a sensible mesh out of it. They tend to have far more vertices than necessary and lack structure.

[hwillis]

If you wanted to show someone a walkaround of the Sistine chapel or David, would you be better off using triangles and PBR and raycast lighting? You don't really gain anything from all that; you're doing a tremendous amount of computation just to recapture the particular lighting at an exact time. If you want the same detail that a few good pictures capture -tens of millions of pixels- you need to have many billions of triangles onscreen.

With splats you can have incredibly high fidelity with identical lighting and detail built in already. If you want to make a game or a movie, don't use splats. If you want to recreate a static scene from pictures, splats work very well.

[text0404]

splats augment 3D scenes, they don't replace them. i've seen them used for AR/VR, photogrammetry, and high-performance 3D. going from splats to a 3D model would be a downgrade in terms of performance.

[dmarcos]

What’s the best use of splats that you’ve seen so far that I can try? AR/VR or regular 3D

[nine_k]

Meshes are editable. Are Gaussian splats?

[dmarcos]

What kind of edits you mean? You can crop / combine splats easily in your browser with supersplat (not affiliated)

https://superspl.at/editor

[text0404]

kinda but not really in a meaningful way, at least not yet. there's some plugins for popular 3D software but it's still early days.

[therealpygon]

Yea, someone can say, “Look, we have just created the first color computer and it displays images. Look at this first ever real life photo on this digital screen!” There will always be the people who ask, “Yeah, but does it run Photoshop?”

[lallysingh]

Expected by whom? Other researchers in this space? That's the audience for this work.

[dheera]

Not necessarily.

If you're using it to render video you don't need to go into the mesh world.

[eMPee584]

Isn't https://svraster.github.io/ just superceding gaussians? Voxels are also not meshes, but might they not prove even more useful for coming rendering engines..?

[text0404]

splats don't have wireframes, and they have an embedded webgpu viewer in the linked page.

[quitit]

I think there should be a standard set of images for comparison, because I've never seen a mesh generator readme that wasn't impressive. I test each one I get my hands on and the results are often disappointing.