[mintone]

I've been bullish[1] on this as a major aspect of generative AI for a while now, so it's great to see this paper published.

3D has an extremely steep learning curve once you try to do anything non-trivial, especially in terms of asset creation for VR etc. but my real interest is where this leads in terms of real-world items. One of the major hurdles is that in the real-world we aren't as forgiving as we are in VR/games. I'm not entirely surprised to see that most of the outputs are "artistic" ones, but I'm really interested to see where this ends up when we can give AI combined inputs from text/photos/LIDAR etc and have it make the model for a physical item that can be 3D printed.

[1] https://www.technicalchops.com/articles/ai-inputs-and-output...

[aabhay]

I have to be a soggy blanket person, but there are some pretty strong reasons why 3D generative AI is going to have a much shallower adoption cycle than 2D. (I founded a company that was likely the first generative AI company for 3D assets)

1. 3D is actually a broad collection of formats and not a single thing or representation. This is because of the deep relation between surface topology and render performance. 2. 3D is much more challenging to use in any workflow. Its much more physical, and the ergonomics of 3DOF makes it naturally hard to place in as many places as 2D 3. 3D is much more expensive to produce per unit value, in many ways. This is why, for example, almost every indie web comic artist draws in 2D instead of 3D. In an ai first world it might be less “work” but will still be leaps and bounds more expensive.

In my opinion, the media that have the most appeal for genAI are basically (in order)

- images - videos - music - general audio - 2D animation - tightly scoped 3D experiences such as avatars - games - general 3D models.

My conclusion from being in this space was that there’s likely a world where 3D-style videos generated from pixels are more poised to take off than 3D as a data type.

[noduerme]

You skipped an important market segment. Porn. The history of technology for making and distributing images, from cave paintings to ancient Rome through the Renaissance, Betamax, Photoshop, Tomb Raider and Instagram filters, has always been driven by improving the rendering of boobs.

[robust-cactus]

At this point GLTF seems pretty darn good and seems broadly usable. It embeds the mesh, textures, animations right in a single file. It can represent a single model or a scene. I also has a binary format.

3d need not be so complicated! We've kinda made it complicated but a simplification wave is likely coming.

The big unlock for 3d though will have to be some better compression technology. Games and 3d experiences are absolutely massive.

[catgary]

There’s a middle ground in animation where you use techniques like text2motion with pre-existing assets that, I think, has a lot of appeal for games.

[fartfeatures]

Doesn't Nvidia's USD format solve the multiple formats issue?

[unconed]

I think you mean Pixar's.

USD was designed for letting very large teams collaborate on making animated movies.

It's actually terrible as an interchange format. e.g. The materials/shading system is both overengineered and underdefined. It is entirely application specific.

For the parts of USD that do work well, we have better and/or more widely supported standards, like GLTF and PBR.

It was a very dumb choice to push this anywhere.

[fartfeatures]

Yeah that's the one. Did you see they were phasing out the materials support in it for materialX?

[slidehero]

>I've been bullish[1] on this as a major aspect of generative AI for a while now, so it's great to see this paper published.

Me too. My first thought when seeing 2D AI generated images was that 3D would be a logical next step. Compared to pixels, there's so much additional data to work with when training these models I just assumed that 3D would be an easier problem to solve than 2D image generation. You have 3D point data, edges, edge loops, bone systems etc and a lot of the existing data is pretty well labeled too.

[lallysingh]

I'm excited to see the main deterrents to Indy gave dev: art and sound, get AI'd away. A single developer could use an off-the-shelf game engine and some AI generated assets (perhaps combining with whatever they can buy cheap) to develop some fun games.

Still, getting from still models to something that animates is necessary:(

[vermilingua]

This is exactly what I’m not excited for: indie game discovery is already hard enough, Steam widening the floodgates has not been a positive experience. Reducing the effort to create games even further is going to DoS the indie game market as we see the same studios that pump out hundreds of hentai games suddenly able to broaden their audiences significantly.

[livrem]

I think having modern game engines reducing the need for game programmers to almost zero caused much of this, but it also resulted in some interesting games when artists could create games without a need to hire programmers.

It will be interesting to see if AI art (and AI 3D models) will mean that we see interesting games instead created by programmers without having to hire any artists.

What I do not look forward to is the predictable spam flood of games created without both artists and programmers.

[l33tman]

To be fair, this is already the case on all the platforms, as you can easily put together a game with free assets from the assetstores (or pay a few dollars for pretty high quality assets). For every standard game genre you can imagine I'm sure there are thousands of generic games released every year on every playform (don't have any real numbers but I get that feeling...)

Rendering the assets by AI or buying them from the asset store is not going to change the number of generic games put out there I think, maybe AI gen can make some of them a bit more unique at best.

[drschwabe]

That's the idea, you won't have to discover - instead you can just create the game you want.

[latentsea]

Be careful what you wish for.

[noduerme]

Passable art is common. Original and interesting game mechanics are exceedingly rare, and will continue to be. The relationship between passable art and throwaway games is like that between bland AI content and marketing blogs.

Really good games will still employ really good artists.

[vermilingua]

This is my point exactly, but even passable art takes some time to create. I’m not excited for the very-soon-to-arrive tide of VNs, deckbuilders, and JRPGs made with effectively 0 time or effort.

[brookst]

I’ve never understood the effort = quality view of art. Just because someone spent thousands of hours does not mean it is good art. And plenty of great art is executed quickly.

It seems as odd to me as bemoaning the way word processors let people write novels without even being good typists.

[noduerme]

What is an example of some great art that was executed quickly and/or without a great many hours of prior experience on the part of the artist?

[noduerme]

But this is a category that didn't exist yet. So who knows what people without art skills or budgets might do? Probably nothing, but maybe one in ten thousand actually isn't garbage. Just like music at the advent of digital home recording. The market is already so flooded it hardly matters.

I'm an artist and a gentleman coder and I'm disgusted and offended by careless work. But I don't think I need to die on the hill of stopping infinite crappy game mills from having access to infinite crappy art.

[edit] I'm also just bitter after years working on pretty great original art / mechanics driven casual games that only garnered tiny devoted fan bases, and so I assume that when it comes to the kinds of long tail copycat games you're talking about, especially with AI art, no one's going to bother playing them anyway.

[kilpikaarna]

Lol, yeah, the main deterrent/obstacle to indie game dev has little to do with actual development, and machine generated content is actively making that worse.

[DrSiemer]

So we should not improve production methods, because it will give us more things for less effort?

Just let the market sort it out. I for one can't wait for the next Cyriak or Sakupen, that can wield the full power of AI assistance to create their unique vision of what a game can be.

[milofeynman]

Autodesk has been building practical 3d models for years with generative design. I have to imagine it's only getting better with these recent advances, but I'm not immersed in the space.