[sho_hn]

> I do not get the point of this at all

Dunno, this seems like an avenue definitely worth exploring.

Plenty of game applications today already have a render path of input -> pass through AI model -> final image. That's what the AI-based scaling and frame interpolation features like DLSS and FSR are.

In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps.

Experiments like the OP's are about moving that boundary and "prompting" with a lower-fidelity input and having the model do more.

Depending on how well you can tune and steer the model, and where you place the boundary line, this might well be a compelling and efficient compute path for some applications, especially as HW acceleration for model workloads improves.

No doubt we will see games do variations of this theme, just like games have thoroughly explored other technology to generate assets from lower-fidelity seeds, e.g. classical proc gen. This is all super in the wheelhouse of game development.

Some kind of AI-first demoscene would be a pretty cool thing too. What's a trained model if not another fancy compressor?

[coolKid721]

Compared to just coming up with and having solid systems for generating game assets? Actually having decent quality style consistent 3d models, texture work, animation, sound effects, etc (especially if it were built into say a game engine or something) would actually revolutionize indie game dev. Games are fundamentally artistic works so yes anything decent will actually require tailoring and crafting, AI set up to serve those people actually makes sense and is totally technically feasible are way easier problems to solve.

And no if you heavily visually modify something with AI models to the extent it significantly alters the appearance it simply has no way of being consistent unless you include the previously generated thing somehow which then has the huge problem of how do you maintain that over an 80 hour game? How do you inform the AI what visual elements (say text, interactive elements) are significant and can be modified and which can't? (You can't)

Actually using AI to generate assets, having a person go in to make sure they look good together and make sure they match then just saving them as textures so they function like normal game assets makes 10000x more sense then generating a user image then trying to extract "hey what did the wrapper of this candy bar look like" from one ai generated image and figuring out how to make sure that is consistent across that type of candy bar in the world and maintains that consistency throughout the entire game, instead of just you know, generating a texture of a candybar?

[BizarroLand]

I think you're making a lot of good points for the current SOTA.

That being said, it took us a few hundred years all in all just to work out paint, so if people keep working with this tech eventually a game designer could, in theory, lay out the skeleton of a game and tell an AI to do the rest, give it pointers and occasional original art to work into the system, and ship a completed playable game in days.

Whether it will be worth playing those games is an entirely different enchilada to microwave.

[thwarted]

> lay out the skeleton of a game and tell an AI to do the rest, give it pointers and occasional original art to work into the system, and ship a completed playable game in days

"But, think of the indie game designer!" is getting to be quite the take.

We have a machine that produces slop and the selling point is how fast it produces it? And how more people should be using it to spend less time on creative aspects? Would the world be a better place if GRRM "finished" his most well-known work sooner rather than never?

Something about the phrase "tell an AI to do the rest, give it pointers" reminds me of "The Sorcerer's Apprentice" from Fantasia. Not in the surface level dire-warning about laziness, automation, and losing control that story is telling but in that Mickey didn't spend any time thinking about what he was doing and the Wizard's disappointment at the end.

[coolKid721]

This whole attitude is just the attitude of tech demos. Nothing good or worthwhile will be "completed playable game in a couple days" actually making anything good takes a huge amount of time and effort and thought. Empowering a small indie studio or solo indie dev so they could make something AA or AAA quality should be the actual goal. If you have 4 people able to make a skyrim level game in a year or two that's an insane feat, that should be the goal. Not someone who doesn't give a shit throwing some prompt and making some slop game that is exactly like 500 other slop games people generate with one prompt.

Like with that tech what kind of games would say random solo developers plugging at it and refining it be able to make in 4 years, that is the extremely compelling stuff. One person being able to make some auteur AAA quality game on their own, even if it takes a long time that might actually be good. If there are AI games those are the ones I'd want to play.

[troupo]

*Edit: misread the post I replied to, disregard comment contents*

> Plenty of game applications today already have a render path of input -> pass through AI model -> final image.

Where "plenty" equals zero?

> In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps.

That is, in these cases you already have high-fidelity input in the form of an actual game, and "AI" contributions to the output are dubious at best.

Do you really believe it's DLSS that's doing all the heavy lifting in a game like Expedition 33 or Cyberpunk 2077?