[haccount]

It's an early demo of interactive realtime inference but it appears to have a promise of promptable game worlds and mechanisms. Or "scriptable dynamic imagination" if you will.

The answer to "why?" when DeepDream demoed hallucinated dog faces in 2015 was contemporary diffusion models.

[hnlmorg]

The dog demo was introducing something new. This isn't.

I don't want to be negative about someone else's project but I can completely understand why people are underwhelmed by this.

What I think will be the real application for AI in gaming isn't creating poorer versions of native code, it will be creating virtual novels that evolve with the player. Where characters are actual AI rather than following a predefined script. Where you, as the player, can change and shape the story as you wish. Think Star Trek Holodeck "holo-novels" or MMORPGs but can be played fully offline.

Rendering the pixels is possibly the worst application for AI at this stage because AI lacks reliable frame by frame continuity, rendering speed, nor an understanding of basic physics, which are all the bare minimum for any modern games engine.

[stale2002]

> What I think will be the real application for AI in gaming isn't creating poorer versions of native code

"Why would you use this basic image diffusion model? It just creates a poorer version of existing images!" Is what your statement sounds like.

Obviously, you'd do the same thing with this game engine as is done with images. You combine together multiple world models, and thats almost the same as creating a new game.

Imagine playing minecraft and you could add a prompt "Now make complex rules for building space ships that can fly to different planets!" and it just mostly works, on the fly.

[hnlmorg]

> “Why would you use this basic image diffusion model? It just creates a poorer version of existing images!" Is what your statement sounds like.

I’m not the OP and that’s not what my reply “sounds like”. You’re not reading my comment charitably.

> Obviously, you'd do the same thing with this game engine as is done with images. You combine together multiple world models, and thats almost the same as creating a new game.

That was actually my point.

Adding some VFX to a game engine doesn’t alter the game engines state.

What some of the proponents of this project are describing is akin to saying “ray tracing alters the game mechanics”.

It doesn’t matter how smart nor performant the AI gets at drawing pixels, it’s still just stateless pixels. This is why I was discussed practical applications of AI that can define the game engines state rather than just what you see on screen.

Image diffusion models specifically are not the right applications of AI to create and alter a persistent game state. And that’s why people are underwhelmed by this project.

[dartos]

> it appears to have a promise of promptable game worlds

Is it a world if there’s no permanence?

We’ve seen demos like this for a while now (granted not as fast) but the core problem is continuity. (Or some kind of object permanence)

It’s a problem for image generators as well.

I’d be more interesting if that was any closer to being solved than to have a real time Minecraft screenshot generator.

I may have missed it, but I didn’t see anything about prompting. I’d be surprised if this model could generalize beyond Minecraft at all.

[thirdacc]

>It’s a problem for image generators as well.

It was, about a year ago. It's a solved problem.

[dartos]

I haven’t seen any image generator maintain spatial consistency for more than a few seconds of video (maybe a min for runway actually)

Definitely nothing that maintains it for hours on end.

If you have, please link them. I’m very interested.