[binalpatel]

This is super impressive.

Interesting they're framing this more from the world model/agent environment angle, when this seems like the best example so far of generative games.

720p realtime mostly consistent games for a minute is amazing, considering stable diffusion was originally released 2ish years ago.

[uoaei]

Pixelspace is an awful place to be generating 3D assets and maintaining physical self-consistency.

[jeroenvlek]

Ultimately even conventional 3d assets are rendered into pixelspace. It all comes down to the constraints in the model itself.

[psb217]

A key strength of conventional 3d assets is that their form is independent of the scenes in which they will be rendered. Models that work purely in pixel space avoid the constraints imposed by representing assets in a fixed format, but they have to do substantial extra work to even approximate the consistency and recomposability of conventional 3d assets. It's unclear whether current approaches to building and training purely pixel-based models will be able to achieve a practically useful balance between their greater flexibility and higher costs. World Labs, for example, seems to be betting that an intermediate point of generating worlds in a flexible but structured format (NERFs, gauss splats, etc) may produce practical value more quickly than going straight for full freedom and working in pixel space.