[theptip]

I've been pondering this recently; I think there are some really fun applications of LLMs to game dev that have not yet been explored.

Going back to the original AI Dungeon (text-based adventure game, fully-generated by LLM), it was a lot of fun, but hallucinations were crazy. You could just ad-lib a change to the world-state and it would run with it.

One possibility would be to have an LLM act as a DM and use Tool APIs to drive a world model; for example, the DM can call `moveMonster(newPosition)` a la the Voyager paper. The game engine enforces constraints, so you can't just conjure items out of thin air. The player isn't interfacing with the LLM directly, so they don't see the hallucination problem.

It feels like lots of experiments have already been done with pulling NPC dialog from LLMs, but what about crafting worlds (most easily in text-based games, but potentially graphical too) and then reifiying them into the engine? Perhaps this could make more cohesive procedural worlds, since you can have the stories in the world's history drive the actual layout of the areas, scenes, and characters.