[IngoBlechschmid]

Gwern shared an idea how to exploit the strength of current-generation LLMs, despite their weaknesses, for "create your own adventure"-style fiction. https://gwern.net/cyoa Having people vote on AI-generated potential continuations should yield better results and cut costs at the same time.

From the title I thought this was an implementation of Gwern's idea, but it's not.

[ianbicking]

There have been experiments in wiki-style cyoa generation (letting the public create options instead of an LLM), but they suffer the same problem as LLM-generated stories: aimless wandering and lack of consistency.

(As I think about it, an LLM generation should be thought of as a many-author situation, as each generation comes in cold)

Stories need pacing, which exists over many passages, not just at the choice level. And then the passages should all be based on a single underlying world. Both of these fall apart quickly without a guiding author.

I think this is resolvable with LLMs and appropriate prompting, but the naive approach seems cool only until you actually play out a few stories

[th0ma5]

How would this be resolvable ?

[ianbicking]

To ground the story in a consistent world: build up secret world lore behind the scenes, and provide that to the LLM as it makes choices. As the story is expanded the lore will expand. But this gets complicated as time progresses, as things that are "true" later in the story aren't earlier in the story (when some events haven't passed), and choices _should_ modify what happens so there isn't "one" truth. An LLM can handle some of that complexity natively, just by how the story is structured and allowing it to put conditionals into the lore. ("If the Snow Queen hasn't been defeated, she will sense when the hero is inside the castle")

You'll also need to keep track of context, so you can place the current passage in the lore. I doubt it will be self-evident. Some of this can probably be shown to the user as well as "metadata" of sorts (like a location name).

You could also pre-build the lore entirely, and then as the LLM expands out choices instruct it to stay within that lore. This is probably easier, but it'll seem less exciting to see it build out those passages. You'll want to include major choices in the lore itself so it can build out some of those conditional parts of the story.

The rhythm is much harder. Pre-building the entire lore will probably make this much easier. I'd probably throw a few queries at Deep Research to get ideas and terms for story structure that I'd want to apply. There's two major issues I see in incrementally created LLM generated stories: never getting to the good parts, and accelerating too fast.

The first is probably worse and more common: the LLM will build up anticipation, but it doesn't know the conclusion and will keep putting off any conclusion. It reminds me of Lost (the TV show)... you can tell the writers didn't have a plan, and only made things worse as they introduced more distractions that couldn't be made into a cohesive reveal. If you already have an outline then it should go forward. Another option is to have the LLM produce conclusions at the same time as it produces choices, and give it criteria for what a good conclusion is. These don't have to be full passages, but will be notes you pass on as the next passage is created.

Going too fast can happen too. If you just ask ChatGPT to write a CYOA story it'll probably stop abruptly when it sees the opportunity to just finish up. I don't have a clear idea here, but I'd try to think about splitting the story into acts, giving those acts clear purposes (as story structure). Then I'd aim for a certain length to each act, but let the LLM decide the exact moment one act moves to the next.

[mmooss]

In other words, create something like an rpg dungeon master does, and feed it to an LLM to execute?

[lawlessone]

>Having people vote on AI-generated potential continuations should

So the story never ends?