[karaterobot]

This story is an opportunity to ask a question I've been wondering about: Is it possible for the same generation of a LLM to produce stories that are significantly better than each other, given the same prompt multiple times? I know you can improve a response by tweaking the prompt. I know that randomness plays a part in the generation process. But, if (let's say) ChatGPT version x.xxx produces a C+ short story the first time, will that same version ever produce an A+ short story if I just keeping asking it the same thing over and over?

[chaps]

Yes, it can. It's not as simple as giving it the same prompt, but literally feeding prompts into its response like, "this is really corny sounding, nobody actually talks like that. Please write it to be more authentic. Please also write it as if this response was the first, sloppy draft from a world-class, multi-disciplinary author, and rewrite it to be their fourth copy that's been edited by an equally talented editor. Also add more dialogue."

Then feed the story it wrote into a new chat and tell it to rewrite it from scratch, but include very specific details that it should change to convey the scene. "have it star out with the main character staring at a childhood photo".

[LeifCarrotson]

To some degree, yes. You can ask it to do some introspection on a previous answer. It's basically an advanced form of prompt tuning, using the frozen model to fix its own outputs.

Have it write a short story, then ask it to identify grammatical problems in the short story, then to find run-on sentences, etc. It sometimes feels like "Check for errors" ought to be a follow up to every prompt...

[slowmovintarget]

Statistically, yes. Infinite monkeys being as productive as they are, you'd eventually get Shakespeare out by accident.

[karaterobot]

Are the number of responses by ChatGPT to a given prompt infinite? I suppose that even monkeys at a typewriter can't produce an infinite number of different books, even if that number is really, really large (hello, Mr. Borges). But, I assumed (without a lot of subject matter knowledge) that the number of possible responses was actually much less than just randomly typing keys, because it's picking words from a frequency table. Again, I have no idea what I'm talking about.

[slowmovintarget]

There's randomization, but the random aspect lands you on a root node and branching structure for filling the text. Given enough tries (you just keep running the same prompt with different starter "seed" values), you'll get something out that seems like quality.

What's doubtful is whether the people attempting this are good enough writers themselves, or educated enough readers themselves, to recognize it when it falls into their hands. I suspect the Venn diagram for the two groups of people doesn't overlap all that much.

[shadowgovt]

Not even by accident, given that Shakespeare's in the training dataset.

[snerbles]

Much of Shakespeare's content would be considered harmful by ChatGPT's filters.