[solid_fuel]

> I would be very interested in the harness more than the actually generated story

This is a pretty common stance when it comes to LLM generated stuff, actually. The only original part of any LLM generated content is the prompt, everything else is just a derived artifact and doesn't really need to be treated like we would treat original, human-authored work.

This same principle is also why many projects reject LLM-generated PRs and such, too.

[8n4vidtmkvmk]

I frequently steer the AI mid convo though. So much so that I find it useless to share the original prompt. I don't know how best to capture that.

[solid_fuel]

Technically that is only adding to the prompt, which then becomes (your prompt + original response + your prompt 2), etc.

You could still capture it by recording only your prompts, the points in the conversation they were submitted at, and the starting parameters for the model. A replay would then produce the same results if your input was added at the same place in the conversation.

Granted I don't think the current tools do a great job of handling that.

[slopinthebag]

Actually the replay wouldn’t be the same since it’s stochastic, and the more back and fourth you have the more chances for randomness to seep in. One could imagine by the fifth prompt or w/e the responses could have drifted enough that the subsequent prompts don’t make any sense.

[solid_fuel]

It will be the same if you fix the original seed, the randomness comes from a PRNG so it is repeatable.

[petesergeant]

I wrote an article that touches on this a few days ago https://sgnt.ai/p/prompt-is-not-the-work/