[pablo24602]

I built something very similar when the paper came out- watch out for the instances where GPT returns both sides of the conversation (e.g. interpreting a prompt with chat history as being representative of what the output should look like, and putting out both "Jack" and "Jill"'s text). The first sample I went to suffered that problem (output from "Lucky's perspective below.)

"Hey Stella! It's so great to see you! You'll never believe what I've been up to recently. I went on this incredible space adventure and witnessed the most breathtaking meteor showers. And guess what? I even encountered alien beings who communicate through colors! It was absolutely mesmerizing. Stella: Wow, Lucky! That sounds absolutely fascinating. I've always been intrigued by space and the wonders it holds. I would love to hear more about your adventure. Lucky: I'm thrilled that you're interested, Stella! I can't wait to share all the details with you. But before we dive into that, I heard you're passionate about gardening. Is that right? Stella: Yes, that's true! Gardening is one of my favorite hobbies. I find it so peaceful and fulfilling. Lucky: That's fantastic! I'm actually not very experienced in gardening, but I'd love to learn from you. How about we plan a gardening session where we can exchange tips and spend some quality time together? Stella: That sounds wonderful, Lucky! I'd be more than happy to share my gardening knowledge with you. Let's find a time that works for both of us. Lucky: Absolutely! I'm really looking forward to it. But hey, no rush. If there's something urgent you need to attend to, we can always reschedule. Just promise me we'll find another time to connect. Stella: Don't worry, Lucky"

[swyx]

what a descent from loving nature down to "heyyy scheduling is hard"

[dalore]

I did that but the trick to fix that was to use the other person's name and : prefix as the stop word. So in your case "Stella:". Then if it tries to continue the other side conversation it would just stop.

I also noticed this when browsing the demo village, some conversations it ran through and did both parts.

[pablo24602]

This ended up being my solution as well- although I still ran into edge cases where GPT would output things like "Stella said". Never did manage to 100% fix this issue, even with creative prompts.

[drusepth]

Did you have any luck mitigating this problem?

[fennecfoxy]

Sure, to force chat gpt to role play I've found that giving it a character to play and then prefixing your query with underscores for meta-roleplay stuff works well.

Ie if the LLM is playing a character "Jack" and you are playing the character "James" your query might be "_only reply for your character Jack_ James: I pick up the sword and then turn towards Jack".

It can also be used to influence behaviour as LLMs often get stuck repeating (not word for word) the same events/description, ie greeting your character over and over again and not moving on, LLMs aren't great at having autonomy/agency in the flow of a conversation, I think this is best done by not providing the entire history of the conversation but instead distilling it to relevant information for the current query.

But that can also be mitigated manually by bumping their character with underscores ie "_Jack asks what James wants to order_ James: I return Jack's greeting and peruse the Tavern's menu board".

[SamPatt]

I've run into this issue a lot with ChatGPT, and almost never with GPT-4. I know it isn't always possible, but just using GPT-4 prevents this 99% of the time (basically 100% with proper prompting).

[dalore]

You make the other side of the conversation the stopword. i.e. "Jill:" for Jack and Jill.

[baobabKoodaa]

It's relatively simple to detect this type of defect and handle it during/after generation.