Looking at the code, it looks like you're maintaining context by re-serializing all previous responses as "ChatGPT" and "User." Out of curiosity, do you know that this is how chatgpt is done, or your take on it? I assumed from this tweet, there was more of a "special sauce" for maintaining chat context:
> GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series:
code-davinci-002 is a base model, so good for pure code-completion tasks
text-davinci-002 is an InstructGPT model based on code-davinci-002
text-davinci-003 is an improvement on text-davinci-002
You're totally true. But it turned out, that proper prompting (like packing dialog context into the prompt) worked great.
I spent many hours with orig ChatGPT and with this recreated version. The main difference I have found is that the recreated ChatGPT is more inclined to ask questions on questions (maybe can be fixed with more prompt engineering). I didn't find any major differences in the quality or usefulness of the answers.
Ok, so OpenAI says that ChatGPT is GPT-3.5, but with extensive fine-tuning applied, based on a complex multi-stage feedback process with human evaluators.
But at the same time, you can apparently just take the "raw" GPT-3.5, give it a prompt to behave like an assistant and get comparable results?
1. You’re right, it’s not ChatGPT exactly. ChatGPT = GPT-3.5 + finetuning on conversational data + alignment (RLHF). But I found out, that GPT-3.5 + proper prompting gives comparable results and works really good.
2. The context of the conversation is preserved by adding whole (or some parts of) dialog to the prompt.
Given it is not ChatGPT and it is especially missing RLHF, I would suggest amending the repo name and description. Maybe “ChatGPT-like” would be more appropriate here.
https://twitter.com/OpenAI/status/1615160228366147585?s=20&t...