[r3trohack3r]

Curious how this performs relative to ChatGPT? IIUC davinci-03 isn't the same model as ChatGPT, I've struggled to get satisfactory responses to prompts through the API compared to the chat interface, am basically in a holding pattern right now waiting for the new models to release on the API.

[parhamn]

It works fine (using GPT3.5/text-davinci-003) with a conversation look-back (+auto-summarization of the lookback when it is too long). Many people claim its more factual too.

[freeqaz]

How do you enable auto-summarization? Is that an option you can toggle?

[ShamelessC]

I assume you just ask GPT3 in plain English. Crazy times.

[kaveenk]

Hey! This is an option in the settings. You can type !gp to view settings and !gs to change them. For your case, !gs summarize_conversations True !gs summarize_threshold (token amount to summarize at)

[ilaksh]

Try temperature 0. And make sure you include all of the conversational context and instructions with examples in each API call.

It's definitely not the same model but similar in a lot of ways as far as I can tell. You could also try code-davinci-002 if that's what you are doing.

[dbish]

You can’t include “all” the context due to the prompt token length limitations. You have to use techniques for surfacing the right context that would fit in the limited context window and sending that along. There are techniques that keep a running and continuously compressed summary, and techniques that use embeddings to hone in on relevant chunks of past conversation and context and send that along based on some ranking and the size limitations.

[ilaksh]

Right I am familiar with that stuff I was just trying to answer for the case that useful context fit within the max_tokens. Your startup looks really interesting. Kind of similar to some of the plans I have for aidev.codes. It seems for code you would need to just send a whole file usually.. maybe a whole function? Because if you break it up too much its doesn't necessarily "compute". I dunno. Are you using code-davinci-002 from OpenAI, and if so, have you managed to get your rate limit increased? They seem to ignore my support requests. text-davinci-003 is pretty good too. One experiment I have done for getting context is just to say "given this user update request, and this directory listing, which files would you need to inspect" then the next prompt includes those files and asks to modify them to complete the request, with a specific format for identifying filenames in the output.

[kaveenk]

Exactly! This is also what I’ll be implementing within the next few days. Embedding conversation history and dynamically doing semantic search for relevant conversation snippets and building a prompt based on that to send to GPT3

[ilaksh]

Nice. How do you determine where the relevant parts start and stop? Do the embeddings work per paragraph?

[r3trohack3r]

Just wanted to say thank you - you helped me make a major breakthrough in getting the API to behave the way I expect.