[Alexanfa]

Hey,

What you can do is to use caching and embeddings. Embeddings are like X,Y coordinates (except with OpenAI it's 1500 dimensions not just two). You use embeddings on any type of text. Here you would embed user prompts and store answers. And what this means is, you would be able to get X,Y for "best restaurants close to the museum", "best places to eat" etc.. If a new user comes along and mentions food and museum, it's likely to rank highest for "best restaurants close to the museum". I haven't done this and idk if possible, but you could try to then check if this is "close enough". If it is, then you can probably return the same generated answer for this user. You can also embed answers and see if that has better result.

Process: 1. When user asks a question, you embed it and search with it. 2. If similar prompts exist; return the answer from the most "relevant" prompt. 3. If no similar prompt exists; generate a new answer, and store it to be associated with the prompt.

This might decrease quality, but maybe in "enough" cases, it does just fine.

I'm developing tzap.io that uses some of these techniques, join our discord if you want to get in touch: https://discord.gg/88xDVYbPVB