[leobg]

I haven‘t been active and don‘t know if they have changed this. But when I was using GPT-3, their semantic search endpoint didn‘t actually use GPT-3 for ranking. It used some other, „cheaper“ algo as a pre-filter (tf-idf, I believe). And only the those results that this „cheaper“ algo returned on top made it into the evaluation by the actual GPT-3 model you had chosen.

(There was a setting that controlled how much of your data was actually ranked by GPT-3 - top_k or something. But to chose anything even approaching a sizable percentage of a large corpus such as the one you are dealing with would be prohibitively expensive. You‘d be looking at several Dollars per search.)

Anyway. This is also why switching the model from Ada to Curie to Davinci on the semantic search endpoint has (or had?) very little effect. It still often missed highly relevant snippets from the source material.

When they came out with their „answers endpoint“, it really was just a semantic search piped into a completion prompt.

I actually found it better to keep my own implementation of this pipeline, because then you at least get to control how the results of the semantic search are prompted to the generation model.