[ilaksh]

5 pages fits in the context window. How exactly do you plan on expanding the limit? Without explanation we have to assume you haven't completely solved your core technical challenges.

In my testing the biggest challenges with using for example OpenAI embeddings with cosine similarity or something are A) figuring out the section breaks or right chunk size so that information stays in context and B) retrieving enough chunks to get the correct hit for a query without having too much extraneous information that confuses it.

I think that it's hard to make a parser that most optimally slices up arbitrary documents.

Since you have some larger documents preloaded I assume for those you have the embeddings search. But for user uploads you are skipping that now and just feeding all of the text extracted from the PDF into the prompt along with the query.

[nathanwh]

This explains why "What if I move out early?" for the sample document doesn't mention any of the information in the lease break section, which is definitely the most important section for moving out early. Whatever space they're projecting the question into doesn't capture that "lease break" and "moving out early" are synonyms.

[ilaksh]

It may only be retrieving the top N results with most similar embedding. If that answer is in the 3rd most similar chunk and it only fed 2 along with the query in the prompt, then GPT never got the information relevant to the question.

[pablo24602]

From the website, it seems as though they are retrieving five chunks. Also looks like they split documents by paragraph sections, unless the paragraphs are small enough- then they put a couple of them together.

[btbuildem]

Same, the is lies in the details. You basically need a good semantic search in front of GPT to feed it the best context given the question.

[nico]

Any code or pseudo-code you could share that does something like that?