Hacker News new | ask | show | jobs
by bigcat12345678 804 days ago
El5 me what is rerank model? Why 4k context window size is considered large?
1 comments

Imagine you have 100 documents in a database and you "query" the documents and return 20 candidate results.

Similarity gave you 20 results but Re-ranking sorted those results further providing relevance.

That 4K is per document.

Edit: With sorted relevance, you can drop the lower scoring documents according to the model's confidence that the information in the subset is adequate to answer the query.

20 results, each have 4k snippet, feed to this ranking model, ranking model produces a score based on query

Is this a correct understanding?