[Xenoamorphous]

How do RAG implementations usually get around the context size limitations in LLMs?

Since it usually deals with PDFs and other docs that can be quite big, do they take only the first N tokens? Are abstractive summarisation techniques used?

[svaha1728]

They split the document. Here’s an example of Markdown splitting. All this is far more an art than science at this point.

https://python.langchain.com/docs/modules/data_connection/do...