[kbrisso]

I built this for local RAG https://github.com/kbrisso/byte-vision it uses llama.cpp and Elasticsearch. On a laptop with 8 GB GPU it can handle a 30K token size and summarize a fairly large PDF.

[busssard]

elasticsearch is the true limitation of rag systems...

[kbrisso]

The vector search works great once you figure it out. I wanted to focus on writing the application and not have to rewrite a document store.