[ramoz]

Very bearish on these frameworks and abstractions.

Yes, obviously useful for prototyping and creating hype articles & tweets with fun examples. However any engineer is capable of doing their own rag with the same effort (minimal data extraction using the ancient pdf/scrape tools that are still open sota, or use cloud ocr for best —-> brute force chunking —-> embed —-> load in Ann with complementary metadata store)

Anyone doing prod needs to know the intricacies and make advanced engineering decisions. There’s a reason there aren’t similar end-to-end abstractions over creating Lucene (solr/elastic) indexes. Hmm, why not after many decades? …

In reality, the RAG tech is not entirely novel— it’s etl. Which in reality, complex etl is often a serious data curation effort. LLMs are the closest thing to enabling better data curation, and as long as you aren’t competing with open ai (arguably any commercial system is) then you can use chatgpt to create your chunks.

Beyond this embedding strategies are nice to abstract but the best approach to embeddings still remains to create your own and figure out contextual integration on your own. Creating your own can also just be fine-tuning. Inference is often an ensemble depending on your use case.

[ddematheu]

I don't disagree with all your points. That said, what we have built has proven useful for us as we have built pipelines for customers and think it might be useful for others.

Probably the main point I disagree with you is that RAG is just ETL. If that was the case, all of the AI apps people are building would be AMAZING because we solved the ETL problem years ago. Yet, app after app being released have issues like hallucinations and incorrect data. IMO the second you insert a non-deterministic entity in the middle of an ETL pipeline, it is no longer just ETL. To try to add value here, our focus has been on adding capabilities to the framework around data synchronization (which is actually more of a vector management problem), contextualization of data through metadata and retrieval (this part being were we have spent the least time to date, but are currently spending the most)

[hobs]

The problem is that business people have been just rubbing together libraries for decades making money (though maybe not the MOST) and will see these frameworks as a "simple" way to accelerate development, when in fact most of them are opinionated (and bad) ETL. Just put langchain together and it's done, right?

I went through building a RAG pipeline for a company and brought up at each stage how there's been no tuning, no efficacy testing for different scenarios, no testing of different chunking strategies, just the most basic work done and they released it almost immediately. Surprisingly to not much fan fare.

It doesn't really matter