[brotchie]

Played around with fine tuning, but ended up just experimenting with RAG.

One thing I haven’t worked out yet is the agent reliably understanding if it should do a “point retrieval query” or an “aggregation query.”

Point query: embed and do vector lookup with some max N and distance threshold. For example: “Who prepared my 2023 taxes?”

Aggregation query: select a larger collection of documents (1k+) that possibly don’t fit in the context window and reason over the collection. “Summarize all of the correspondence I’ve had with tax preparation agencies over the past 10 years”

The latter may be solved with just a larger max N and larger context window.

Almost like it’s a search lookup vs. a map reduce.

[jdthedisciple]

Interesting, been waiting for some free time to do this myself.

Mind sharing how you set up your RAG pipeline and which (presumabely FOSS) components you incorporated?