[beyang]

This was a great listen that really resonated with the approach we've taken in building our code AI tool (Sourcegraph's Cody: https://cody.dev). We found that the biggest levers we had to drive improvement in the accuracy of Q&A and code generation quality was fetching relevant snippets of code and docs into the context window. So we pivoted from investigating more expensive, long-iteration-cycle updates at the model training level and invested more into code-specific information retrieval mechanisms (it helps that we spent the past 10 years building a code search and code intelligence engine). We've found RAG to be cheap, fast, and directly impactful compared to model-layer improvements (but as Jerry points out, still a very hard engineering problem).

Anyway, Jerry and LlamaIndex has been a huge source of learning and inspiration—please keep tweeting and publishing, Jerry!