[jrflo]

Interesting for sure, but I think I need to be convinced this is necessary. How many tokens does Grep use on the average response anyways? Does this reduce input token usage from 100k to 2k per query or from 1k to 20? How does this effect output code quality?

[Bibabomas]

There's a "semble savings" command you can run to see token savings. It varies heavily based on the repo: in general, the larger the project the larger the savings. Output code quality is something we're still trying to measure, but it's much harder than measuring retrieval quality.

[jrflo]

Gotcha. Might be interesting to show some plots of average/median/max input token savings per query as a function of repo size if you're going to do some more comparison testing. The efficiency gains are compelling, but I'd want to see the magnitude of gains as well to get a full picture. Regardless, cool project and I'll check it out for myself