| I spent 10 years at Google, where part of my work involved analyzing and improving cost, performance, and latency. While I was exhausting my token limits, I searched for tools that claimed to be saving tokens. but what they don't say is the full agent $ cost saving. That is why I created LemonCrow: to optimize agent context end to end rather than focusing on a single token category. Result: on average 30% cost saving and 25% faster. No model routing trick. Same tasks, Same quality, Same model, just cheaper and faster. Agent cost includes more than output tokens: fresh input, cache writes, cache reads, and repeated context across turns all contribute to the bill. Every input/output/tools/context reread on every turn making cost O(n^2). Optimizing one category may not reduce total cost by the same percentage. I ran matched benchmarks against a clean Claude Code baseline and few other tools. Caveman — 300 runs per arm Output tokens: −44.1%
Total cost: $26.0058 → $26.0204
Cost difference: +0.06% The answers were shorter, but the final bill was essentially unchanged. CodeGraph — 35 runs per arm Total token volume: −86.5%
Total cost: $19.1064 → $15.9871
Cost difference: −16.3% This was a real saving, but much smaller than the token reduction claims. LemonCrow on the same tasks Total cost: $19.1064 → $6.2869
Cost difference: −67.1%
Total token volume: approximately −91.7% Indexing quality is the main driver for cost. if Index can't give LLM in one turn what it is looking for it will have to rephrase and ask again or start using grep instead.
I also measured retrieval quality across 7,213 examples from 14 repositories: LemonCrow: 0.676
cocoindex-code: 0.557
codebase-memory-mcp: 0.502
ripgrep: 0.376
CodeGraph: 0.296 MRR measures retrieval ranking, not final answer quality. LemonCrow was also tested on SWE-bench and Terminal-Bench. On SWE-bench Verified, it resolved 232/250 runs versus 202/250 for baseline while costing 29.5% less. Terminal bench 2.1 corectness 0.79-0.80. Lemoncrow able to maintain same or higher quality at lower ~30% lower cost and ~25% faster. Since I maintain LemonCrow, this is not an independent review. I published the raw data and asked the Caveman and CodeGraph maintainers to review the configurations and flag corrections. https://github.com/lemoncrow-lab/lemoncrow/blob/main/BENCHMA... |