[gbnwl]

Their comment reporting stats here:

“Provider: OpenAI (gpt-4o / o1)”

Uh so is it 4o or o1? Very different models. When you read this, how did you interpret this?

    “Suite: 11-task core suite (atomic coding tasks)”

- OK ill take your word for it

    “Configuration: autoroute_first=true, single_file_fast_path=false

Run Variant Token Delta (per call) Step Savings (vs Baseline) Task Success Baseline (2026-03-13) -18.62% — 11/11 Hardened A +8.07% — 11/11 Enhanced (2026-03-27) -6.73% +27.78% 11/11 Key Takeaways:

- What useful information do you glean from this vova_hn2? Perhaps Im just ignorant.

    “The ROI of Precision: While the "Enhanced" run used roughly 6.73% more tokens than the baseline per request, it required 27.78% fewer steps to reach a successful solution.”

So it actually takes MORE tokens but less “steps”? This could all use actual discussion feom the creator. A blog post or detailed comment. Instead we get this.

What sets me off is projects like this that throw random numbers and technical jargon at you because the user simply asked their LLM to do so. It gives the veneer of “oh it must be legitimate look at all the data” to people mentally stuck in 2024 not realizing anyone can generate junk and pass if off in a way that (used to be) convincing.