|
|
|
|
|
|
by MysticBirdie
157 days ago
|
|
|
(Edit: Updated results + Windsurf coding demo showing same 40%→100% pattern in production AI workflows. Domain grounding > model scale.) Compositional Chaining Benchmark — by Chain Type Chain Type Triad: 81.8% Raw: 72.7% ∆ (Triad–Raw): +9.1pp FAULT_INJECTION Triad: 100% Raw: 71.4% ∆: +28.6pp CROSS_CHARACTER Triad: 50%* Raw: 100%
*Triad cross-char losses were 100% rate limit timeouts (ERRORs), not model mistakes. Depth Drift — Accuracy vs. Hops
Step | Triad | Raw
1⃣ 75% | 87.5%
2⃣ 75% | 75%
3⃣ 83.3% | 66.7% ← Raw degrades
4⃣ 100% | 100%
5⃣ 100% | 100% Raw drops 21pp between steps 1–3, while Triad improves. Fault Injection — The Key Result
Injected Error | Triad | Raw
Hadrian’s Wall as existing | PASS | FAIL
Julius Caesar as living | PASS | PASS
Paper money replacing coins | PASS | FAIL Triad: 3/3. Raw: 1/3. In short: Triad consistently maintains compositional reasoning under stress and even strengthens across depth |
|
|