| I'm exhausted seeing this same mistake happen over and over and over.
Why nobody tries to write a spec from existing code is just crazy to me. The git test suite is a behavioral spec. But at the same time it is not, why? It's archaeological site... I'd say. You can dig, and dig, and dig, and find some truth, but also a lot of historical decisions that doesn't matter today. Here's the flow I suggest: - first, reverse-engineer the behavioral intent from tests/docs/code - build a taxonomy of what git promises - group that into small "conformance slices" - hand those slices to agents/humans/whatever - start writing Rust... Or Visual Basic... At this point it doesn't matter almost Without this critical layer, agents are optimizing for "make this test green" instead of "preserve this semantic contract". And this is exactly where the funny stuff happens... shelling to real git, hardcoding expectations, implementing sha256 metadata but not the behavior, etc. Why are we doing stupid things and winning stupid prizes? I have to admit they are impressive, but I STRONGLY believe if we did this in two passes you would have not $15k check but maybe closer to $1-5k. Who should I talk to, to make this happen? Making the first pass is almost deterministic. LLMs help. The only problem is making sure that YOU understand the spec, and this will be a bottleneck for a while (i.e. can't outsource understanding). I can guide what exactly needs to happen. I already validated this idea on my own project and it worked: 10k LoC -> 250 acceptance criteria. And you can find it on my GitHub, I even described steps. If you're too lazy here it is: You need a SCIP graph. Agent goes over it and builds ledger, one symbol at a time, looks at what kind of problem this symbol solves. This leads you to v0 specs, then you can figure out taxonomy, then you try to fit all those thousands of specs into taxonomy / groups that make sense. And that would be v1 spec but you might want to refine it to v2/v3. And then the only thing would be left is figure out what kind of tests those specs should have (e2e/unit/integration/api/whatever). This is tricky part but doable. I'm thinking for git you wanna do e2e specs. Yes, that's a lot of e2e but the purpose is that we build same expectations for git, and then we replace git with grit and spec should still stay green, right? Hope that makes sense. |