Nice article :) Ive got some maybe unrelated questions about the feedback loop of the strong typing in code generation.
Ive experimented with automated code generating agent systems on localhost multiple times. Ive got some ok results when it was a project spanning over multiple files without any dependencies. But when I tried to cross a milestone of creating a web apps with lets say flask and some db, ive hit a wall. The problem were the dependencies and api hallucinations and the version/api inconsistencies of dependencies in genereated code. The system had a feedback loop with self-written tests, but even the output of the tests fell into neverending loop. Maybe the problem was python? Maybe annotations would help, maybe should use go?
great question. i am no expert on the topic, and my weekend project ended up being pretty small (a few hundred LoCs in total). I was able to get the core logic "work function" to work, but all of the stuff around it, I wrote by hand.
Ive experimented with automated code generating agent systems on localhost multiple times. Ive got some ok results when it was a project spanning over multiple files without any dependencies. But when I tried to cross a milestone of creating a web apps with lets say flask and some db, ive hit a wall. The problem were the dependencies and api hallucinations and the version/api inconsistencies of dependencies in genereated code. The system had a feedback loop with self-written tests, but even the output of the tests fell into neverending loop. Maybe the problem was python? Maybe annotations would help, maybe should use go?