[slopinthebag]

> don’t let a code writing agent have too much scope—just a narrowly scoped ticket

it's interesting cuz my intuition is to give the language model writing the files as much context as possible, which means all of the previous planning thread. but I also thought you should plan with a small model and implement with a large one, and the meta seems to be plan with an expensive one and delegate code output to smaller ones. so what do I know.

> The agent should make very small changes at a time and then test that everything still works.

yeah I think if it's treated like a codegen machine it's basically just outputting code as if you're using a dsl, except the dsl is natural language and the output is meant to be edited, no `// this is generated code, do not edit` headers

> I think AI done properly can genuinely make some tasks better

thank god I dont need to write html by hand anymore, what a pita

[throwaway894345]

Models seem to perform worse if you give them too much context. Even if you have a large context window, it seems like they’re only “smart” in the first few tens of thousands of tokens (including the system prompt, which is often huge). Also, it seems like they’re do better if you start a fresh agent off with a very narrow task and give them access to more context as necessary rather than shoving everything you have into their context window and wishing them well.

But I should also emphasize my limited experience and the rapid pace that this stuff is evolving.