[slopinthebag]

Yeah this is the same conclusion I have. I primarily use AI for UI code, and guess what, it's all basically mechanical drudgery anyways. Put a div here, or put a Box here, apply some style rules, etc. This shit should have been automated decades ago yet for some reason we're still writing the same stuff with a different "twist" today.

Now if your career is built on writing out the same boilerplate code in its infinite slight variations every day, congrats, you've been automated. Thank god we can free up our intellects to focus on the actual hard problems, the ones that are somewhat cutting edge, the ones that actually push our field and humanity forward.

Literally every example of AI generated code (without significant human input) is just basic stuff that is wholly unimpressive. Oh wow, you had an AI generate a Next.js app? It's writing HTML for you? It made a generic SAAS? Guess I'll become a farmer now.

Or, wait, I'll continue to write my multithreaded real-time multiplayer network for a MMO, since the AI currently generates something that would get me fired 10 seconds ago if I tried to push it to production.

It's amazing how you introduce just the slightest difficulty or novelty to an AI and it just craps the bed. And then you go online and apparently we're gonna be replaced -6 months ago or something.

People need a reality check.

[throwaway894345]

I genuinely appreciated this comment—it made me chuckle. That said, I think there are better approaches to working with AI besides “here’s a big vague thing to work on, go write some code”. I think you have to iterate somewhat closely with the AI to write a doc describing exactly what you want the system to do and then scope out very narrow tickets and then have a separate agent do the TDD to actually produce the thing. The key insights here are (1) don’t let a code writing agent have too much scope—just a narrowly scoped ticket, (2) keep the coding agent’s context minimal, (3) don’t let the coding agent write much code without testing it. The agent should make very small changes at a time and then test that everything still works.

You will still need to QA stuff and review PRs, but I think AI done properly can genuinely make some tasks better.

[slopinthebag]

> don’t let a code writing agent have too much scope—just a narrowly scoped ticket

it's interesting cuz my intuition is to give the language model writing the files as much context as possible, which means all of the previous planning thread. but I also thought you should plan with a small model and implement with a large one, and the meta seems to be plan with an expensive one and delegate code output to smaller ones. so what do I know.

> The agent should make very small changes at a time and then test that everything still works.

yeah I think if it's treated like a codegen machine it's basically just outputting code as if you're using a dsl, except the dsl is natural language and the output is meant to be edited, no `// this is generated code, do not edit` headers

> I think AI done properly can genuinely make some tasks better

thank god I dont need to write html by hand anymore, what a pita

[throwaway894345]

Models seem to perform worse if you give them too much context. Even if you have a large context window, it seems like they’re only “smart” in the first few tens of thousands of tokens (including the system prompt, which is often huge). Also, it seems like they’re do better if you start a fresh agent off with a very narrow task and give them access to more context as necessary rather than shoving everything you have into their context window and wishing them well.

But I should also emphasize my limited experience and the rapid pace that this stuff is evolving.

[tisdadd]

I had it throwing in free advice on my code working as intended, but not a normal pattern. It was something like:"Bonus! This bug exists!" And I had to tell it stop doing that. Or, for generated SQL renaming to keep deeply linked table columns human readable via comments it was - "You can't have a comment of this style here." It works perfectly so yes, yes I can.

I can certainly get it to do things that are reasonably common it seems like.

As for the article itself, I can agree with much of it.

[habinero]

I had AI fuck up writing a scraper[0]. A scraper. It hit a snag with cookies and spiraled into a tizzy. I liked the part where it assured me it could resume from the point of failure, while starting over for the 10th time because it had written no such code lol

[0] For those with AI scraping PTSD, it was a government site with public domain info and I know how to scrape politely