[rbren]

OpenDevin maintainer here. This is a reasonable take.

I have found it immensely useful for a handful of one-off tasks, but it's not yet a mission-critical part of my workflow (the way e.g. Copilot is).

Core model improvements (better, faster, cheaper) will definitely be a tailwind for us. But there are also many things we can do in the abstraction layer _above_ the LLM to drive these things forward. And there's also a lot we can do from a UX perspective (e.g. IDE integrations, better human-in-the-loop experiences, etc)

So even if models never get better (doubtful!) I'd continue to watch this space--it's getting better every day.

[anotherpaulg]

As a comparison, I use aider every day to develop aider.

Aider wrote 61% of the new code in its last release. It’s been averaging about 50% since the new Sonnet came out.

Data and graphs about aider’s contribution to its own code base:

https://aider.chat/HISTORY.html

[dartos]

It’d be really great to see a video or cast of you using aider to work on aider.

I can’t get anything useful out of these AI tools for my tasks and I’d really like to see what someone who can does.

I’d like to know if it’s me or my tasks that aren’t working for the llm.

[Arubis]

Can I ask what language/stack you’re using for your project? More specifically, is it in Python? I’ve had mediocre (though at least partly usable) results on JavaScript repos, and relatively poor ones on anything less popular.

[wcallahan]

Aider is written in Python (they have a great Discord community, btw). My experience matches yours: for Python, aider/Sonnet seems to do much better than for Javascript so far. I strongly recommend aider despite LLM limitations at the moment for anyone interested in this space.

It's also very sensitive, unsurprisingly, to development documentation that is moving quickly, e.g., most AI APIs right now. A lot of manual intervention is still required here because of out-of-date references to imports, etc.

[Lerc]

How heavy are the API costs for that?

For a project like yours I guess you should be given free credits. I hope that happens, but so far nobody has even given Karpathy a good standalone mic.

[harisec]

If you use DeepSeek Coder V2 0724 (that is #2 after Claude 3.5 Sonnet on the Aider leaderboard), the costs are very, very small. https://aider.chat/2024/07/25/new-models.html

[anotherpaulg]

Not much. I spent $25 on Anthropic in July.

[ode]

Using sonnet?

[wcallahan]

I'm an active aider user, I spent ~$120 last month on a combo of Sonnet and Opus. It was much more expensive, as you probably know, with Opus. Now it's rather reasonably priced and more sustainable, IMO.

[harisec]

aider is great, i also use it almost daily. thanks for writing it Paul!