[bad_haircut72]

Anthropic would be better off letting the community do this. Their harness sucks. Great scientists but not the best app developers. I suspect they just dont want to relinquish control of anything because they think the world cant be trusted with AI, we can only be trusted to pay them.

[bob1029]

Custom agents using the low level completion APIs tend to outperform these generic tools, especially when you are working with complex problems.

It's hard to beat domain specific code. I can avoid massive prompts and token bloat if my execution environment, tools and error feedback provide effectively the same constraints.

If I had to pick only one tool for a generic agent to use, it would definitely be ExecuteSqlQuery (or a superset like ExecuteShell). If you gave me an agent framework and this is all it could do, I'd probably be ok for quite a while. SQL can absorb the domain specific concerns quite well. Consider that tool definitions also consume tokens.

[Someone1234]

Could you go into more details about why their "harness sucks?" This feels like a shared conclusion, but I've used several and theirs is better than many.

[steve_adams_86]

I generally agree that the harness isn't good, but it works and gets the job done and that seems to be the singular goal of the top 4 or 5 companies building them.

We saw what Claude Code looks like inside, and it's objectively bad-to-mediocre work, but the takeaway seemed to be 'yeah but it works and they've got crazy revenue'.

That's where we're at. The harness is kind of buggy. The LLM still wanders and cycles in it sometimes. It's a monolithic LLM herding machine. The underlying model is awesome and the harness works well enough to make it super effective.

We can do so much better but we could also do worse. It's a turbulent time. I'm not super pleased with it all the time, but it's hard to criticize in many ways. They're doing a good job under the circumstances.

I see it kind of like they're at war. If they slow down to perfect anything, they will begin to lose battles, and they will lose ground. It's a highly contentious space. The harness isn't as good as it could be under better circumstances, but it's arguably a necessary trade off Anthropic needs to make.

[theshrike79]

> We saw what Claude Code looks like inside, and it's objectively bad-to-mediocre work

Based on this, are there any open source harnesses that have objectively good-to-excellent work in their code?

[jeena]

I've been using OpenCode until yesterday (with some plugin to let me use their model until they implemented what it seems very sophisticated detection to reject you).

It just has a sane workflow it's easy to use, doesn't bother you with 1000 questions if you allow this or that to run and generally it feels like the model is dumber and makes more mistakes since yesterday since I have to use claude code.

[pmorelli]

pi.dev

very minimal, extensible.

[steve_adams_86]

Agreed, this is the best I've seen so far.

[torhowawy7]

> We saw what Claude Code looks like inside, and it's objectively bad-to-mediocre

Do you have an example to contrast by what measure is good besides your word?

[paulddraper]

?

Anthropic made the most popular harness for developers.

Anthropic made the most popular desktop tool for AI automation.

[circularfoyers]

Not sure popularity necessarily suggests it's good, but possibly just what people have most heard of or is easiest to setup with. This is going to be even more true now that Claude subscriptions are going to be essentially vendor locked.

[mwigdahl]

Given how far ahead OpenAI was in mindshare, monthly users, and revenue over Anthropic when Claude Code came out, I think we can conclude there's at least some substance behind claims of Claude Code's better product quality.

[paulddraper]

It definitely suggests it.

[bredren]

Yes. Gemeni's web interface was atrocious even when the model was the best frontier.

And codex still uses phrases and syntax in prose ostensibly for the user as though they forgot people are actively reading this stuff.

Product is unquestionably where Anthropic excels. It is what carried it through periods where its thinking model lagged.

[cultofmetatron]

I've been greatly enjoying jetbrain's air IDE for some tasks. it uses claude behind the scenes.

[yodon]

Wow - as a JetBrains customer I've been waiting for JetBrains to show what a good Agentic IDE should look like.

Excited to try it and surprised to find out about it in a random HN comment.

EDIT ADDED: bummer - looks like this only works with API keys, not Claude Pro or Max or similar for Codex

[cultofmetatron]

its still new. I'm hoping it will improve overtime.