[taspeotis]

The harness is super important, what tools are available and the system prompts vary from harness to harness.

Anthropic seems to have a modest lead on their harness and models, so it’s a best-of-both-worlds scenario.

> I'm not sure what Microsoft is doing behind the scenes

It’s probably the exact same model, but the tools and the prompts around it are worse, so you get worse results.

[irthomasthomas]

Claude in Claude code has been shown to perform persistently worse in evals than claude + a minimal harness.

[kilburn]

The harness was absolutely not an issue in my case.

The new pricing model where I got banned from using Opus entirely and half a day of work (with weaker models) consumed the 10$ plan was.

I'm now using a Claude Max subscription and I can get close to the daily limits but I'm fairly happy with the overall plan consumption.

[Vinnl]

So if you use Claude via Copilot in Zed... You use Zed's harness, I think? What does Copilot do, at that point?

[acpdev]

I believe you are using https://github.com/github/copilot-cli or potentially this https://github.com/github/copilot-language-server-release#ag... via the Agent Client Protocol https://github.com/agentclientprotocol/agent-client-protocol which means you are indeed using Copilot's harness

ACP is just a standard that bridges harnesses easily into IDEs, Text Editors, or whatever consumes it (I wrote a TUI that consumes them)

The registry for all the agents (tool harnesses) is here https://github.com/agentclientprotocol/registry if you ever are curious to what Zed or IntelliJ are really hooking into

[Vinnl]

Ah OK, so the ACP connector ensures tool calls work with Zed, and communicates the available tools and their results to the harness, and then the harness mainly provides a system prompt and the API calls?

[acpdev]

Harness does a lot of things potentially, but this depends on the harness.

1. Manages what is loaded in the context (I.e. sent to the model), this can be about compaction, summarization, removing things it judges no longer relevant, proving memory 2. Handling retries and how to resend data , lot of model specific behavior here 3. Wires up tools like web search or lsp servers 4. Skills and how to use them 5. Plugins/sdks 6. Management of rules (AGENTS.md)

About everything interesting really, a ton of work goes into this and getting it right is surprisingly hard

[Vinnl]

Thanks! Although these, possibly with the exception of #2, sound like things that Zed handles, rather than the harness?

[acpdev]

No, not especially, the client (Zed in this case) can add some things (like @ing a file or class) and hint stuff but in all seriousness the harness is doing the things I’ve mentioned

You can see it from the different transcripts from the different tooling harnesses (´agents’ in agent client protocol speak) you can also just read the source for pi or zed or anything here don’t take my word for it.

You will see different web search from pi and codex, different tool calls, different context compaction strategies, etc.

The client has some control over this stuff but very little, like zed can pass in mcp servers to use or files to add to the context but the tooling harnesses can ignore those requests

[pantulis]

It’s providing the inference of Anthropic models