Hacker News new | ask | show | jobs
by istrice 10 days ago
The "yes, with caveats" thing is boilerplate for both Codex and Claude since this current generation.

It's actually a bit annoying because it primes you to think that the caveats are real, but most of the time it's just something terribly obvious and not a real caveat, but the model probably has some system prompt that tells it to always consider caveats or something like that.

Same as the model starting every reply with a commitment to be "honest". LLMism are fun but I tend to just suppress them via AGENTS.md because they distract me

3 comments

At least Opus up to 4.7 or so, my experience is that Claude often uses "yes, with caveats" in place of "no, you numbskull".

"Is a meter the same as a foot?"

"Yes, exactly, you have it now, except they're different distances."

Maybe it's because I ask it to quiz me, and it really doesn't like to tell me I'm wrong. I also got a fair amount of

Claude: Ok, I will test your understanding. Question A? Question B? Question C? Question D?

Me: A=10. B=2. C=121. D is not solvable.

Claude: You got most of them right! You're very astute in saying that A=10, but actually it's 7. B=2 is exactly right! C could be 121 if we were talking base 4, but we're actually in base 10 so it's 25. D is trivially solvable and is 0.

Me: ...isn't that like 1 out of 4? How is that "most of them right"?

Claude: You're absolutely right! ...blah blah blah

A lot of things are not black and white and it’s nuanced so someone with higher intelligence will point it out. It’s an emergent behavior not a defect
This has to be one of the more annoying characteristics of the recent models.

Every answer is met with "But did you think about X", or "One more thing to consider" and it almost starts to feel manic as you dig down that hole.

Multiple times I've had to literally say "Stop with the follow-ups and suggestions. Just answer the questions clearly".