[01100011]

I had a moderately complex review in a large C/C++ codebase that Codex/GPT-5.6-sol already cleaned up so I threw it at Opus 5. 4 errors found. That seemed odd, so I handed it back to GPT. All were false. Opus doesn't seem to look at the wider context and understand which functions were called in certain contexts. I gave GPT's analysis back to Opus and it admitted its mistake. Maybe it's good for writing code, but as far as analysis it seems like it needs some work.

[teaearlgraycold]

A bit worrying that at no point in here did you say you investigated the errors. LLM 1 is disagreeing with LLM 2. Shouldn’t you be the tie breaker?

[zarzavat]

A man with one LLM knows if his code has errors. A man with two LLMs is never sure.

[noisy_boy]

> I gave GPT's analysis back to Opus and it admitted its mistake

How do you know if it was not mistakenly admitting its mistake?

[hodgehog11]

That has always been the major strength of GPT, that's the model you use for checking. It often nearly isn't as good for creation though.

[ranguna]

Did you ask ChatGPT the same question you asked Claude from a fresh context?