|
|
|
|
|
|
by latentsea
1 day ago
|
|
|
> I'm not ready to blame Opus 5 for being stupid. Perhaps we have a prompt buried somewhere that's essentially asking it to be pedantic, and it's just obeying the prompt. This is a known failure mode. Sadly, working on a software engineering team doing agentic engineering now means we need to build and maintain suites of evals that measure these things, so that we can measure the effects of changes to harness components including how they perform under model upgrades. But... for a traditional software engineering team that has no experience in this... How do we even do it? |
|
|