|
|
|
|
|
by dannyw
20 days ago
|
|
Modern frontier models, including Fable/Opus and 5.6, are often very loose with tool calls, and often don’t follow your schema precisely. For example, see this post for Claude models hallucinating properties for an edit/replace tool call in Pi: https://lucumr.pocoo.org/about/ I suspect some part of this comes from the noticed intelligence degradation when you do constrained decoding. Yes, you’re guaranteed schema validation, but you lose a lot of intelligence. It’s fine if you just want a classifier, a summary, a prompt enhancement, etc; but I’d be careful in agentic loops. Harnesses like Claude Code do a lot of preprocessing, repairing, cleaning, etc; as the blog post shows. You usually don’t see it. In practice it’s easier and better to just make your harness “looser” and work better with the model (they’re coming out every month or two anyway, each with their own idiosyncrasies) than to assume and force perfect correctness. Welcome to vibe applied AI ;) |
|