Better Models, Worse Tools: Why Opus 4.8 Breaks Third-Party Edit Schemas

Armin Ronacher reported on July 4, 2026 that Anthropic’s newest models call custom edit tools with fields the tool’s schema never declared — and that this is worse on the flagship models than the small ones. Simon Willison flagged it the same day as a case where “better models” produce “worse tools.” (Source: Armin Ronacher, lucumr.pocoo.org, 2026-07-04)

Key facts:

• Opus 4.8 and Sonnet 5 both show the regression. They call the edit tool in Pi (Armin’s harness) with extra, invented keys. (Source: Armin Ronacher, 2026-07-04)
• Haiku and older Anthropic models do not. The failure tracks the newest, most capable models — not the weakest. (Source: Armin Ronacher, 2026-07-04)
• The core edit is usually correct. The oldText/newText content is right; only the surrounding arguments violate the schema, so the harness rejects an otherwise-good call. (Source: Simon Willison, 2026-07-04)
• Codex models tested clean. Armin did not observe the same schema drift on OpenAI’s coding models. (Source: Armin Ronacher, 2026-07-04)

What the model actually sends

Pi’s edit tool expects a flat object with a nested edits[] array:

{
  "path": "some/file.py",
  "edits": [
    { "oldText": "text to replace", "newText": "replacement text" }
  ]
}

Opus 4.8 and Sonnet 5 keep oldText/newText correct but bolt on keys that were never in the schema. Armin catalogued “a whole zoo” of them, including:

type, id, kind, unique, requireUnique, matchCase, in_file,
forceMatchCount, children, notes, cost, oldText2, newText2,
event.0.additionalProperties

Because those keys aren’t in the declared schema, a strict harness rejects the tool call outright — even though the edit inside it would have worked.

The likely cause: trained on Claude Code’s own format

Armin’s hypothesis is that recent Anthropic models were reinforcement-trained hard on Claude Code’s built-in edit tool, which uses a flatter, search-and-replace schema — file_path, old_string, new_string, and an optional replace_all flag. That focused training built a strong prior for one specific edit shape. Point the model at a different edit schema (nested arrays, different key names) and the prior misfires, sprinkling in fields that would have made sense for Claude Code’s tool but are noise everywhere else.

That’s the uncomfortable part: the same training that makes Opus 4.8 excellent inside Claude Code is what makes it sloppier inside your harness.

What this means if you’re building agent tools

If you define your own tools for Claude — an edit tool, a patch tool, any function with a non-trivial nested schema — this is a live gotcha on the newest models:

• Don’t assume newer = more schema-compliant. Test tool-call conformance on Opus 4.8 and Sonnet 5 specifically. A harness that passed on older Claude models can start failing after you bump the model id. See our guide to switching to Opus 4.8 and switching to Sonnet 5 — run your tool suite before you flip production.
• Consider mirroring Claude Code’s edit shape. If your tool matches the old_string/new_string flat format the model was trained on, you swim with the prior instead of against it. Simon Willison raised exactly this — whether harnesses should ship multiple edit-tool variants tuned per underlying model.
• Validate, then coerce, don’t just reject. Since the oldText/newText payload is usually correct, a harness that strips unknown keys before validating can salvage most of these calls rather than erroring the whole turn.
• This is the sharp edge of specialization. It rhymes with the broader pattern we wrote up in constraint decay in LLM coding agents: models tuned aggressively for one harness get subtly worse at the general case.

If Opus 4.8 is your daily coding model, our Claude Code daily-driver setup covers running on it — where, notably, this bug doesn’t bite, because the tool is the one it was trained on.

Sources

• Better Models: Worse Tools — Armin Ronacher, lucumr.pocoo.org, 2026-07-04 (primary report)
• Better Models: Worse Tools — Simon Willison, 2026-07-04 (commentary + takeaway)