LLM Edit Op

The bug this exists to prevent

Ask a model to propose edits to a contract and it returns something like:

  { "anchor": "Tenant shall pay all Operating Expenses", "replacement": "..." }

Then you search the document for that anchor to apply the edit. It isn't there. What is there is "Tenant shall pay Operating Expenses" — the model added "all" while quoting.

The edit silently does nothing. No exception, no failed schema check: the JSON was valid, the anchor was a plausible string, the apply step simply matched zero occurrences. In a batch of twelve proposed redlines, three quietly don't apply and the document looks reviewed.

Worse is the near-miss that does match — an anchor the model paraphrased into something that happens to exist elsewhere in the document, so the edit lands in the wrong clause.

Rule 1 — verify the quote against the source, locally

Every anchor and quote the model claims to have copied is checked to appear verbatim in the original, modulo whitespace and smart punctuation.

Those two tolerances are necessary, not sloppy: documents arrive with curly quotes, non-breaking spaces and soft line-wraps that no model reproduces byte-for-byte, and rejecting on those alone fails every real document. Anything beyond that is treated as not-found.

This runs locally. No second model call to "check" the first — that costs money, adds latency, and asks the same class of system to audit itself.

Rule 2 — coerce what is safely coercible, reject the rest

Models wrap JSON in prose, in markdown fences, in fences tagged as the wrong language, and occasionally emit legacy field values. All of that is recoverable and is recovered.

null is returned only on a real parse failure. The distinction matters: a caller that cannot tell "the model wrote prose around valid JSON" from "the model produced nothing usable" will either throw away good output or ship broken output, and both look like the model being flaky.

Rule 3 — one authoritative definition of the shapes

Redline and summarize both ask a model for JSON, and both are parsed here. The types are shared deliberately, so a field renamed in one flow cannot silently diverge from the other — a drift that surfaces as one panel rendering blank while the other works.

What this does NOT do

No model calls, no prompting, no .docx reading or writing, no applying of edits. It defines the contract and verifies the reply against your source text. You bring the model and the applier.

It cannot tell you whether a correctly quoted edit is a good edit. It checks provenance, not judgement.

Verified

33 tests covering fenced and prose-wrapped JSON, wrong fence tags, legacy field coercion, whitespace and smart-punctuation tolerance, anchors that do not appear in the source, and the parse failures that must return null.