RUN — {449.2} The bounded-vs-open comparison, executed

RUN — {449.2} id-449 extraction comparison results

Task: id-449. Date: 12/08/2026 (S555). Reads: RESEARCH.md {449.1} (the run plan, §6). Reference: the owner-approved docs/reference/testing/entity-extraction-reference.json @ canonical 35e615465 (OQ-4 closed on the S555 board). Prompt baseline: RELATIONSHIP_PROMPT and ENTITY_MENTION_PROMPT post-strip (3303f992a, 7b54b421d). Harness: direct-call, byte-faithful to extract_entity_mentions’ request (same streaming helper, retry, cache block, claude-opus-4-6, max_tokens 16384); corpus docling-converted locally with the walk’s exact converter config — the xlsx converted to exactly 4,238 chars, matching the manifest’s assertion, proving conversion parity. Memo hazard avoided by construction (no cocoindex, no LMDB). Artifacts (raw results, harness, scorer, scored summary): run-artifacts/.

Headline table

108 calls (A/B/C × 3 reps × 9 docs + A_notab × 3 × 9), $3.77 total, zero parse errors, zero API failures.

	A (bounded, control)	B (open)	C (guided-open)	A_notab (A minus tabular §)
Required recall (53 doc-groups, per-rep mean)	100%	100%	100%	100%
Table-only xlsx sectors (6)	6/6 every rep	6/6 every rep	6/6 every rep	6/6 every rep
Parse errors (strict, extra="forbid")	0	0	0	0
Excluded-term false positives	12	37	12	0
Out-of-vocab type labels	n/a	region×8, programme×3	0	n/a
Mentions total	345	349	349	323

Type agreement: every one of the 24 canonical groups has the SAME majority type in every lane — including all five contested groups. No variant fragmented the vocabulary.

Findings

F1 — Opening the vocabulary costs nothing measurable on this corpus; C is the shape

Variant C (guided-open — prefer the twelve, mint only when nothing fits) behaved identically to the bounded control: same 100% recall, same FP count, same type distribution to within noise, and zero minted labels — given permission to leave the menu, the extractor never needed to on this corpus. Variant B (fully open) minted a small, clean, semantically-reasonable tail (region for the UK regions — exactly the gap the reference predicted; programme for GreenCycle) but tripled the source-label false positives (37 vs 12) and once extracted a generic (“education trusts”). C dominates B; C ties A on quality while being DR-141-consistent.

F2 — The five contested typings are unanimous, and S551’s divergence is closed

ISO 9001:2015, ISO 9001, ISO 27001, FAS 2100, SSCM → certification in every variant, every rep, every document (60+ mentions). Procurement Act 2023 → regulation everywhere — even fully-open B never minted legislation. The real extractor is consistent where the prompt meanings point; the DB’s standard typing of ISO tokens was purely the mock artefact ({449.1} §3), now confirmed from the real tier. id-451’s grain design should treat certification as the real-tier label for this family.

F3 — The tabular section is measured net-negative: delete it

The ablation (A_notab) kept 100% recall including all six table-only sector entities and dropped false positives from 12 to 0. The 12 FPs all trace to Trigger 2 (“treat each first-column entry as a candidate”) misfiring on the xlsx Source column’s “Synthetic ONS series A/B” labels. On the only tabular fixture the corpus has, the 2,726-char section contributes no recall and all of the precision damage. claude-opus-4-6 reads markdown tables natively. Disposition per the owner’s standing instinct + this measurement: delete; re-add only with a fixture that demonstrates a gap (which also collapses OQ-7 — Trigger-2/3 fixtures are only needed if a table-recall regression ever appears).

F4 — The span-offset instruction does not work; DR-135’s fallback does all anchoring

Exact span matches: 2/345 (A), 0/349 (B), 1/349 (C), 0/323 (A_notab) — the model’s character offsets are consistently wrong (drifting deltas), while entity_name is verbatim-present essentially always. Production behaviour is unaffected — the S539/S543 _admissible_context_snippet name-search fallback anchors everything — but the prompt’s “MUST equal” span language and the model’s actual capability are unrelated. Disposition candidate for the implementing task: keep the fields as hints, drop the MUST framing, or drop span emission; the DR-135 gate itself is untouched.

F5 — Prompt caching never fires (ID-61.1 unrealised), confirmed

cache_creation_input_tokens: 0 and cache_read_input_tokens: 0 across all 108 calls — the ~1.5k-token system prompts sit under the model’s 4,096-token minimum cacheable prefix, exactly as {449.1} §5 predicted. Every extraction call bills its full system prompt uncached. Record-only: the marker is a harmless no-op; the stated ID-61.1 saving does not exist at current prompt sizes.

Honest limits

• One synthetic corpus, nine documents, entities deliberately prominent. A client corpus with dense tables/source indexes could behave differently — that is what F3’s “re-add only with a measured fixture” clause is for.
• The reference set was authored in-session (owner-reviewed and approved, S555 board).
• One model (claude-opus-4-6, the hardcoded production constant). A model change could shift typing behaviour — which is precisely why the verdict’s stability rider matters more than the vocabulary bound.
• Metric 4 (resolution-partition effects) is NOT measured here — it needs the confirming /walk at the winning variant and only becomes meaningful post-id-434 (per-type mounted resolution). Owned by the implementing task.

Verdict put to the owner (AC3)

Recommended: open the vocabulary as variant C (guided-open), with three riders — (1) a type-stability rule extending DR-147’s discipline to the type axis (labels may be added, never re-labelled by a re-run; probe as an AC), because entity_type is a component of the surrogate id and the natural key, and a flapping label is whole-corpus row churn by the type axis; (2) entity_type_override is bound by the same ruling as the base column (it has no CHECK at all today); (3) the DB CHECK is replaced by a shape check (short lower-case label), never silently dropped. Plus: delete the tabular section (F3). Alternative: keep the closed set as a dated deviation — the measured quality cost of that is zero on this corpus; the cost is DR-141 inconsistency and the silent-mint refusals a future corpus may need.