Skip to content

DR-135 — An unanchored entity mention is not an admissible record

DR-135 — An unanchored entity mention is not an admissible record

Section titled “DR-135 — An unanchored entity mention is not an admissible record”

entity_mentions.context_snippet is written by entity_context.entity_context_for_mention, which resolves in three steps: the extractor’s declared span when it genuinely brackets the name, else a case-insensitive search for the name, else ''.

The '' case is reachable and was being written. It means one specific thing: the extractor returned a name whose surface form does not occur in content_text, and whose declared span does not bracket it either. Either the extractor normalised the name into something the document never says (“Acme Ltd” for a document that only writes “Acme Facilities Group Limited”), or it synthesised one.

id-53 PRODUCT Inv-17 has said two things about this since it was written, and they disagree:

  • its prose: “The column is NOT left NULL; downstream MCP / API / dashboard consumers see the column populated for every Stage-5-produced row.”
  • its Verifiable clause: “pipeline-produced context_snippet matches the Python-port output of extract_entity_context(...) for the same inputs” — which the empty string satisfies.

S539 implemented the span-validation half, recorded the contradiction, and escalated the wording. S540 carried it as an owner pre-req. Both framed it as a question about how the invariant is worded.

It was not a wording question, and the measurement that shows this is new. mock_llm.py emitted, for every document, a mandatory canary mention named MOCK Org <sha12> carrying the span (0, len(name)) — in-bounds, pointing at the document’s opening characters, describing nothing. That name occurs in no document, so span_brackets_entity rejects the span and the search finds nothing. The snippet was '' by construction, on every mock-tier document, whatever fixture the spec staged. Inv-17’s integration test could not have passed on the mock tier at any point, and repointing it to a content document — which is what id-415 AC-3 was going to do — would not have changed that.

The requirement the column actually serves has a current source, and it is not in the invariant text. producer/enrich.py mints a bundle anchor for a mention’s parent source_documents row, with the reason stated in the code:

A mention’s context_snippet is genuinely-read content from its parent source_documents row, so that parent sd is citable provenance and gets its anchor minted.

A pipeline-produced entity_mentions row whose context_snippet would be empty is REFUSED, not written empty. The mention does not become a record.

Three things follow, and each is part of the ruling rather than a consequence of it:

  1. The refusal is logged per mention and is never silentcocoindex.entity_mention.unanchored_refused, carrying op_id, source_document_id, rel_path, the entity name and its declared span.
  2. It is NOT an item failure and does not red the run. The item ingested correctly; one claim about it was refused. Counting it as a drop would misreport a working ingest, which is the mirror of the S522 defect id-414 exists to prevent.
  3. A curator-pinned row is never subject to the rule. An admin-merged canonical is a human decision and the walk changes nothing on it.

Inv-17’s prose stands and its Verifiable clause is the half that was wrong. The clause defined correctness as agreement with the port’s output, which makes the port correct by definition and the invariant unfalsifiable. The prose states the requirement: every produced row carries a populated snippet — now true because a row that cannot is not produced.

  • Ratify the empty string (weaken the invariant to the Verifiable clause). Cheapest, and it was on the table. Rejected: it settles the contradiction by deleting the requirement. Downstream consumers would keep rendering empty snippets, and producer/enrich.py would keep minting citation anchors for documents nothing evidences having read — a bundle citation pointing at a document on the strength of a mention that document does not contain.
  • Substitute a fallback snippet (the document’s opening text, the chunk the mention came from). Rejected on the ground entity_context.py’s own docstring already states: “Substituting unrelated text to avoid an empty string does not satisfy that requirement; it corrupts it.” A fabricated snippet is worse than an absent one, because it reads downstream as provenance.
  • Fix only the mock, leaving the pipeline writing '' in production. This would have turned Inv-17 green and is the narrowest change. Rejected: the mock is not the only extractor that can normalise a name, so the production case is real and would have been left open with its test now passing — the worst of both.
  • The mock’s canary had to become anchored, and that is a real improvement rather than test-fixing. It is now the document’s first substantial line with its true offsets. It keeps the property the canary exists for — the always-present row’s constraint-keyed fields stay a function of content_text, so two documents cannot collide on a natural key — while being anchored by construction, since no document can fail to contain its own first line.
  • A document that mentions nothing it contains now produces zero mentions. That is the intended reading, not an edge case: such a document supports no entity claims. Specs asserting mentions.length > 0 must stage a document that genuinely names something, which the per-test-content tree does.
  • Two existing Python fixtures were making unsupported claims and had to be anchored: test_cocoindex_flow_write_path.py and test_cocoindex_ingest_once.py each returned "ACME Ltd" from a fake extractor over a document that never named it. Both classes assert about primary keys and row survival, not about anchoring — the claims were always unsupported and nothing had cared until the pipeline started to.
  • No stage_counts key was added. That map is a fixed seven keys enforced by Zod at POST /api/internal/pipeline-runs/record; an eighth counter is a contract change, not an edit at the declare site. The structured log is the whole surface today, which is stated in the code rather than left to be discovered.