ID-101 PRODUCT — cocoindex entity-relationship + holder-rule extraction parity

ID-101 — cocoindex entity-relationship + holder-rule extraction parity (PRODUCT)

Spec chain: {101.1} RESEARCH (folded into the Task description — no separate RESEARCH.md) → {101.2} PRODUCT (this document) → {101.3} TECH → {101.4} PLAN (if warranted). Artefact kind: {N.2} PRODUCT. Behaviour-only; mechanism and validation live in the companion TECH.md (write-tech-spec).

Summary

When cocoindex becomes the sole ingest path (ID-45), the cocoindex pipeline must capture the same directed, typed entity-relationship triples (the 10-type relationship vocabulary) and the same self-versus-supplier certification-holder attribution that the legacy TypeScript classifyContent path produces today. This spec defines the parity contract for what is captured at ingest — not how it is later consumed. The acceptance bar is behavioural parity with the legacy surface for identical input, not a new capability.

Problem

cocoindex ingest today extracts flat entity mentions only and writes zero entity_relationships rows; it also never stamps certification-holder metadata onto entity mentions. Relationship triples and the S203 holder rule are produced solely by the legacy TS classifyContent path. When ID-45 makes cocoindex the sole ingest path, directed/typed relationships and holder attribution (self vs supplier) silently stop for cocoindex-ingested documents. This is an ingest-parity gap that was mis-retired as bl-92 “N/A” (see Provenance). If it is not closed before the ID-45 re-ingest, recovery costs a second full-corpus LLM pass.

Consumers of this surface (the “users” whose perspective this spec adopts):

• The pipeline operator running the ID-45 full-corpus re-ingest — they need the re-ingested corpus to land relationship triples + holder metadata in one pass.
• The code that reads entity_relationships and entity_mentions.metadata — downstream MCP tools (e.g. get_certification_status), the entity-management UI, and the quality gate — which must see the same row shapes regardless of which ingest path wrote them.

Code-intelligence orientation

gitnexus_query({query: 'entity relationship extraction holder classification ingest', repo: 'knowledge-hub'}) returned no process-grouped execution flows (processes: []); the relevant symbols surface only as standalone definitions. The matched definitions confirm the legacy surface and the absence of any cocoindex relationship/holder writer:

• Function:lib/ai/classify.ts:deriveHolderMetadata (lines 524–595, module Entities).
• Interface:lib/ai/classify.ts:ExtractedRelationship (lines 653–667) — the 10-member relationship union.
• scripts/eval-holder-rule-ts.ts:fetchHoldsRelationships, runEvaluation — existing holder-rule evaluation harness (parity oracle candidate for TECH).
• scripts/tests/test_cocoindex_flow_write_path.py (_stub_canonical_extractors, _run_ingest) — the cocoindex write-path test surface a relationship extractor must slot into.

gitnexus_context({name: 'deriveHolderMetadata', repo: 'knowledge-hub'}) — verdict-level context: 1 direct caller (classifyContent); outgoing calls to lib/entities/entity-aliases.ts:resolveAlias and lib/entities/entity-dedup.ts:canonicalise; processes: [] (not indexed into a named execution flow). This is the function whose behaviour the cocoindex holder-stamping port must reproduce.

gitnexus_context({name: 'classifyContent', repo: 'knowledge-hub'}) — verdict-level context: 5 direct callers (app/api/cron/classification-quality/route.ts:GET, app/api/items/[id]/classify/route.ts:POST, lib/ingest/markdown-orchestrator.ts:importOneFile, scripts/batch-reclassify-keywords.ts:main, scripts/reclassify-items.ts:main); it calls deriveHolderMetadata and writes both entity_mentions (classify.ts:1762 upsert) and entity_relationships (classify.ts:1799 upsert). Top-3 affected flows for parity are the ingest path (importOneFile), the on-demand classify route, and the batch-reclassify script — none of which the cocoindex path participates in today.

processes: [] for both symbols reflects that the legacy relationship/holder surface is not indexed into a named GitNexus execution flow — consistent with it being a non-orchestrated, best-effort write tail inside classifyContent rather than a discrete flow. This is itself the parity gap: cocoindex has no equivalent tail at all.

Goals / Non-goals

Goals. Achieve behavioural parity, at ingest time, between the cocoindex path and the legacy classifyContent path for two surfaces: (a) directed/typed relationship triples written to entity_relationships, and (b) self/supplier certification-holder metadata stamped onto entity_mentions.metadata.

Non-goals (explicitly OUT OF SCOPE — must not creep into TECH or implementation):

• ID-60 — Mempalace KG integration (entity_mentions temporal + provenance, mempalace_kg_* wrappers). How relationship triples are later projected into a knowledge graph is out of scope. This spec defines capture at ingest only.
• ID-71 — MCP tooling rationalisation (the 54-tool consumption surface). How entity_relationships / holder metadata is later read by MCP tools, the UI, or any consumer is out of scope.
• Schema / DDL. The entity_relationships table (10-type CHECK constraint + the entity_relationships_unique_tuple NULLS NOT DISTINCT unique index) and the entity_mentions.metadata JSONB column already exist. This spec writes to them; it does NOT create or alter them.
• ExtractByLlm / cocoindex built-in LLM extraction. Empirically ABSENT in the pinned cocoindex[postgres]==1.0.7 (see Behaviour invariant 1). Must not be specified.
• The legacy classifyContent path itself. It is the parity oracle, not a modification target. This Task does not change legacy TS behaviour.

Provenance

• Supersedes the bl-92 “N/A” retirement. The relationship-extraction parity gap was previously mis-retired as backlog item bl-92 “N/A”. This Task re-opens that finding as a concrete pre-re-ingest gate; the bl-92 N/A retirement is hereby superseded.
• C1 pre-re-ingest gate for ID-45. The capture behaviours below MUST land before the ID-45 (T7 — full-corpus reingest via cocoindex) re-ingest runs. If they land after, the relationship + holder data for the entire re-ingested corpus is absent and recovery requires a second full-corpus LLM pass.
• Mechanism precedent: ID-94 (done). ID-94 (“Q&A alternate-phrasings capture at ingest — QAFormExtraction + q_a_extractions column”) shipped the @coco.fn(memo=True) extractor pattern (direct Anthropic SDK call + Pydantic TypeAdapter validation) that this Task mirrors for relationship extraction.

The parity contract (the 10 relationship types + holder rule)

These are the legacy contracts the cocoindex path must reproduce. They are stated here so the Behaviour invariants can reference them by name; they are sourced verbatim from lib/ai/skills/classification.md (§Relationship Extraction, §Holder Disambiguation) and the entity_relationships_relationship_type_check migration constraint.

Relationship vocabulary (exactly 10 types)

Relationship	Meaning
holds	Organisation holds a certification
complies_with	Entity complies with a regulation/standard
delivers_to	Organisation delivers to a sector
uses	Entity uses a technology/product
demonstrated_by	Capability demonstrated by a project
requires	Entity requires another entity
part_of	Entity is part of another
supersedes	Entity supersedes another
references	Entity references another
evidences	Entity provides evidence for another

This set is identical to the entity_relationships_relationship_type_check CHECK constraint (ARRAY['holds', 'complies_with', 'delivers_to', 'uses', 'demonstrated_by', 'requires', 'part_of', 'supersedes', 'references', 'evidences']) and to the ExtractedRelationship['relationship'] union in lib/ai/classify.ts. No eleventh type may be invented; any extracted triple whose predicate is outside this set is dropped, not written (the DB CHECK would reject it).

Holder-disambiguation rule (self vs supplier)

For holds relationships targeting a certification entity, the holder is attributed to the relationship’s source entity, then resolved to one of two metadata states stamped on the certification’s entity_mentions row:

• { holder: 'self' } — when the resolved (canonicalised, alias-resolved, lowercased) source equals the configured client organisation name (BRANDING.organisationName).
• { holder: 'supplier', supplier_name: <resolved source name> } — when the resolved source is any other organisation (a third party: supplier, partner, landlord, data-centre operator, etc.).

Two LLM-side disambiguation rules feed this (from classification.md §Holder Disambiguation), which the cocoindex relationship-extraction prompt must carry verbatim:

1. Trigger phrases (sentence-level): if a certification mention shares a sentence or adjacent paragraph with “held by [party]”, “managed by [party]”, “maintained by [party]”, “via supplier [party]” / “via [party]”, “delivered through [party]”, “outsourced to [party]”, “provided by [party]” (party ≠ author), or “operated by [party]”, the holds triple’s source is the named third party, not the author organisation.
2. Disclaimer paragraphs (content-level): an explicit disclaimer (e.g. “Note: Certifications … are held by [party], not [author]”) makes ALL certification holds triples within its scope use [party] as source.

And one derivation-side rule (from deriveHolderMetadata, the S196 synonym fallback): where no canonical holds triple exists for a certification target, a complies_with or evidences triple targeting that certification is accepted as a holder signal only when its source is the client organisation or an organisation entity extracted in the same document. Canonical holds always wins over a synonym on a tie.

Behavior

All invariants are extraction-time, parity-with-legacy invariants. “The legacy path” means lib/ai/classify.ts:classifyContent (relationship writer at :1785, deriveHolderMetadata at :524). “The cocoindex path” means the scripts/cocoindex_pipeline/ ingest flow.

Mechanism constraint

1. The cocoindex relationship extractor MUST be implemented as a @coco.fn(memo=True) extractor that calls the Anthropic SDK directly and validates the LLM JSON via a Pydantic TypeAdapter — mirroring the three existing extractors (extract_classification, extract_qa_form, extract_entity_mentions in scripts/cocoindex_pipeline/extraction.py). It MUST NOT use cocoindex.ExtractByLlm / cocoindex.LlmSpec: both are ABSENT in the pinned cocoindex[postgres]==1.0.7 (verified — see Verification). Specifying either is a spec defect.

Relationship-triple capture parity

2. For any document ingested through the cocoindex path, the set of relationship triples captured is at parity with the set the legacy classifyContent path produces for the same input. “At parity” means: for a fixture document, the cocoindex path emits the same set of { source, relationship_type, target } triples (after canonicalisation — see invariant 5) as the legacy path emits for that same input, modulo non-deterministic LLM variation that the chosen parity oracle accounts for (the exact oracle + tolerance is a TECH concern).

3. Each captured triple is written as one row to the existing entity_relationships table with: source_entity, relationship_type, target_entity, source_item_id (the ingested document’s content-item id), and confidence. The cocoindex path writes the same column set the legacy writer writes (legacy sets confidence = 1.0); the cocoindex path sets the same default unless TECH justifies otherwise.

4. relationship_type is always one of the exactly-10 vocabulary members (parity contract above). A triple whose predicate falls outside the 10-type set is not written (it would violate the entity_relationships_relationship_type_check CHECK constraint). The pipeline does not crash on such a triple; it is dropped, consistent with the legacy path’s reliance on the same constrained vocabulary.

5. source_entity and target_entity are stored in the same canonical form the legacy path uses: alias-resolved then canonicalised then lowercased (resolveAlias(canonicalise(x)).toLowerCase() in the legacy writer). The cocoindex path produces byte-identical canonical strings for the same raw entity name, so triples written by either path collide correctly on the unique tuple (invariant 6).

6. Writing relationship triples is idempotent across re-ingestion and re-classify runs. Re-ingesting the same document does not accumulate duplicate rows: a repeated { source_entity, relationship_type, target_entity, source_item_id } tuple is a no-op (the existing entity_relationships_unique_tuple NULLS NOT DISTINCT unique index makes the duplicate insert collide; the write site treats the 23505 collision as success, not an error — same contract the legacy ignoreDuplicates: true upsert relies on).

7. Relationship capture is best-effort / non-blocking with respect to the rest of ingest, matching the legacy path (legacy wraps the relationship write in a try/catch and logs on failure rather than failing the classification). A relationship-write failure for one document does not abort the document’s other extraction outputs (entity mentions, classification, Q&A) nor the wider ingest batch. Failures are logged, not swallowed silently (the failure must be observable in pipeline logs).

8. When a document contains no clearly-stated or strongly-implied relationships, the cocoindex path writes zero relationship rows for it (the extractor returns an empty set) — it does not fabricate triples. This matches the legacy path, which only writes when result.relationships?.length is truthy.

Holder-attribution parity

9. For every certification entity_mentions row written by the cocoindex path, the holder metadata stamped onto metadata is at parity with what the legacy deriveHolderMetadata would stamp for the same (entity rows, relationships) input. The three holder states are:

   • { holder: 'self' } — resolved holds source equals the client organisation name.
   • { holder: 'supplier', supplier_name: <source> } — resolved holds source is any other organisation.
   • metadata unset / untouched — no holds (or accepted synonym) triple targets this certification. The row’s metadata is left as-is (the legacy function leaves non-matching rows untouched); holder is NOT defaulted to 'self'.

10. The “untouched when no holder signal” state (invariant 9, third bullet) is load-bearing and must not regress to a 'self' default. Downstream (post-S191) a certification with unset holder is treated as “not yet attributed” — defaulting unset to 'self' would silently mis-attribute third-party certifications. Parity here means cocoindex reproduces the legacy “leave unset” behaviour exactly.

11. Holder attribution honours the S196 synonym fallback at parity: where no canonical holds triple exists for a certification target, a complies_with or evidences triple targeting that certification is accepted as a holder signal only when (a) the target is a certification entity, AND (b) the triple’s source is the client organisation OR an organisation entity extracted in the same document, AND (c) no canonical holds triple already exists for that target (canonical holds wins over a synonym on a tie). A canonical holds triple always takes precedence over a synonym for the same target.

12. Holder source-attribution honours the LLM-side disambiguation rules (parity contract): sentence-level trigger phrases and content-level disclaimer paragraphs cause the holds triple’s source to be the named third party, which the derivation step then resolves to { holder: 'supplier', supplier_name: <party> }. The cocoindex relationship-extraction prompt MUST carry these disambiguation rules verbatim from lib/ai/skills/classification.md so the LLM attributes third-party holds triples correctly at source — the derivation step cannot recover an attribution the LLM got wrong.

13. Holder-source comparison is done on the canonical, alias-resolved, lowercased form of both the triple source and the client organisation name — identical normalisation to the legacy function (resolveAlias(canonicalise(x)).toLowerCase() vs BRANDING.organisationName.toLowerCase()). The cocoindex path produces the same normalised strings so the self/supplier decision is byte-for-byte identical to legacy for the same input.

14. Holder stamping is applied only to entity mentions of type certification. Mentions of any other entity type are never given holder metadata, matching the legacy function (which iterates rows and acts only when row.entity_type === 'certification').

15. Holder stamping is best-effort / non-blocking in the same sense as relationship capture (invariant 7): a stamping failure for one document is logged and does not abort the document’s entity-mention write or the wider batch.

Cross-path consistency invariant

16. A consumer reading entity_relationships rows or entity_mentions.metadata cannot tell which ingest path wrote a given row from its shape: the column set, canonicalisation, relationship_type vocabulary, holder-metadata shape, and idempotency behaviour are identical whether the legacy path or the cocoindex path produced it. This is the headline parity invariant — the entire point of the Task is that ID-45’s switch to a cocoindex-sole ingest path is invisible to downstream relationship / holder consumers.

Verification

Per the Q-EX2 pre-ratification empirical-verification forcing function (external-library API citations must be import-and-call verified against the installed pin before a spec is ratified):

• Date: 09/06/2026.
• Pinned version: cocoindex[postgres]==1.0.7 (requirements.txt:54).
• Symbols checked: cocoindex.ExtractByLlm, cocoindex.LlmSpec — via python3 -c "import cocoindex; print(hasattr(cocoindex, 'ExtractByLlm'), hasattr(cocoindex, 'LlmSpec'))".
• Result: ExtractByLlm → ABSENT (False); LlmSpec → ABSENT (False). Confirms the mechanism constraint (invariant 1): the @coco.fn direct-Anthropic-SDK pattern is the only viable mechanism on the pinned version. The ID-94 precedent extractors (extract_classification, extract_qa_form, extract_entity_mentions) in scripts/cocoindex_pipeline/extraction.py are confirmed present and use exactly this pattern (@coco.fn(memo=True) + anthropic.AsyncAnthropic() + Pydantic TypeAdapter).
• Schema preconditions confirmed present (not created by this Task): entity_relationships_relationship_type_check 10-type CHECK constraint and entity_relationships_unique_tuple NULLS NOT DISTINCT unique index (supabase/migrations/20260421171520_entity_relationships_unique_tuple_constraint.sql); entity_mentions.metadata JSONB column holds the { holder, supplier_name } shape (lib/ai/classify.ts:584-589).

Notes for {101.3} TECH (fresh Planner)

• Parity oracle choice is the central TECH decision. Invariant 2’s “at parity” needs a concrete oracle and tolerance. Candidate: the existing scripts/eval-holder-rule-ts.ts harness (fetchHoldsRelationships, runEvaluation) already evaluates the holder rule end-to-end — TECH should decide whether to reuse / extend it as the cross-path parity gate, and how to handle non-deterministic LLM variation (fixture-pinned vs distributional).
• Write-site location. entity_mentions are written in flow.py around lines 2142–2200; the relationship write + holder stamp must slot into the same flow without breaking the best-effort/non-blocking contract (invariants 7, 15). The cocoindex write-path test surface is scripts/tests/test_cocoindex_flow_write_path.py (_stub_canonical_extractors, _run_ingest).
• Prompt sourcing. The relationship + holder-disambiguation prompt content must be ported verbatim from lib/ai/skills/classification.md (§Relationship Extraction lines 698–717, §Holder Disambiguation lines 719–778) into a new cocoindex prompt constant (scripts/cocoindex_pipeline/prompts.py is the precedent home alongside CLASSIFICATION_PROMPT / ENTITY_MENTION_PROMPT / Q_A_FORM_PROMPT).
• Canonicalisation parity. Invariants 5 & 13 require byte-identical canonical strings between the TS resolveAlias/canonicalise pair and the Python scripts/cocoindex_pipeline/canonicalisation.py surface. TECH must verify these two canonicalisers agree (or specify the reconciliation) — this is the highest-risk parity seam.
• Estimated effort: > 2h, multi-slice (new extractor + new prompt + flow write site + holder-stamp port + canonicalisation-parity verification + parity oracle). A {101.4} PLAN decomposition is recommended.
• Sibling-only dependency note: all anticipated decomposition (extractor, prompt, write site, holder stamp, oracle) lives within ID-101. No cross-Task Subtask dependency surfaced during PRODUCT authoring; the ID-45 relationship is a Task-level ordering gate (Task.dependencies / the C1 pre-re-ingest gate), not a Subtask-level dependency.