Skip to content
Canonical

ID-45 {45.2} PRODUCT — Full-corpus re-ingest + cutover (testable invariants)

ID-45 {45.2} PRODUCT — Full-corpus re-ingest + cutover

Section titled “ID-45 {45.2} PRODUCT — Full-corpus re-ingest + cutover”

Status: PRODUCT artefact (the {45.2} step of the ID-45 spec chain). Input: the ratified {45.1} RESEARCH.md (read in full). This doc turns RESEARCH §4’s per-target write-verification contracts and §5’s candidate quality-gate dimensions (D1–D11) into numbered, testable invariants (INV-1..N). {45.3} TECH and {45.4} PLAN are DEFERRED (RESEARCH §1.3) — PRODUCT defines WHAT a successful run + cutover must satisfy; HOW is TECH. Where an invariant depends on a deferred mechanism (e.g. cutover method, OQ-45-2), the invariant is stated on the OUTCOME and the mechanism noted as TECH-deferred.

De-identification: the V1 client is referred to only generically — the first client / the single-tenant V1 client / the client corpus / the handover DB. No client codename appears anywhere in this doc.

Summary

Section titled “Summary”

A clean full-corpus re-ingest of all content types (Q&A pairs, capability / methodology, sector intelligence, procurement; markdown / PDF / DOCX / URL) runs the canonical cocoindex flow end-to-end on the on-prem stack into a fresh non-prod Supabase Preview branch; every stage writes correctly; the schema is migration-complete; a quality gate passes; and the verified result is reversibly cut over onto prod in place as the first-client handover DB. The “user” of this surface is the operator running the walk plus the downstream MCP/UI consumers of the resulting rows — so the invariants below are observable conditions on the run, its written data, and the cutover, each with a concrete validation method.

Problem

Section titled “Problem”

The single-tenant V1 corpus must be re-ingested through the current pipeline so that all write targets (not the historical Q&A-only framing) are repopulated with classification, embeddings, chunks, entity mentions, relationship triples, and reference items consistent with the post-{101.7}/{101.9} write surface — then handed over on prod without data loss of retained tables. The run owns ingest end-to-end; the risk is silent partial writes, polluted verification baselines from a prior run, and an irreversible cutover.

Code-intelligence orientation (cited verbatim). GitNexus query for the re-ingest write surface resolves the run + mount surface to app_main (scripts/cocoindex_pipeline/flow.py:3072–3607, module Cocoindex_pipeline, 0 direct callers — it is the flow entry; outgoing calls include _run_stage_5_resolution, load_workspace_manifest, the _FlowStageCounter / _FlowTaxonomyMissCounter / _FlowItemFailureCounter telemetry, and _emit_pipeline_run_webhook) and the walk entry to _walk_handler (scripts/cocoindex_pipeline/server.py:383–483). The quality-gate surface this PRODUCT adapts resolves to excludeArtefacts (scripts/quality-gate.ts:506–510), whose incoming callers are the first-arc gate dimensions themselves — chunk_coverage, embedding_coverage, classified_domains_not_empty, guide_domain_filter_resolves, entity_mention_coverage, entity_relationship_coverage, countNonDraftByType, summary_coverage (cluster Scripts). This grounds the D1–D11 dimensions below: they map one-to-one onto existing named gate functions, not aspirational prose — and artefact exclusion is already a cross-cutting filter every dimension applies. (ccc search fallback was attempted; the cocoindex-code daemon was unreachable in this environment, so GitNexus findings above stand as the orientation evidence.)

Goals / Non-goals

Section titled “Goals / Non-goals”

Goals: define the observable pass/fail conditions for (a) pre-flight readiness, (b) per-target write correctness across the 9 verified write targets, (c) the quality gate, and (d) reversible in-place cutover — each testable by a SQL count/predicate, a row-shape check, a parity diff, a smoke assertion, or a rollback drill.

Non-goals (OUT of scope — see also §Out of scope / deferred): the TECH/PLAN mechanics of how each invariant is achieved; the ID-57 question_matches schema (in-flight, soft dep — the pipeline does not write it, RESEARCH §0.2); the q_a_extractions → q_a_pairs UC5 promotion ordering details (referenced but owned by bl-74 / {64.x}, RESEARCH OQ-45-3); ID-46 old-path retirement; the keep-set retention decisions (owned by {64.7} — ID-45 consumes the ratified keep-set as corpus input).

Behavior

Section titled “Behavior”

Invariants carry stable IDs (INV-1..N) so the Checker and {45.3} TECH can reference them. They are grouped: A — Pre-flight gates (must hold BEFORE the walk), B — Per- stage write correctness (one per write target + idempotency), C — Quality gate (D1–D11 promoted to thresholds), D — Cutover + reversibility. Every invariant has a validation method; the consolidated method table is in §Validation method per invariant.

A. Pre-flight gates (must all hold BEFORE the walk)

Section titled “A. Pre-flight gates (must all hold BEFORE the walk)”

  1. INV-1 — Fresh Preview branch, migration-complete. A fresh non-prod Supabase Preview branch is provisioned and the guarded repo migrations are replayed to completion before any walk. The branch is migration-complete because cocoindex never performs DDL (RESEARCH §0.1 / §3.1) — it UPSERTs into pre-existing tables only. Validation: supabase migration list on the branch shows zero pending; the 9 write targets all exist (information_schema.tables).

  2. INV-2 — Schema parity is exact (0-diff). The Preview branch schema matches the migration-intended schema with zero information_schema.columns differences, and the regenerated supabase/types/database.types.ts produces a green supabase-types-parity CI run. Validation: information_schema.columns diff old↔new = 0 rows (S187 precedent: 44 tables / 605 columns / 0 diffs = green); supabase-types-parity CI status = success.

  3. INV-3 — Pipeline service-account health probe passes. The pipeline service account is healthy on the target before the walk. Validation: SQL on a0000000-0000-4000-8000-000000000001 returns token_null = false AND identities = 1 (RESEARCH §6.4).

  4. INV-4 — Reference data restored (name-verified set). The canonical reference-data set is restored on the Preview branch using the current table names (several renamed post-T2; e.g. form_template_requirements, not template_requirements). The restore set excludes user_roles and feed_prompts (seeded separately as auth / created_by FK targets). Validation: each named reference table has a non-zero row count matching the restore source; form_template_requirements resolves (post-T2 name); user_roles and feed_prompts present via their separate seed. Open question (OQ-45-4, informational): the guide / MCP-content restore source (export-restore vs prompt-recreate) is a {64.7} decision and does not block this invariant’s pass condition (presence on target), only its provenance.

  5. INV-5 — RETAIN-class tables preserved (not truncated). The RETAIN-disposition tables are present and untruncated on the target: user_roles, user_profiles, company_profiles, feed_sources, feed_prompts, entity_aliases, entity_pair_resolutions. Truncating entity_pair_resolutions would break Stage-5 idempotency (RESEARCH §2.1), so its preservation is a hard pre-flight condition. Validation: row count > 0 (or equal to retained source) for each; explicit assertion that entity_pair_resolutions was NOT truncated.

  6. INV-6 — Pre-walk zero-row gate on every write target (OQ-45-7, HARD). Immediately before the walk, all 9 write targets are confirmed empty / truncated so a partial prior run cannot pollute the verification baseline. The gate covers source_documents, content_items, q_a_extractions, entity_mentions, entity_relationships, form_templates, form_template_fields, content_chunks, reference_items (and q_a_pairs, the downstream promotion target). Validation: SELECT count(*) = 0 for each of the 9 targets (+ q_a_pairs) on the Preview branch, asserted as the final pre-walk step. This is a blocking gate: a non-zero count aborts the walk.

  7. INV-7 — Keep-set staged with artefact exclusion. The corpus presented to the walk is the {64.7}-ratified keep-set with test artefacts excluded: title NOT LIKE '[E2E%' AND title NOT LIKE '[SUPERSEDE%'. Validation: the staged corpus contains zero titles matching either artefact pattern (RESEARCH §2.2). Open question (OQ-45-1, run-blocking): the keep-set ingest_source filter exactness depends on the bl-106 ingestion_source canonical-list ratification (Liam-owned); until ratified, the exact retained-string set is unverified. The invariant’s pass condition (artefact exclusion + the ratified keep-set) holds the moment bl-106 lands.

B. Per-stage write correctness (one invariant per write target + idempotency)

Section titled “B. Per-stage write correctness (one invariant per write target + idempotency)”

Each invariant below states the testable “correct write” condition for one of the 9 verified write targets (RESEARCH §0.1, §4.2). Integrity across targets is by construction via deterministic uuid5 seeds, not by FK constraint — the 5 cross-target FKs were dropped in S297 (RESEARCH R1), so every “linkage” check is a uuid5-derivation check, never a constraint check.

  1. INV-8 — source_documents 0→N (the cleanest oracle). After the walk, source_documents goes from 0 to N rows: one row per source file AND one per URL, PK sd:{rel_path} / sd:{url}, with content_hash and file-metadata columns populated. Validation: count(*) = (staged file count + distinct URL count); zero null content_hash; legacy app-side rows never wrote this table, so any non-zero pre-walk count is a baseline-pollution failure (cross-ref INV-6).

  2. INV-9 — content_items one-row-per-document, classified, no workspace_id. One row per ingested non-URL document; PK ci:{rel_path}; content and title non-null; content_type non-null and taxonomy-valid (with the "other" floor); no workspace_id (content is workspace-agnostic, ID-69 BI-1). Validation: count(*) = staged non-URL document count; zero null title / content / content_type; content_type ∈ taxonomy; workspace_id column is absent/unused.

  3. INV-10 — q_a_extractions one-row-per-pair, phrasings retained. One row per extracted QA pair; PK qa:{rel_path}:{idx}; extracted_question_text + extracted_answer_text + the 4 form-metadata fields populated; alternate_question_phrasings text[] (ID-94) present (possibly []). Validation: extraction count plausible vs source QA count; alternate_question_phrasings is a non-null array column on every row. The promotion to q_a_pairs is a SEPARATE downstream step (UC5) and is verified separately post-run, not by this invariant.

  4. INV-11 — entity_mentions deduped, holder metadata merged. Rows deduped per (canonical, type) with PK em:{rel_path}:{canonical}:{type}; the natural-key UNIQUE (canonical, type, content_item_id) holds (dedup done in Python before declare_row, RESEARCH R3); span + holder metadata merged (ID-101 §{101.8}). Validation: zero UniqueViolationError during the walk; mention count plausible vs corpus; holder keys present on certification-class mentions.

  5. INV-12 — entity_relationships triples at ID-101 parity (best-effort). Relationship triples (relationship_type, target_entity, source_item_id, confidence) present after the _er_dedup loop; a relationship declare failure must not fail the file (best-effort parity, RESEARCH §4.2 row 5). Validation: triple-set matches the ID-101 ground truth within benign predicate-synonym tolerance (holds/complies_with) — the {101.9} parity gate is a CLOSED proven-artifact (S348), so this invariant consumes that satisfied baseline and does not re-litigate it.

  6. INV-13 — form_templates Path-B, form_type value-safe. One template row per form file; form_type maps to a form_types.key value or is written None on the fail-path (safe, never an exception); FKs to pre-existing workspaces / user_profiles resolve. Validation: template row per form file; form_type is valid-or-null on every row.

  7. INV-14 — form_template_fields per-field, uuid5 integrity. Per-field rows with integrity via the ftf: uuid5 seed (cross-target FK dropped). Validation: field count per template plausible; zero orphan fields by uuid5 derivation (every field’s parent seed resolves to an existing template PK).

  8. INV-15 — content_chunks coverage + heading fields. RecursiveSplitter chunks ({56.8}); PK chunk:{rel_path}:{position}; content_item_id by uuid5 parity; one embedding per chunk; heading-hierarchy fields populated (ID-93 §5.3 defect closed via {56.14/15/18}). Validation: every content_items row has ≥1 chunk (uuid5 parity); zero null chunk embedding; heading fields non-null where the source has headings.

  9. INV-16 — reference_items one-per-URL, no URL-minted content_items. One reference row per distinct URL; PK ri:{url}; source_url UNIQUE; N workspace rows backlink the ONE reference row; no content_items seed is minted from a URL (flow.py:2715 BI-1/BI-2). Validation: reference-row count = distinct URL count; source_url UNIQUE holds; zero content_items rows whose source is a URL.

  10. INV-17 — Idempotency: a second walk produces zero net new rows. Re-running the walk on the same corpus produces zero net new rows across all 9 targets — UPSERT on the deterministic natural-key PKs is stable (reference_items deterministic uuid5 ids, em:/ci:/sd:/chunk: seeds identical run-to-run). Validation: snapshot count(*) per target before a second walk; post-second-walk counts are identical (delta = 0) for every target.

C. Quality-gate invariants (D1–D11 promoted to numbered thresholds)

Section titled “C. Quality-gate invariants (D1–D11 promoted to numbered thresholds)”

The gate is an ID-45 spec deliverable (the {64.8} runbook references it; ID-45 defines it for the new write surface). RESEARCH §5 marks D1–D6 + D11 must-pass and D7–D10 candidate; this PRODUCT promotes the new-surface candidates D8 (relationship parity), D9 (reference integrity) and D10 (heading fields) to must-pass, because their backing dimensions already exist as named gate functions (entity_relationship_coverage, etc. — see Problem orientation) and the run is the first to exercise them at corpus scale; D7 (entity-mention recall) stays must-pass for dedup correctness but with a plausibility band rather than an exact count (recall is probabilistic). Each threshold is enforced by the gate as a hard pass/fail.

  1. INV-18 — D1 Chunk coverage (must-pass). Every retained/ingested content_items row has ≥1 content_chunks row by uuid5 parity; zero zero-chunk documents. Validation: count of content_items with no derivable chunk = 0.

  2. INV-19 — D2 Classification coverage ≥85% (must-pass). ≥85% of content_items rows have a taxonomy-valid content_type that is not the "other" floor. Validation: count(content_type ≠ 'other' AND taxonomy-valid) / count(*) ≥ 0.85.

  3. INV-20 — D3 Guide-domain-filter resolution (must-pass). The domain filter resolves for guide-bound content (carried from the first arc; backing function guide_domain_filter_resolves). Validation: the gate’s guide_domain_filter_resolves check returns pass for every guide-bound domain.

  4. INV-21 — D4 Embedding coverage (must-pass). Every content_items row and every content_chunks row has a non-null embedding, with embedding_model recorded. Validation: zero null embedding across both tables; embedding_model non-null.

  5. INV-22 — D5 Source-document linkage by derivation (must-pass). source_documents is 0→N and every non-URL content_items.source_document_id resolves to an existing source_documents PK by uuid5 derivation (the FK is dropped — verify by derivation, never by constraint). Validation: zero non-URL content_items whose derived sd: seed has no matching source_documents row.

  6. INV-23 — D6 Q&A extraction completeness (must-pass). Extraction count is plausible vs source QA count and alternate_question_phrasings is present; and the UC5 promotion q_a_extractions → q_a_pairs succeeds post-run as a separate assertion. Validation: extraction-count plausibility check passes; post-promotion q_a_pairs row count > 0 and consistent with promoted extractions. Open question (OQ-45-3, blocks the promotion step): the bl-74 q_a_pair_history lineage columns (superseded_by, source_workspace_id) must land before the promotion — history is append-only, so a late landing loses lineage forever. The extraction half of this invariant (the ingest write) is unaffected; only the promotion half is gated.

  7. INV-24 — D7 Entity-mention recall + dedup (must-pass, plausibility band). Mention count is plausible vs corpus; zero UNIQUE violations; holder metadata present on certification-class mentions (bl-93 recall guidance ported to ENTITY_MENTION_PROMPT, G1 satisfied). Validation: entity_mention_coverage gate check passes within the plausibility band; zero UniqueViolationError in the walk log.

  8. INV-25 — D8 Relationship-triple parity (must-pass, NEW surface). The entity_relationships triple set matches the ID-101 ground-truth baseline (the post-{101.9} proven-artifact baseline) within benign predicate-synonym tolerance. Validation: entity_relationship_coverage parity diff vs the proven-artifact baseline = 0 outside the synonym-tolerance set.

  9. INV-26 — D9 Reference-item integrity (must-pass, NEW surface). One reference row per distinct URL; workspace rows backlink correctly; no URL-minted content_items. Validation: reference-row count = distinct URL count; backlink integrity holds; zero URL-sourced content_items (cross-ref INV-16).

  10. INV-27 — D10 Chunk heading-fields contract (must-pass, NEW surface). Heading- hierarchy fields on content_chunks are populated where the source document has headings (ID-93 §5.3 defect closed via {56.14/15/18}). Validation: zero chunks with null heading fields among chunks derived from heading-bearing source sections.

  11. INV-28 — D11 Artefact exclusion (must-pass). The verified corpus contains zero [E2E% / [SUPERSEDE% rows (backing function excludeArtefacts, applied across every gate dimension). Validation: count(content_items WHERE title LIKE '[E2E%' OR title LIKE '[SUPERSEDE%') = 0.

D. Cutover + reversibility invariants (outcome-stated; mechanism TECH-deferred)

Section titled “D. Cutover + reversibility invariants (outcome-stated; mechanism TECH-deferred)”

The cutover model is in-place onto prod (ID-108/S348): run into a fresh Preview branch → verify (groups A–C) → cut the verified result onto the existing prod knowledge-base project in place. This is NOT a project transfer/swap, and NOT a change to the prod project ref or its persistent staging branch — ID-64 retentions stay. The cutover mechanism (data restore vs branch promotion) is deferred to {64.8}/TECH (OQ-45-2); the invariants below are therefore stated on the OUTCOME.

  1. INV-29 — Cutover precondition: groups A–C all green. Cutover onto prod may proceed only when every group-A pre-flight gate, every group-B write-correctness invariant, and every must-pass group-C gate invariant has passed on the Preview branch. Validation: a recorded green status for INV-1..28 (must-pass subset) before the cutover step is initiated.

  2. INV-30 — Post-cutover: prod is migration-complete + parity-verified. After cutover, the prod project is migration-complete and schema-parity-verified, with the same 0-diff information_schema.columns condition as INV-2. Validation: prod migration list shows zero pending; prod↔intended column diff = 0.

  3. INV-31 — Post-cutover: smoke-green across surfaces. After cutover, a smoke pass is green across the downstream surfaces. Validation: dev-tools console clean across 5–10 surfaces (not a landing-page-only check) + a screenshot-parity browser baseline (old→new→diff) with no unexplained regression (this method caught 2 real regressions at the first cutover, RESEARCH §6.4).

  4. INV-32 — No silent data loss of RETAIN-class tables. The cutover preserves all RETAIN-class reference / workspace / user data intact: user_roles, user_profiles, company_profiles, feed_sources / feed_prompts, entity_aliases, entity_pair_resolutions, the 3 intelligence workspaces, and the reference-data set. Validation: post-cutover row counts for each RETAIN table ≥ pre-cutover counts; a diff confirms no retained row was dropped.

  5. INV-33 — A rollback path exists and is drilled. A reversible rollback path exists for each failure class and has been drilled (not merely documented): a structural failure rolls back via restore-from-snapshot (image-tag rollback is burn-safe by construction — rollback = roll-forward with a prior COCOINDEX_IMAGE_TAG, LMDB memo untouched, no KH data loss because Supabase is canonical); a content-quality failure rolls back via mass-supersede (preserving history). Validation: a recorded rollback drill for each class restores the prior verified state with zero retained-table data loss. Mechanism note: which failure classes trigger which path, and the exact restore commands, are the {64.8}/TECH decision (OQ-45-2) — this invariant fixes the outcome (a drilled, reversible path exists), not the mechanism.

Validation method per invariant

Section titled “Validation method per invariant”
INVGroupValidation method (concrete check)
INV-1A pre-flightsupabase migration list zero pending; 9 targets exist in information_schema.tables.
INV-2A pre-flightinformation_schema.columns diff old↔new = 0; supabase-types-parity CI = success.
INV-3A pre-flightSQL on service account → token_null=false AND identities=1.
INV-4A pre-flightEach named reference table row count = restore source; form_template_requirements present (post-T2 name).
INV-5A pre-flightRETAIN tables row count > 0; explicit assertion entity_pair_resolutions not truncated.
INV-6A pre-flightcount(*)=0 for all 9 targets + q_a_pairs (blocking; non-zero aborts walk).
INV-7A pre-flightStaged corpus: zero [E2E% / [SUPERSEDE% titles; keep-set per ratified {64.7} (bl-106 gates ingest_source exactness).
INV-8B writesource_documents count(*) = file count + distinct URL count; zero null content_hash.
INV-9B writecontent_items count(*) = non-URL doc count; zero null title/content/content_type; content_type ∈ taxonomy; no workspace_id.
INV-10B writeq_a_extractions count plausible vs source QA; alternate_question_phrasings non-null array per row.
INV-11B writeZero UniqueViolationError; mention count plausible; holder keys on cert mentions.
INV-12B writeTriple-set vs ID-101 ground truth within synonym tolerance ({101.9} proven-artifact).
INV-13B writeTemplate row per form file; form_type valid-or-null per row.
INV-14B writeField count per template plausible; zero uuid5-orphan fields.
INV-15B writeEvery content_item ≥1 chunk (uuid5 parity); zero null chunk embedding; heading fields non-null where source has headings.
INV-16B writeReference-row count = distinct URL count; source_url UNIQUE; zero URL-sourced content_items.
INV-17B writeSecond-walk delta count(*) = 0 per target (idempotency).
INV-18C gatecount(content_items with no derivable chunk) = 0.
INV-19C gatecount(content_type≠'other' AND valid)/count(*) ≥ 0.85.
INV-20C gateguide_domain_filter_resolves passes for every guide-bound domain.
INV-21C gateZero null embedding across content_items + content_chunks; embedding_model non-null.
INV-22C gateZero non-URL content_items whose derived sd: seed has no source_documents match.
INV-23C gateExtraction plausibility passes; post-promotion q_a_pairs count > 0 (gated on bl-74 history cols, OQ-45-3).
INV-24C gateentity_mention_coverage within plausibility band; zero UNIQUE violations.
INV-25C gateentity_relationship_coverage parity diff vs proven-artifact baseline = 0 outside synonym set.
INV-26C gateReference-row count = distinct URL count; backlink integrity; zero URL-minted content_items.
INV-27C gateZero chunks with null heading fields among heading-bearing source sections.
INV-28C gatecount(title LIKE '[E2E%' OR '[SUPERSEDE%') = 0.
INV-29D cutoverRecorded green status for must-pass INV-1..28 before cutover initiated.
INV-30D cutoverProd migration list zero pending; prod↔intended column diff = 0.
INV-31D cutoverDev-tools console clean across 5–10 surfaces + screenshot-parity baseline, no unexplained regression.
INV-32D cutoverPost-cutover RETAIN-table row counts ≥ pre-cutover; diff confirms no retained row dropped.
INV-33D cutoverRecorded rollback drill per failure class restores prior verified state, zero retained-table loss.

Coverage: every INV-1..33 has a concrete validation method (SQL count/predicate, row-shape check, parity diff, smoke assertion, or rollback drill). No invariant is aspirational.

Open questions / decisions that gate full ratification

Section titled “Open questions / decisions that gate full ratification”

These are carried forward from RESEARCH §8 (run-blocking ones marked); answers are not invented here — they surface for Liam. Each is tagged with whether it must resolve before the run or before TECH.

  • OQ-45-1 (BEFORE the run — blocking). bl-106 ingestion_source canonical list — ratify the canonical set (8 observed values; Zod enum 4→9). Gates the keep-set ingest_source filter exactness (INV-7) and pipeline_runs read-path validation. The one outstanding {64.8} G5 tail.
  • OQ-45-7 (BEFORE the run — blocking, HARD pre-walk gate). Sequence a fresh zero-row re-check on the Preview-branch write targets immediately before the walk, so a partial prior run cannot pollute the verification baseline. Promoted to INV-6 as a blocking pre-flight gate.
  • OQ-45-3 (BEFORE the promotion step — blocking that step, not the ingest). Confirm bl-74 q_a_pair_history lineage cols land before the q_a_extractions → q_a_pairs promotion (INV-23 promotion half). History is append-only; late = lineage lost forever.
  • OQ-45-2 (BEFORE TECH/cutover — does NOT block the run). Cutover mechanism — data restore (pg_dump --data-only verified branch → prod) vs branch promotion. The {64.8} authoring decision; INV-29..33 are stated on the outcome so the mechanism stays TECH- deferred.
  • OQ-45-6 (BEFORE TECH — not run-blocking). question_matches / ID-57 reconciliation: the pipeline does not write question_matches (RESEARCH §0.2); ID-57 owns its schema (in-flight). Confirm reconciliation is fully deferred to TECH and no ingest-time capture is wanted.
  • OQ-45-4 / OQ-45-5 (informational, {64.7}-owned, not run-blocking). Guide/MCP-content restore source (export-restore vs prompt-recreate); coverage_targets intent (intentionally-empty vs missing-seed).

Out of scope / deferred

Section titled “Out of scope / deferred”
  • {45.3} TECH and {45.4} PLAN — the how of every invariant (run mechanics, restore commands, gate implementation, cutover mechanism). Gated on {64.7}/{64.8}/ID-57.
  • ID-57 question_matches schema — in-flight soft dep; the pipeline does not write question_matches. Reconciliation deferred to TECH (OQ-45-6); do not re-decide ID-57’s schema here. Any “question_matches populated” expectation is OUT of scope for the ingest invariants.
  • q_a_pairs promotion mechanics — the q_a_extractions → q_a_pairs UC5 promotion ordering/lineage mechanics are owned by bl-74 / {64.x}; PRODUCT references the promotion (INV-23) and gates its precondition (OQ-45-3) but does not own its mechanism.
  • ID-46 old-path retirement — the retirement of scripts/kb_pipeline/ is a separate Task.
  • Keep-set retention decisions — owned by {64.7}; ID-45 consumes the ratified keep-set as corpus input and does not decide retention.

Findings for the Checker (RESEARCH alignment notes)

Section titled “Findings for the Checker (RESEARCH alignment notes)”

No contradiction with RESEARCH was found; the following are deliberate PRODUCT decisions the Checker should confirm against RESEARCH rather than treat as drift:

  1. D8/D9/D10 promoted candidate → must-pass. RESEARCH §5.1 marks D8 (relationship parity), D9 (reference integrity), D10 (heading fields) as candidate; PRODUCT promotes all three to must-pass (INV-25/26/27). Justification: their backing gate functions already exist (entity_relationship_coverage etc., per the GitNexus orientation), and this run is the first corpus-scale exercise of the new write surface, so a soft gate would under-protect it. RESEARCH §5.1 explicitly delegates the promote-to-must-pass decision to PRODUCT (“PRODUCT decides which candidates promote”).
  2. D7 kept must-pass but with a plausibility band (INV-24) rather than an exact count, because entity-mention recall is probabilistic — an exact-count gate would be brittle. This is consistent with RESEARCH §4.2 (“mention count plausible vs corpus”).
  3. No RESEARCH gap surfaced. Every invariant is grounded in a RESEARCH-verified fact (the 9 targets §0.1/§4.2, R1–R5 §4.1, the keep-set §2, D1–D11 §5.1, the cutover model §6, the dependency states §7). The {101.9} parity gate is treated as the SATISFIED proven-artifact (RESEARCH §7 adjudication) and not re-litigated (INV-12/INV-25).