ID-45 {45.1} RESEARCH — Full-corpus re-ingest (scope · run · verification · cutover)

ID-45 {45.1} RESEARCH — Full-corpus re-ingest

Status: RESEARCH artefact (the {45.1} step of the ID-45 spec chain). Input to a FRESH planner’s {45.2} PRODUCT.md. PRODUCT turns §5’s candidate quality-gate dimensions and §4’s per-stage write-verification facts into numbered, testable Behavior invariants. {45.3} TECH and {45.4} PLAN are DEFERRED (§1.3) until the retention/readiness gates clear.

De-identification: the V1 client is referred to only in generic terms — the first client / the single-tenant V1 client / the client corpus / the handover DB. No client codename appears anywhere in this doc.

---

0. Code-intelligence orientation (run before authoring; cited verbatim)

The cocoindex pipeline write surface is Python (scripts/cocoindex_pipeline/*, NOT the retiring scripts/kb_pipeline/ path). ast-dataflow covers TypeScript only, so the write surface was verified by grep + Read against the live code, and the migration corpus by grep over supabase/migrations/*.sql (SQL is outside both the TS and Python symbol indexes). Per ID-93’s adversarial-verification rule, every “must be captured at ingest” claim below is backed by a citation against flow.py, the migration corpus, or a live row count — no claim from prose alone.

0.1 Verified pipeline write targets (mount_table_target — the definitive list)

Read of scripts/cocoindex_pipeline/flow.py app_main() (the mount block, lines 3195–3260) finds NINE mount_table_target(..., managed_by=ManagedBy.USER) calls:

#	Target table	flow.py mount	declare_row write site(s)	Path
1	content_items	:3195	:2091	A (content)
2	q_a_extractions	:3201	:2186 (per qa pair loop)	A (Q&A)
3	source_documents	:3207	:2055 (file), :2715 (URL)	A + URL
4	entity_mentions	:3215	:2293	A (Stage-5)
5	entity_relationships	:3226	:2378 (_er_dedup loop)	A (Stage-5, ID-101)
6	form_templates	:3235	:2907, :2977	B (form)
7	form_template_fields	:3241	:3019	B (form)
8	content_chunks	:3250	:2156 (per-position loop)	A (chunking)
9	reference_items	:3259	:2733	URL (Layer-5)

managed_by=ManagedBy.USER on every target: cocoindex UPSERTs rows only, never DDL — KH migrations own the schema (flow.py:3192–3194). This is load-bearing for the run mechanics (§3): the pipeline does not create tables; the Preview branch must be migration-complete first.

⚠️ CONTRADICTION ADJUDICATED (write-target count). ID-93 RESEARCH §1.3 (flow.py:2987–3045 truth set) lists 8 verified write targets and does NOT include entity_relationships. The live code has 9 — entity_relationships was added at ID-101 §{101.7} (flow.py:3226, “Stage-5 relationship writes land here”). ID-93’s list predates the {101.7} relationship-write landing. This RESEARCH adopts the live 9-target list. The brief’s “8 write targets” framing is therefore +1: relationship triples are now a first-class pipeline write target post-{101.7}/{101.9}, and the §5 quality gate must cover them. This is recorded to kill re-litigation of “8 vs 9”.

0.2 NEVER-written tables (verified against the live pipeline)

grep over scripts/cocoindex_pipeline/ confirms the pipeline never writes:

• q_a_pairs — referenced only in comments (flow.py:2205, extraction.py:340) as the downstream UC5 promotion target. The cocoindex write is q_a_extractions; the extraction → q_a_pairs promotion is a SEPARATE step (the “ID-45 spec chain” promotion — see §1.1, §4 target 2). NOT a mount_table_target.
• question_matches — appears only as a code comment in extraction.py:322 (question_kind note). No mount, no declare_row. (See §1.3 fact 3 + OQ-45-6: this is the ID-57 reconciliation surface, NOT an assumed ingest write.)
• citations — zero occurrences in any mount/declare/table context.
• content_history — zero occurrences. (Clean-slate at handover — §2.)

This matches ID-93 §5.1 (“the pipeline never writes q_a_pairs, question_matches, citations, or content_history”) — corroborated against the live code on re-verify.

0.3 Empirical verification block (OQ-3 / Q-EX2)

No external-library API symbols are newly asserted by THIS RESEARCH (all cocoindex / asyncpg facts are carried verbatim from cocoindex-write-model.md, which was itself verified against installed cocoindex==1.0.3 + asyncpg 0.31 at S297). The R1–R5 constraints in §4 are reproduced, not re-derived; any NEW external-API claim introduced at TECH ({45.3}) re-triggers the import-and-call check. Verification date: 13/06/2026 — internal write-surface facts confirmed against the live flow.py (line cites above); external-library facts inherited from the S297-verified write-model doc.

---

1. Purpose & scope boundary

1.1 What ID-45/T7 IS

A clean full-corpus re-ingest that runs the canonical cocoindex flow end-to-end on the on-prem cocoindex stack, writing into a fresh non-prod Supabase Preview branch; every stage writes correctly; the schema is migration-complete; and the verified result is cut over to prod IN PLACE (§6). ALL content types are in scope — Q&A pairs, capability/methodology, sector intelligence, procurement content; markdown / PDF / DOCX / URL. This supersedes any older Q&A-only framing (the ledger description’s PLAN §4.7 Q&A-only line was superseded by the S262 clarification).

The run owns ingest end-to-end (ID-64 §4 SUPERSEDED list: “the cocoindex run owns ingest end-to-end (ID-45); chunking at ingest ({56.8} RecursiveSplitter)”). Q&A extractions are written by the pipeline (q_a_extractions, target 2); their promotion to q_a_pairs is a separate UC5 step that ID-45 owns but the pipeline does not perform — the promotion ordering vs the bl-74 history-column landing is a hard constraint (§7, OQ-45-3).

1.2 ALL content types → write-target mapping

Content type	Source format	Primary write targets
Capability / methodology / sector-intel / procurement	markdown / PDF / DOCX	source_documents + content_items + content_chunks + entity_mentions + entity_relationships
Q&A pairs	form / markdown	source_documents + content_items + q_a_extractions (→ later UC5 promotion to q_a_pairs)
Form templates	form files (Path-B)	form_templates + form_template_fields (+ source_documents/content_items)
URL-sourced	URL (pullmd)	source_documents + reference_items (Layer-5; NO content_items seed minted from a URL — flow.py:2715 BI-1/BI-2)

1.3 Scope boundary — what’s DEFERRED and why

{45.3} TECH and {45.4} PLAN are DEFERRED. Execution is gated on:

1. {64.7} — Liam-owned retention export / re-link (Spine S3). Keep-set query, artefact exclusion, workspace-config export, guide/MCP-content restore source.
2. {64.8} — readiness surface + types regen + cutover runbook (Spine S4, the LAST {64.x} to complete; it gates cutover). Residual tail: bl-106 ingestion_source ratification (G5, Liam-owned).
3. The ID-57 schema slice — ID-45 writes downstream of ID-57’s question_matches schema; ID-57 is spec_needed / in-flight (terminal T1). Reference ID-57’s schema as in-flight; do NOT re-decide it here. Reconciliation is deferred to TECH (OQ-45-6). Verified write-surface fact: the cocoindex pipeline does NOT write question_matches today (§0.2) — so this is a reconciliation question for TECH, not an assumed ingest write.

RESEARCH (this doc) + PRODUCT (sibling planner) are authored NOW so the run is spec-ready the moment the gates clear. RESEARCH establishes the corpus scope, run mechanics, per-stage verification, the quality-gate seed, and the cutover model; PRODUCT turns §5’s candidate dimensions into numbered invariants; TECH/PLAN wait on the gates.

---

2. Re-ingest corpus scope (the keep-set)

Disposition vocabulary (ID-64 RESEARCH §6 + the five-category framework from historic-reingest-doc-review.md §2): RETAIN (carry unchanged) · RE-SEED (truncate + reload from seed) · MIGRATE (transform into the new shape) · RE-INGEST (content re-run through the new pipeline; classification/embedding discarded + regenerated) · DROP.

2.1 Per-table disposition (the keep-set)

Table	Disposition	Owner	Note (cite)
content_items	RE-INGEST (content only)	ID-45 spec + {64.7} export	Retain rows that are (a) client-employee added/edited OR (b) successfully ingested articles/URLs — keyed by ingest_source ∈ {'rss feed','manual','url_import'} (nullable text — verify exact stored strings, OQ-64-6/bl-106). Export SOURCE content only; DISCARD classification_*, embedding, quality_score, classification_cache_* (regenerated). Export-then-re-ingest, NOT in-place UPDATE (ID-64 §6.3). Live count 631 (ID-93 07/06).
content_history	clean slate	{64.7} (Liam)	OQ-64-9 resolved: clean slate at handover; v1-history backfill machinery NOT carried; only pre-step = client-action UUID sweep (historic-reingest-doc-review.md §2). Live count 1,225 → cleared.
feed_articles	RETAIN structure; re-link	{64.7} (Liam)	Already exists as the RSS record (workspace-scoped; content_item_id nullable link). Retain rows whose linked content is retained; re-link content_item_id after re-ingest mints new ids (C3 renumbering, §below). Live count 410 (ID-93).
feed_sources / feed_prompts	RETAIN	{64.7}	RSS source configs + relevance-prompt versions; re-ingest re-reads from these (ID-64 §6.4).
source_documents	clean slate (write target)	ID-45 run	Live count 0 — the cleanest 0→N oracle (§4).
q_a_extractions	clean slate (write target)	ID-45 run	Live count 3 (test rows); truncate before run.
q_a_pairs / q_a_pair_history	clean slate then UC5-promote	ID-45 promotion	Live count 0. Filled by the UC5 promotion step (§1.1); bl-74 history cols must land BEFORE promotion (§7).
entity_mentions	clean slate (write target)	ID-45 run	Live count 1,397 (legacy) → truncate; pipeline repopulates.
entity_relationships	clean slate (write target)	ID-45 run	ID-101 §{101.7} relationship triples. Parity gate {101.9} CLOSED proven-artifact (§7).
content_chunks	clean slate (write target)	ID-45 run	Live count 926 (legacy app-side chunker) → truncate; cocoindex RecursiveSplitter ({56.8}) repopulates.
form_templates / form_template_fields	clean slate (write target)	ID-45 run	Live count 0; Path-B writes. NB distinct from form_template_requirements (the catalogue — reference data, §2.3).
reference_items	clean slate (write target)	ID-45 run	Live count 0; URL Layer-5.
entity_aliases / entity_pair_resolutions	RETAIN (do NOT truncate)	{64.7}	OQ-64-4 resolved S296: admin-curated read-only Stage-5 input; no entities table → no FK; truncating entity_pair_resolutions breaks idempotency (ID-64 §6.4).
user_roles / user_profiles	RETAIN (mandatory)	{64.7}	user_roles drives get_user_role() which ALL RLS depends on; user_profiles is the FK target for nearly every *_by/owner column. Untouched (ID-64 §6.2).
company_profiles	RETAIN	{64.7}	Single real client/org profile record, not pipeline output (ID-64 §6.1).
pipeline_runs	clean slate / RE-SEED	ID-45 run	Run-record table; bl-106 ingestion_source Zod 4→9 read-path validation applies (§7).
si_processing_queue	RE-SEED/DROP (OQ)	{64.7}	Transient SI work items; almost certainly truncate-and-rebuild (ID-64 §6.4).
coverage_targets	OQ	{64.7}	Intentionally-empty vs missing-seed — carried open since S182 (OQ-45-7).

Reference-data restore list (the canonical pg_dump/restore set — historic-reingest-doc-review.md §2, verify current names, several renamed post-T2): taxonomy_domains, taxonomy_subtopics, taxonomy_sync_state, layer_vocabulary, guides, guide_sections, entity_aliases, company_profiles, form_template_requirements (was template_requirements pre-T2 rename — confirmed against supabase/migrations/20260520120828 which references public.form_template_requirements), feed_flags, tag_morphology_drift_flags. Excludes user_roles (auth FK) and feed_prompts (created_by FK), seeded separately. New tables since (form_types, procurement_vehicles, procurement_vehicle_instances) extend the list — taxonomy/CV disposition is owned by ID-63, not ID-45.

2.2 Keep-set query shape

• Retain content_items where ingest_source IN (<verified real strings for 'rss feed' / 'manual' / 'url_import'>) — verify exact stored values first (SELECT DISTINCT ingest_source FROM content_items; nullable text; OQ-64-6 / bl-106).
• Exclude test artefacts: title NOT LIKE '[E2E%' AND title NOT LIKE '[SUPERSEDE%' (+ the 3 demo items lost-acceptable) — the first arc proved quality baselines must exclude these (historic-reingest-doc-review.md §2).
• 8 legacy Google-News opaque-URL rows carry opaque news.google.com/rss/... source_urls — keep-set data note (re-resolve / keep-as-is / drop is a {64.7} disposition, OQ-45 informational).

2.3 Workspace + catalogue concepts

• Workspaces: re-create 3 intelligence workspaces (purpose, company-profile binding, guide, relevance_threshold) on the fresh target; GNEWS-DEDUP deleted (OQ-64-7 resolved — historic-reingest-doc-review.md §2). Content (Path-A) is workspace-AGNOSTIC (content_items has no workspace_id, ID-69 BI-1); only Path-B form-write resolves a workspace_id (cocoindex-write-model R5).
• form_template_requirements (the global catalogue, Path C) is reference data — distinct from form_template_fields (the pipeline write target). Stays globally readable (Inv-25). Do NOT conflate the two.

2.4 Disposition ownership split

• {64.7}-owned (Liam): content_items keep-set export, feed_articles re-link plan, content_history clean-slate decision, workspace-config export, guide/MCP-content restore source, reference-data restore, RETAIN-table preservation.
• ID-45-spec-owned: the write-target clean-slate + repopulation contract (the 9 targets), per-stage write verification (§4), the quality gate (§5), the cutover verification (§6). ID-45 does NOT decide retention; it consumes {64.7}‘s ratified keep-set as the corpus input.

---

3. The on-prem cocoindex run into a fresh Preview branch

The run mechanics (carried from cocoindex-write-model.md §6 + the spine S2 step). The on-prem app kh-onprem-pipeline-production (Coolify uuid ybiczck7f7e1xbdev3bk89cr, image ghcr.io/ai-solution-hub/kh-cocoindex-pipeline:sha-<commit>) is wired to prod today — the re-point to the Preview branch DSN is the S2 step.

1. Provision a fresh Supabase Preview branch (non-prod target, OQ-64-8 resolved to Preview-branch). Task 82 proved the git-integrated migration replay path: the branch replays guarded repo migrations cleanly. Schema must be migration-complete before any walk (cocoindex never does DDL — §0.1).
2. Schema-parity check post-provision: diff information_schema.columns old↔new (S187 precedent: 44 tables / 605 columns / 0 diffs = green — historic-reingest-doc-review.md §3). Regen supabase/types/database.types.ts; supabase-types-parity CI green.
3. Re-point the cocoindex env: wire the Preview branch DSN into COCOINDEX_DB_DSN on the Coolify cocoindex env (the S2 re-point; on-prem is wired to prod today). The {66.21} staging Coolify path is the non-prod option not yet wired (cocoindex-write-model §5).
4. Clear the LMDB memo for a fresh run: /var/lib/docker/volumes/ybiczck7f7e1xbdev3bk89cr_cocoindex-state/_data/lmdb/mdb (the engine memo — clearing forces fresh processing). LMDB is a cache, not the system of record (Supabase is canonical) — clearing only costs a memo rebuild.
5. Confirm COCOINDEX_SOURCE_PATH=/cocoindex-state/corpus (persistently set per S312; deploy is burn-safe regardless — “boot never walks”, ID-83). Keep it a single key (Coolify SOURCE_PATH duplication footgun).
6. Burn-safe deploy ({66.11}): the sole deploy trigger is the deploy-cocoindex job in .github/workflows/onprem-deploy.yml — it builds, bumps COCOINDEX_IMAGE_TAG to the freshly-built sha-<commit>, then deploys that pinned tag. Coolify git-auto-deploy stays disabled.
7. Smoke sequence (cocoindex-write-model §6): push → wait for the ghcr image build → bump IMAGE_TAG → stop app → clear LMDB → deploy → watch docker logs → issue a bearer-gated POST /walk (boot never walks — ID-83; ingestion fires only on this explicit one-shot update_blocking(live=False) pass) → verify rows. source_documents going 0→N is the cleanest oracle (legacy app-side rows never wrote it; live count 0).
8. Long-running ingest needs nohup … & disown (+ PYTHONUNBUFFERED=1) — plain & died by SIGHUP mid-corpus in S187 (historic-reingest-doc-review.md §3).

Pre-run C1 porting gates (must be right BEFORE the run — ID-93 §2, landed per ledger status_note): bl-92 holder-rule fix N/A on cocoindex (ID-101 supersedes as the C1 gate); bl-93 entity-recall guidance ported to ENTITY_MENTION_PROMPT (G1 satisfied 08/06); ID-94 alternate-phrasings capture-at-ingest landed; bl-42/43 coercion + PK rename landed; {64.14} bid→form rename landed. bl-106 ingestion_source ratification is the one outstanding G5 tail (OQ-45-1).

---

4. Per-stage write verification

For each of the 9 write targets, “correct write” + how to verify. The live smoke is the oracle (cocoindex-write-model §3): the mocked unit suite can prove the declare_row shape but cannot prove anything the DB enforces — cross-target FK behaviour, UNIQUE constraints, the per-target transaction model, jsonb encoding are DB-level and only the live (preview-branch) smoke proves them. A real smoke is a required gate, not a formality (it peeled a fresh latent layer on every S297 run, bugs A–G).

4.1 The per-target write model + R1–R5 constraints (cocoindex-write-model §1–2)

• Per-target autocommit, no shared txn (flow.py:12, :27–34): each target writes on its own pooled connection in AUTOCOMMIT; cross-target order is Rust-core, not parent-before-child. Related rows in different targets commit in independent, possibly-concurrent transactions.
• R1 — NO cross-target FK constraints. The 5 cross-target FKs were DROPPED in S297 (20260602073942): content_items→source_documents, content_chunks→content_items, entity_mentions→content_items, q_a_extractions→source_content_item, form_template_fields→form_templates. Integrity is by construction: every child’s FK value is a deterministic uuid5(_KH_PIPELINE_DOC_NS, "<kind>:<rel_path>...") equal to the parent’s PK on every idempotent run. Trade-off: no ON DELETE CASCADE — a wholesale wipe must delete children explicitly.
• R2 — jsonb via a connection codec (registered once on the KH pool; set_type_codec('jsonb', json.dumps/json.loads)). New jsonb columns are auto-handled; do NOT add per-call-site json.dumps.
• R3 — natural-key UNIQUE must be deduped in Python before declare_row (ON CONFLICT targets the PK only). Example: entity_mentions UNIQUE (canonical, type, content_item_id) → dedup per (canonical, type), PK em:{rel_path}:{canonical}:{type} (S297 BUG-F; flow.py:2293).
• R4 — the real FileLike API: await file.size(), (await file.content_fingerprint()).hex() (flow.py:2055); file.file_path.path is ABSOLUTE in prod — normalise to source-relative before using as storage_path/uuid5 seed (S297 BUG-A).
• R5 — workspace manifest (.kh-workspace-map.json) loaded once at flow start; Path-B concern only; skipped in ingest_file before conversion (flow.py:1740, S297 BUG-B). Content (Path-A) writes must never be affected by a workspace-resolution failure.

4.2 Per-target “correct write” + verification

#	Target	”Correct write” means	Verify (oracle)
1	content_items	One row per ingested non-URL document; deterministic PK ci:{rel_path}; content/title (NOT NULL) + taxonomy-validated content_type (NOT NULL, or "other" floor) populated (flow.py:2091). No workspace_id (ID-69).	Row count vs corpus file count (minus URL items); content_type ∈ taxonomy; title non-empty.
2	q_a_extractions	One row per extracted QA pair; PK qa:{rel_path}:{idx}; extracted_question_text + extracted_answer_text + 4 form-metadata fields + alternate_question_phrasings text[] (ID-94, migration 20260608210723) retained (flow.py:2186, extraction.py:338).	Extraction count vs source QA count; phrasings array present (possibly []). Promotion to q_a_pairs is a SEPARATE downstream step — verify separately post-run (§1.1).
3	source_documents	One row per source file AND per URL; PK sd:{rel_path} / sd:{url}; content_hash (renamed from content_fingerprint, S296), file-metadata cols populated (flow.py:2055, :2715).	0→N is THE cleanest oracle (live count 0; legacy never wrote it).
4	entity_mentions	Deduped per (canonical, type); PK em:{rel_path}:{canonical}:{type}; span+holder metadata MERGED (ID-101 §{101.8}, flow.py:2293).	No UniqueViolationError; mention count plausible vs corpus; holder keys present on cert mentions.
5	entity_relationships	Deduped _er_dedup loop; relationship triples (relationship_type, target_entity, source_item_id, confidence) (flow.py:2378, ID-101 §{101.7}). Best-effort parity (Inv-7/15 — a relationship declare failure must not fail the file).	Triple-set vs ID-101 ground truth (parity gate {101.9} CLOSED proven-artifact — §7).
6	form_templates	Path-B; form_type → form_types.key value-safe (fail path writes form_type=None, safe); FKs to PRE-EXISTING workspaces/user_profiles OK (flow.py:2907/2977).	Template row per form file; form_type valid-or-null.
7	form_template_fields	Per-field rows; integrity via ftf: uuid5 seed (cross-target FK dropped) (flow.py:3019).	Field count per template; no orphan (uuid5 parity).
8	content_chunks	RecursiveSplitter ({56.8}); PK chunk:{rel_path}:{position}; content_item_id (uuid5 parity), embedding per chunk; heading-fields contract (ID-93 §5.3 defect — closed via {56.14/15/18}) (flow.py:2156).	Chunk coverage (every content_item has ≥1 chunk); embedding non-null; heading fields populated.
9	reference_items	One row per URL; PK ri:{url}; N workspace rows backlink the ONE reference row; source_url UNIQUE (flow.py:2733, BI-3).	Reference row per distinct URL; no content_items seed minted from a URL.

Live-smoke-is-the-oracle discipline: the DB-enforced behaviours (FK-by-construction, UNIQUE dedup, per-target txn, jsonb codec) are provable only on the preview-branch smoke — the mocked suite is structurally blind to them (cocoindex-write-model §3). The preview-branch smoke is therefore a HARD gate before cutover.

---

5. Quality gate equivalent for the cocoindex write surface (an ID-45 spec deliverable)

Seed: the first-arc 8-dimension must-pass profile (old quality-gate.ts; docs 2+3 via historic-reingest-doc-review.md §3): 6 must-pass dimensions — chunk coverage, classification coverage ≥85%, guide-domain-filter resolution, embedding coverage, … This gate is an ID-45 spec deliverable (NOT {64.8} scope) — the {64.8} runbook references the gate; ID-45 defines it for the NEW write surface (incl. the new tables). PRODUCT will turn the candidate dimensions below into numbered, testable invariants.

5.1 Candidate gate dimensions (adapted to the 9-target surface)

#	Dimension	Must-pass?	Target(s)	Adapted contract
D1	Chunk coverage	YES	content_chunks	Every retained/ingested content_items row has ≥1 content_chunks row (uuid5 parity); no zero-chunk document.
D2	Classification coverage ≥85%	YES	content_items	≥85% of rows have a taxonomy-valid content_type ≠ "other" floor.
D3	Guide-domain-filter resolution	YES	content_items / taxonomy	Domain-filter resolves for guide-bound content (carried from first arc).
D4	Embedding coverage	YES	content_items + content_chunks	Every content row + chunk has a non-null embedding (embedding_model recorded).
D5	Source-document linkage (0→N)	YES	source_documents	source_documents 0→N; every non-URL content_items.source_document_id resolves by uuid5 (the FK is dropped — verify by derivation, not constraint).
D6	Q&A extraction completeness	YES	q_a_extractions	Extraction count plausible vs source QA; alternate_question_phrasings present; UC5 promotion to q_a_pairs succeeds post-run (separate assertion).
D7	Entity-mention recall + dedup	candidate	entity_mentions	Mention count plausible; no UNIQUE violation; holder metadata on cert mentions (bl-93 recall guidance ported).
D8	Relationship-triple parity (NEW — ID-101)	candidate	entity_relationships	Triple set matches ID-101 ground truth (post-{101.9} proven-artifact baseline); benign predicate-synonym tolerance (holds/complies_with).
D9	Reference-item integrity (NEW — URL)	candidate	reference_items	One reference row per distinct URL; workspace rows backlink correctly; no URL-minted content_items.
D10	Chunk heading-fields contract (NEW)	candidate	content_chunks	Heading hierarchy fields populated (ID-93 §5.3 defect closed via {56.14/15/18}).
D11	Artefact exclusion	YES	content_items	Zero [E2E% / [SUPERSEDE% rows in the verified corpus.

D1–D6 map the first-arc must-pass profile; D7–D11 are the new-surface additions (relationship triples, reference items, heading-fields, artefact exclusion) that the old profile predates. PRODUCT decides which candidates promote to must-pass.

---

6. Cutover-onto-prod mechanics

Model (ID-108 ratification, S348): the verified Preview-branch re-ingest cuts over onto prod IN PLACE. The current prod knowledge-base (project ref rovrymhhffssilaftdwd)

• its persistent staging branch (turayklvaunphgbgscat) remain the first-client handover DB; ID-64 retentions stay in place. The platform DB is OUR separate dev instance — NOT the re-ingest target. So: run into a fresh Preview branch (non-prod) → verify (§4, §5) → cut the verified result onto the existing prod project in place. This is NOT a project transfer/swap (the old Vercel-env-repoint / Supabase-project-transfer cutover is SUPERSEDED — historic-reingest-doc-review.md §4).

6.1 Open mechanism question (flag, do NOT decide — {64.8} authoring)

Verified Preview branch → prod by (a) data restore (pg_dump --data-only of the verified branch → prod) OR (b) branch promotion. The old project-transfer answer is dead; the choice is the {64.8} authoring decision (OQ-64-4 of that doc / OQ-45-2 here).

6.2 Carried operational gotchas (verbatim — historic-reingest-doc-review.md §3)

• Session-mode pooler (port 5432) for any migration touching auth.* — export SUPABASE_DB_URL_DIRECT for that invocation only.
• pg_dump --data-only emits no SET search_path — prepend it or fully-qualify; use the Postgres-17 binaries (/opt/homebrew/opt/postgresql@17/bin/).
• Protected-branch reset is a silent no-op via MCP — supabase db push --linked only.
• cat supabase/.temp/project-ref before any prod db push; relink to prod only at the deliberate cutover step (CLAUDE.md project-ref drift gotcha — a stale prod link lands a push on prod silently).
• Wipe-rebuild fallback only (if a Preview branch is ever NOT used): DROP SCHEMA public CASCADE does not drop extensions (move/drop vector first); storage policies survive (DO-block drop); config tables need TRUNCATE CASCADE before restore.

6.3 Rollback patterns

• Structural failure → restore-from-snapshot. Image-tag rollback is burn-safe by construction ({66.12}): rollback = roll-forward, same deploy-cocoindex PATCH+deploy with a prior COCOINDEX_IMAGE_TAG; the LMDB memo is untouched; the rolled-back image boots lifespan-only and never walks. LMDB restore (if needed) is a memo rebuild only — never KH data loss (Supabase canonical, inv 20).
• Content-quality failure → mass-supersede (preserves history).
• Decide which failure classes trigger which in the {64.8} runbook.

6.4 Post-cutover triage

Cutover exercises rarely-fired paths → triage = dev-tools console across 5–10 surfaces (not just a landing-page check) + a 4-agent screenshot-parity browser baseline (old→new→diff) — caught 2 real regressions at the first cutover (historic-reingest-doc-review.md §3). Pipeline service-account health probe: SQL on a0000000-0000-4000-8000-000000000001 expects (token_null=false, identities=1).

---

7. Dependency map

Dep	Spine	State at spec-authoring	Note
Non-prod target (Preview branch)	S2	OPEN (wire at re-ingest time)	staging Coolify exists; wire fresh Preview branch DSN into COCOINDEX_DB_DSN (Task 82 proved git-integrated migration replay).
{64.7} retention export/re-link	S3	OPEN (Liam-owned; all OQ inputs resolved S341)	keep-set query, artefact exclusion, workspace export, guide/MCP restore source.
{64.8} readiness + types + cutover runbook	S4	OPEN (LAST; gates cutover)	G1✅ G2-dropped G3✅ G4✅ G5 bl-42/43✅ + bl-106 escalated (Liam, the one outstanding G5 tail) · G6 ID-58/59✅ prod + ID-57 spec-slice outstanding.
ID-57 schema slice (question_matches)	Lane B	in-flight, soft dep	spec_needed; ID-45 writes downstream. Reference as in-flight; do NOT re-decide. Reconciliation deferred to TECH (OQ-45-6). Pipeline does NOT write question_matches (§0.2).
ID-101 relationship/holder capture	S0b	SATISFIED ({101.9} CLOSED proven-artifact S348)	See adjudication below. Listed as a dep, satisfied.
ID-28	—	DONE	(ledger dep).
ID-36 / ID-41 / ID-69	—	(ledger deps)	ID-69 = content workspace-agnostic ratified (BI-1).
ID-94 / bl-92/93 / {64.14} / {64.15}	Lane A	landed (per ledger status_note)	C1 pre-run constraints: ID-94 phrasings, bl-93 recall port, {64.14} rename, bl-74→{64.15} history cols.
bl-106 ingestion_source ratification	Lane A G5	OPEN (Liam)	the one outstanding {64.8} G5 tail; blocks the keep-set ingest_source filter exactness + pipeline_runs Zod validation.

CONTRADICTION ADJUDICATED ({101.9} state). v1-completion-sequence.md (Spine S0b, S341 snapshot) shows {101.9} parity gate RED and lists it as “the one open ID-45 gate”. That doc is STALE on this single point. The live ledger (read 13/06/2026) shows ID-101 done, status_note: “CLOSED S348 — {101.9} parity PROVEN-ARTIFACT (Liam-adjudicated): post-{101.10} coco holder=self = legacy+ground-truth; residual = benign predicate synonym (holds/complies_with); symmetric gate blocked by harness gap (bl-300). NOT a re-ingest trigger; ID-45 stays gated on V1-sequencing.” This RESEARCH treats ID-101 relationship/holder capture as a SATISFIED pre-req (closed proven-artifact post-S341), while still listing it in the dependency map as “satisfied”. Do NOT re-litigate {101.9}.

Gates still OPEN at spec-authoring time: S2 (Preview wiring), S3 ({64.7}), S4 ({64.8} — incl. bl-106 + the ID-57 spec-slice). These are why TECH/PLAN are deferred (§1.3).

---

8. Open questions for Liam (OQ-45-N)

#	Question	Blocks the run?
OQ-45-1	bl-106 ingestion_source canonical list — ratify the canonical set (8 observed values; Zod enum 4→9). Needed for the keep-set ingest_source filter exactness + pipeline_runs read-path validation.	BLOCKS (keep-set + run-record validation). The one outstanding {64.8} G5 tail.
OQ-45-2	Cutover mechanism — data restore (pg_dump --data-only verified branch → prod) vs branch promotion. {64.8} authoring decision.	Informs TECH/{64.8}; does not block the run (blocks cutover).
OQ-45-3	UC5 promotion ordering — confirm bl-74 q_a_pair_history lineage cols (superseded_by, source_workspace_id) land BEFORE the q_a_extractions → q_a_pairs promotion step (landed via {64.15}, re-confirm sequencing). History is append-only — late = lineage lost forever.	BLOCKS the promotion step (post-ingest).
OQ-45-4	Guide / MCP-content restore source — export-restore vs prompt-recreate: are SQL dumps of guides/guide_sections (+ feed config) preserved in data/exports? Which standalone source docs survive in Storage/local? ({64.7}).	Informs {64.7}; not run-blocking.
OQ-45-5	coverage_targets intent — intentionally empty (feature inert) or missing seed? Affects {64.7} disposition (carried open since S182).	Informs {64.7}; not run-blocking.
OQ-45-6	question_matches / ID-57 reconciliation — the pipeline does NOT write question_matches (§0.2); ID-57 owns its schema (in-flight). Confirm reconciliation is fully deferred to TECH and no ingest-time capture is wanted.	Informs TECH; not run-blocking (ID-57 is downstream of q_a_pairs).
OQ-45-7	UAT-before-run zero-row re-check — sequence a fresh zero-row re-check on the Preview branch write targets immediately before the walk (source_documents/q_a_pairs/etc. confirmed empty/truncated), so a partial prior run can’t pollute the verification baseline.	BLOCKS (pre-walk gate) — recommend HARD pre-walk gate, consistent with the §3/§4 run procedure.

---

9. Verification (OQ-3 pre-ratification empirical check)

• Internal write-surface facts (the 9 targets, NEVER-written tables, R3 dedup, R4 FileLike usage): verified 13/06/2026 against the live scripts/cocoindex_pipeline/flow.py
  • extraction.py (line cites in §0–§4). PRESENT.
• Migration-state facts (form_template_requirements post-T2 name; dropped cross-target FKs 20260602073942): verified by grep over supabase/migrations/*.sql. PRESENT.
• External-library facts (cocoindex 1.0.3 / asyncpg 0.31 autocommit model, jsonb codec, async FileLike): inherited verbatim from cocoindex-write-model.md (itself verified against installed cocoindex==1.0.3 + asyncpg 0.31 at S297). No NEW external-library symbol is asserted by this RESEARCH; any introduced at TECH re-triggers the import-and-call check.
• No greenfield disclaimer needed — every claimed symbol resolved against live code or the migration corpus.