ID-138 {138.4} PLAN — Corpus durable home decomposition

PLAN — Corpus durable home + record identity/curation durability

Status: DRAFT — 03/07/2026. Authored by a FRESH Planner (Q-PLANNER-2 — a different instance from the {138.3} TECH author). Decomposition source: the Checker-gated ratified TECH.md (specs/id-138-corpus-durable-home/TECH.md). Controlling scope frame: specs/id-138-corpus-durable-home/notes/s440-corpus-durable-home-decision.md §10 (S441 reframe); in-force rulings DR-023/024(i)/025/026/027, DR-020, DR-015.

Twelve implementation Subtasks {138.5}–{138.16}. Sibling-only deps throughout — every cross-Task seam (id-131/132/127/134/45/69/133) is a Task-level coordination note inside details, never a Subtask dependency. No constraint was bent; no escalation required.

1. Overview

ID-138 gives the admitted corpus a durable home and makes record identity + curation survive the ingest engine. The change is mechanically risky, behaviourally settled (TECH+PLAN tier — behaviour is fixed by the DR rulings). The decomposition follows the TECH change inventory (§3: migrations M1–M4, Python P1–P5, TypeScript T1–T4, ops O) plus the two cross-cutting rulings that need their own home: R(a) (SEED-CONTRACT freeze + identity-neutral lift) and R(d) (promotion-boundary audit).

ID-138 owns the contract AND the durable-persistence mechanism (Orchestrator ruling): the off-engine write path that makes ingest-once derived rows survive orphan-cleanup, the registry-keying, and the schema. id-45 is a downstream consumer (Task-level coordinate) — its onboarding execution is NOT in ID-138.

2. Architecture decisions carried from TECH

Schema-first foundation. source_documents columns (M1) and the identity resolver fn (M2) are shared foundations everything else consumes; they land first and serialise on the migration chain + a single db push target.
Two load-bearing cores, kept separate. R(id) admission-minted identity (the walk re-key, {138.10}) and R(e) survives-the-engine (the one-shot ingest-once path, {138.11}) are the hard problems. Both touch flow.py, so {138.11} sequences AFTER {138.10} to avoid file-ownership conflict.
Identity is content_hash-keyed, never path-keyed. The M2 resolver (called by both the Python walk and the TS upload leg) is shared identity infrastructure — hence its own Subtask, so neither consumer depends on the other.
The corpus bucket is demoted (DR-025): kept-evidence store + gated-upload landing + optional ingest-once archive. It is provisioned once ({138.8}) as the storage-side foundation the write-back re-point, upload leg, pull-sync, and seed all consume.
Writer fencing is a shared cross-language primitive ({138.9}: pg advisory-lock RPC + TS
- Python helpers). Each ID-138-owned writer (write-back, upload, pull-sync) acquires it; ID-69’s bulk-load acquiring it is ID-69’s own work.
No record-curation store (R(d)/DR-026): the former (d) design point is KILLED. R(d) resolves to a stated promotion-boundary invariant + an audit + a regression test — not code that builds a store.
Mutable path column shape — TECH §2.2 delegated the pick to PLAN; decided here: keep storage_path as the frozen object key (unchanged — the uuid5 seed source + bucket key) and ADD a new mutable logical_path text ({138.5}). On mint logical_path := storage_path; a client rename updates logical_path only (via {138.10} walk + {138.13} upload), leaving storage_path + identity frozen. Repurposing/renaming storage_path (NOT NULL, the object key + uuid5 seed) was rejected as churny + identity-risky. This is a landed PLAN decision, NOT a choice re-delegated to the {138.5} Executor.

3. Dependency graph

1
{138.5} M1 schema (source_documents cols)          {138.8} T3 corpus bucket
2
   │  │  │  │                                          │  │  │  │
3
   │  │  │  └────────► {138.9} writer-fence ◄──────────┘  │  │  │
4
   │  │  │                   │  │  │                       │  │  │
5
   │  │  └► {138.6} M2 id fn  │  │  │                       │  │  │
6
   │  │        │  │           │  │  │                       │  │  │
7
   │  └► {138.7} M3/M4 fns    │  │  │                       │  │  │
8
   │           (erasure/reaper)  │  │                       │  │  │
9
   │                             │  │                       │  │  │
10
   ├──► {138.10} identity core ──┼──┼───────────────────────┼──┼──┐ (flow.py)
11
   │        │ (P1/P2/P3, +M2)    │  │                       │  │  │
12
   │        └► {138.11} ingest-once P4 (flow.py, after 10)  │  │  │
13
   │        └► {138.16} R(d) promotion-boundary audit/test  │  │  │
14
   │                                                        │  │  │
15
   ├──────────────► {138.13} T2 upload leg (+M2, fence, bucket)  │
16
   │                                                        │  │
17
   ├──► {138.15} R(a) SEED-CONTRACT + identity-neutral lift ┘  │
18
   │                                                           │
19
   └──► {138.14} P5 pull-sync (server.py; +bucket, +fence) ────┘
20
                    {138.12} T1 write-back re-point (+bucket, +fence)

4. Task list (ordered)

Phase 1 — Foundations (parallel: schema ∥ storage)

{138.5} M1 — source-binding register + admission lifecycle schema. deps: none.
{138.8} T3 — corpus bucket provisioning (env-isolated). deps: none.

Phase 2 — Shared primitives (depend on M1)

{138.6} M2 — resolve_or_mint_source_identity fn. deps: {138.5}.
{138.7} M3/M4 — tombstone cascade + orphan reaper + citations pre-flight. deps: {138.5}.
{138.9} Writer-fence barrier primitive + ops docs. deps: {138.5}.

Checkpoint A — Foundations

bun run test + pytest green; types regen clean; migration chain intact (no id-131 collision); bucket provisions privately + idempotently; fence proves mutual exclusion.

Phase 3 — Mechanism cores + write paths (mostly parallel; flow.py serialises 10→11)

{138.10} Identity core (P1/P2/P3) — walk re-key + registry-keyed seeds. deps: {138.5}, {138.6}.
{138.12} T1 — write-back file leg → Storage PUT (+ q_a sidecar). deps: {138.8}, {138.9}.
{138.13} T2 — upload leg: gate-pass → Storage PUT + minted sd row. deps: {138.5}, {138.6}, {138.8}, {138.9}.
{138.14} P5 — content-hash-gated pull-sync. deps: {138.5}, {138.8}, {138.9}.
{138.15} R(a) — SEED-CONTRACT freeze + identity-neutral lift test + dry-run gate. deps: {138.5}, {138.8}.

Phase 4 — Depend on the identity core (flow.py)

{138.11} P4 — one-shot ingest-once extraction path (survives-the-engine). deps: {138.5}, {138.6}, {138.10}.
{138.16} R(d) — promotion-boundary audit + regression test. deps: {138.10}.

Checkpoint B — Complete

All R(a)–R(e) + R(id)/R(ops) acceptance checks (TECH §4) green; identity-neutral staging dry-run on the synthetic Platform corpus shows zero identity churn + a clean re-walk (Liam’s eyes); id-132 first-publication blockers (R(id) identity + R(d) boundary) satisfied.

5. Parallelisation & serialisation

Parallel: {138.5} ∥ {138.8} (schema vs storage, disjoint). {138.10} (flow.py) ∥ {138.12} (write-back.ts) ∥ {138.13} (folder-drop.ts) ∥ {138.14} (server.py) ∥ {138.15} (doc/test) — disjoint files once their deps land.
Serialise on flow.py: {138.10} → {138.11} (both edit flow.py; {138.11} deps {138.10}). {138.16} reads the same engine-target region — dep {138.10}.
Serialise on the migration chain: {138.5}, {138.6}, {138.7}, {138.9} each add migrations to supabase/migrations/ against one db push target. They touch disjoint DB objects (new columns vs distinct new fns/RPC) so they do not object-collide, but their db push must be applied in sequence — dispatch sequentially or coordinate the push (Task-level operational note, not a Subtask dep).

6. Cross-Task coordination seams (all Task-level; none is a Subtask dep)

Seam	How modelled	Where noted
id-131 (schema family)	M1 uses `id138_` stems + timestamps after `20260702120000_id131_`; must not collide with id-131 migrations / the `content_items` elimination.	{138.5} details
id-132 (publication gate)	ID-138 BLOCKS id-132 first bundle publication on R(id) identity + R(d) boundary (reverse dep, Task-level).	{138.10}, {138.15}, {138.16} details
id-127 / {127.20} (DR-015 ingress + env-isolation)	Pull-sync is outbound HTTPS — removes the push-ingress need, strengthens DR-015; bucket key-scoping to project-ref routes {127.20}.	{138.8}, {138.14} details
id-134 (sync scripts → seeding)	`sync-platform-corpus.sh` retires to fixtures-to-bucket seeding; the R(a) lift references id-134’s seeding tool.	{138.14}, {138.15} details
id-45 (ingest-once onboarding)	ID-138 owns the R(e) mechanism; id-45’s one-time onboarding EXECUTION + client sign-off stays with id-45 (Orchestrator ruling).	{138.11} details
ID-69 (operator bulk-load)	Fifth bucket writer; must acquire the {138.9} barrier. Wiring ID-69’s participation is ID-69’s own work.	{138.9}, {138.8}, {138.15} details
id-133 (ontology CV medium, DR-027)	NAMED SEAM, NOT ID-138 work. Base CVs move docs-site → canonical repo; every bundle carries the effective ontology. Flagged so sequencing avoids id-133’s register-medium change; no ID-138 Subtask.	this PLAN §6

7. Risks and mitigations

Risk	Impact	Mitigation
R(id) ships wrong → first publication orphans every citation	CRITICAL	Identity-neutral migration ({138.15}) + pre-first-publication gate (id-132) + staging dry-run; “the last cheap moment”.
flow.py file-ownership conflict between {138.10} and {138.11}	Med	{138.11} deps {138.10} — serialised, never same-wave.
Migration-chain race across {138.5/6/7/9}	Med	Serialise `db push`; disjoint DB objects; `id138_*` stems + monotonic timestamps.
LMDB loss → silent full re-extraction bill	High	Documented ({138.9}) + writer-fenced; VPS store non-rebuildable.
Citations CASCADE fires on `full_reprocess`	High	Pre-flight gate ({138.7}); `full_reprocess` demoted to pre-launch tool.
Env-isolation blast radius (bucket = irreplaceable byte layer)	Med	Project-ref scoping ({138.8}); backup/restore drill flagged as a §8.5 launch gate (roadmap, before client #2).

8. Open questions (for the Orchestrator / owner)

User-facing erasure surface. TECH §3 lists only the DB fn (M3) for GDPR tombstone + client-initiated discard; no API route / admin CLI is in the change inventory. {138.7} scopes to the fn + tests (callable directly). Confirm whether a user-facing erasure/discard surface is ID-138 scope or a follow-up. (Not a bent constraint — a scope boundary question.)
ID-69 ↔ fence sequencing direction. ID-69’s bulk-load must acquire the {138.9} barrier; {138.9} must land before ID-69 wires acquisition (Task-level dep, correct direction). Flag for the Orchestrator’s cross-Task sequencing so ID-69 does not ship an unfenced writer.
retention_class default assignment site. R(b) defaults are distributed: uploads → keep_and_watch in {138.13}; ingest-once → ingest_once in {138.11}; external_referenced is schema-only at launch (no connector yet). Confirm this distribution matches the intended binding-gate design (no single central classifier expected at launch).

No place made the sibling-only constraint want to bend — every cross-Task relationship is naturally Task-level (reverse deps to id-132, coordination notes to the rest).