{56.17} — Curator handoff packet: amend canonical-pipeline-sequencing.md §2.5

{56.17} — Curator handoff packet: amend canonical-pipeline-sequencing.md §2.5

Status: HANDOFF — produced by the ID-56 heading-cols sub-orchestrator (worker id56-heading, S286). The parent’s Curator applies this edit to the canonical sequencing doc on the appropriate track branch. This packet is the doc/handoff artefact; the sub-orchestrator does NOT edit the sequencing doc directly (it lives outside this lane’s ownership and the §2.5 bullet itself names {56.17} as the formal handoff vehicle).

Target

• File: docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md
• Section: §2.5 “Why not the obvious CocoIndex ops” → the Text/chunking bullet (currently lines 212-224 at the time of authoring; the Curator should match on the verbatim text below rather than line numbers, which may drift).

Why this amend

§2.5’s Text/chunking bullet records the chunking stage as ratified but impl PENDING and explicitly defers the formal handoff to “{56.17}, not this doc”. Two outcomes now make the bullet stale:

1. {56.8} is BUILT. The cocoindex chunking stage was implemented and re-landed (post-ID-63 recovery) on the chunk lane — cmux-worker-id56-chunk-e387975c worktree id56-chunk @ commit 3d5870ed (“feat(cocoindex): ID-56.8 re-land — chunking stage in flow.py (post-ID-63 recovery)”). The @coco.fn chunking stage now lives inside ingest_file at scripts/cocoindex_pipeline/flow.py:1294-1341: bare RecursiveSplitter() budget-split (flow.py:1306) on the Variant-B byte budgets (2000/200/1000, flow.py:892-894), writing one content_chunks row per chunk via cc_target.declare_row (flow.py:1315-1335). The original build was 574d0a0f (“budget-driven chunking stage writes content_chunks”); ID-63 reverted it; 3d5870ed re-landed the same budget-split design (no AST/language= arg — confirmed by git log -S 'language=' -- flow.py returning empty across all branches).
2. [GAP-CMI-004] heading-cols disposition is RESOLVED-(a). Liam ratified disposition (a) keep nullable legacy directly in session S286 (OQ decision oq-c1a83ca097293fb6); recorded in PRODUCT.md [GAP-CMI-004] block and audited in docs/specs/id-56-content-model-invariants/heading-cols-disposition-RESEARCH.md ({56.14}, Checker PASS_WITH_NOTES). The four heading-derived columns (heading_text / heading_level / heading_path / parent_chunk_id) stay in the schema, NULL/'{}' on every cocoindex-emitted chunk — matching the deliberate OMIT at flow.py:1329-1333. The AST heading-population route is NOT v1.1-deferred — it is escalated as a NEW spike Subtask (go/no-go) tracked as OQ oq-fedd4fc673da6349.

The edit — find / replace

FIND (verbatim — the current Text/chunking bullet):

- **Text/chunking — chunking stage RATIFIED (reverse-split), impl pending (ID-56).** The
  earlier "no chunking stage by design / document-granular only" framing is **superseded**.
  ID-56 OQ-CMI-56-1(c) reverse-split is ratified S276 and the PLAN ratified S277: cocoindex
  becomes the **sole writer of `content_chunks`** via a NEW budget-driven chunking stage using
  **`cocoindex.ops.text.RecursiveSplitter`** (NOT `cocoindex.functions.SplitRecursively`, which
  is absent in cocoindex 1.0.3); the app-side chunker (`lib/content/chunking.ts` +
  `chunk-store.ts` + 8 callers) retires. Whole-document `content_items` remains
  document-granular; chunk-granular retrieval lives at `content_chunks`. **{56.8} chunking-stage
  build still PENDING (S281)** — the spike ({56.5}) ratified **Variant-B** (2000B/200B/1000B,
  `RecursiveSplitter`, `TextPosition` char_offset) and {56.6} landed `content_chunks.op_id`, but
  the `@coco.fn` chunking stage inside `ingest_file` is not yet built; **re-ingest waits on it**.
  Cross-ref `docs/specs/id-56-content-model-invariants/{PRODUCT,TECH,PLAN}.md`.
  (Formal Curator handoff packet is ID-56's {56.17}, not this doc.)

REPLACE WITH:

- **Text/chunking — chunking stage RATIFIED (reverse-split) and BUILT (ID-56).** The
  earlier "no chunking stage by design / document-granular only" framing is **superseded**.
  ID-56 OQ-CMI-56-1(c) reverse-split is ratified S276 and the PLAN ratified S277: cocoindex
  becomes the **sole writer of `content_chunks`** via a budget-driven chunking stage using
  **`cocoindex.ops.text.RecursiveSplitter`** (NOT `cocoindex.functions.SplitRecursively`, which
  is absent in cocoindex 1.0.3); the app-side chunker (`lib/content/chunking.ts` +
  `chunk-store.ts` + 8 callers) retires. Whole-document `content_items` remains
  document-granular; chunk-granular retrieval lives at `content_chunks`. **{56.8} chunking-stage
  BUILT (S286 re-land, post-ID-63 recovery)** — the spike ({56.5}) ratified **Variant-B**
  (2000B/200B/1000B, `RecursiveSplitter`, `TextPosition` char_offset), {56.6} landed
  `content_chunks.op_id`, and the `@coco.fn` chunking stage now lives inside `ingest_file`
  (`scripts/cocoindex_pipeline/flow.py:1294-1341`; bare `RecursiveSplitter()` budget-split,
  no `language=`/AST arg — original build `574d0a0f`, reverted by ID-63, re-landed
  `3d5870ed`). The build blocker is cleared; corpus re-ingest is now a downstream
  operational step rather than waiting on the stage. **Heading-derived columns —
  [GAP-CMI-004] RESOLVED-(a) keep-nullable (Liam S286, OQ `oq-c1a83ca097293fb6`):**
  `heading_text` / `heading_level` / `heading_path` / `parent_chunk_id` stay in the schema
  but are NULL/`'{}'` on every emitted chunk (budget-split preserves no heading boundary;
  deliberate OMIT at `flow.py:1329-1333`). Typed-consumer safety (the `search_content_chunks`
  RPC Returns nullability — TECH Migration 2 — plus the {56.10} consumer fix-up) is owned by
  those Subtasks, not the chunking stage. An **AST heading-population route is under a NEW
  go/no-go spike Subtask** (NOT v1.1-deferred; OQ `oq-fedd4fc673da6349`) — retaining the
  columns under (a) is its precondition. Cross-ref
  `docs/specs/id-56-content-model-invariants/{PRODUCT,TECH,PLAN}.md` +
  `heading-cols-disposition-RESEARCH.md`.
  (This bullet reflects the applied {56.17} handoff packet, S286.)

Curator apply checklist

1. Confirm the FIND block still matches verbatim in §2.5 (line numbers may have drifted; match on text). If it has changed since S286, reconcile rather than blind-replace.
2. Apply the replacement on the correct track branch (the canonical sequencing doc is not in this lane’s worktree ownership — the Curator applies it where the doc lives).
3. Sanity-check internal cross-refs resolve: flow.py path, the spec dir artefacts (PRODUCT.md / TECH.md / PLAN.md / heading-cols-disposition-RESEARCH.md), and the two OQ ids.
4. Optionally bump the sequencing doc’s own provenance/last-updated marker per its convention.
5. No ledger write is implied by this packet beyond what the parent already owns; the spike Subtask creation is tracked separately via OQ oq-fedd4fc673da6349.

Provenance

• {56.14} RESEARCH-spec: docs/specs/id-56-content-model-invariants/heading-cols-disposition-RESEARCH.md (commits d60a3974 + 8e0da2c3; Checker PASS_WITH_NOTES, notes fixed).
• {56.15}(a) closing-ack: PRODUCT.md [GAP-CMI-004] RESOLVED-(a) (commit 7d691f0d; Checker PASS).
• Ratification: Liam direct in-session S286 → OQ decision oq-c1a83ca097293fb6 (decider liam).
• AST spike escalation: OQ oq-fedd4fc673da6349 (non-blocking, parent ledger action requested).
• As-built source: id56-chunk @ 3d5870ed, scripts/cocoindex_pipeline/flow.py:1294-1341.