Skip to content
Canonical

S312 — ID-61 / bl-45 / ID-45 reconcile (READ-ONLY analysis)

S312 — ID-61 / bl-45 / ID-45 reconcile (READ-ONLY analysis)

Section titled “S312 — ID-61 / bl-45 / ID-45 reconcile (READ-ONLY analysis)”

Provenance: S312 (2026-06-04). Read-only tee-up analysis — no code, no commits, no ledger writes. Answers three questions from Liam: (1) scope + decompose ID-61 (T13 pre-launch ops + observability), separating already-done from still-needed and PRE- vs POST-reingest; (2) bl-45 progressability; (3) what gates ID-45/T7.

Inputs read: docs/reference/task-list.json (ID-61, ID-45, deps), product-backlog.json (bl-45 + fold-ins 165/166/169/176/70), v1-completion-sequence.md, cocoindex-write-model.md, 07-collapse-list.md, docs/runbooks/onprem-b1-deploy.md, two-stage-re-ingestion-runbook.md, S279/S299 continuation prompts. “Already-shipped” claims grounded via gitnexus (detectStalls/dispatchAlerts in deploy/onprem/monitor/datapath-watch.ts; app/api/cron/datapath-watch/route.ts; recordPipelineRun in lib/pipeline/record-run.ts)

  • direct file reads.

⚠️ bl-45 ID-COLLISION — read this first

Section titled “⚠️ bl-45 ID-COLLISION — read this first”

The jq query select((.id|tostring)=="45") against product-backlog.json returns the WRONG item for the intent in the brief. Backlog id:"45" is a database tech-debt item:

bl-45 — “Convert remaining SECDEF functions to SECURITY INVOKER + RLS” (type:tech_debt, status:ready, priority:high, track:database, ~70 min). 35/39 functions already shipped (batches S22–S34); 4 remain, of which only 2 actually flip (detect_reupload, find_exact_duplicates → INVOKER; the other 2 are documented keep-SECDEF). Single batch-5 migration + a comment-hygiene strike at lib/mcp/tools/content.ts:416-418. Nothing to do with the canonical pipeline / observability / re-ingest.

The backlog has exactly one id:"45" (verified: length == 1), and its top-level array is .items[] (not .backlog_items[]). So the “bl-45” in the brief’s framing (implied pipeline-adjacent) does not exist under that id. Section (b) below answers the literal item that the id resolves to, and flags the mismatch for Liam to re-point if a different backlog id was intended.

(a) ID-61 — proposed subtask decomposition

Section titled “(a) ID-61 — proposed subtask decomposition”

Task today: ID-61 “Pre-launch operational pre-decisions + observability (T13)”, status:spec_needed, priority:should, zero subtasks, deps [28,37,44] (all done). Effort estimate 1.5 PLAN units. The MCP per-tool audit was already lifted out to ID-71.

Named scope in the task description = three code sub-parts + a set of S299 fold-ins: (a) prompt-cache spike (T13.1 — live cost gap in built extractors); (b) tracked-changes DOCX regression; (c) bid→procurement terminology rename audit; fold-ins bl-165 (Pydantic error-class persist, Option B+D), bl-166 (PipelineRunStatus cast — already closed S309 via bl-224), bl-169 (error_class runbook Cloud Run→Coolify), bl-176 (cocoindex operation-value case align), bl-70 (bids.ts defer-rename).

Honest verdict on “observability”: the brief’s hypothesis is correct — observability is substantially SHIPPED; the residual is operational pre-decisions + a couple of small persistence/audit gaps, not new monitors. Evidence (all grounded):

  • {66.15} datapath monitor — ARMED. Vercel cron app/api/cron/datapath-watch/route.ts, registered in vercel.json at */15 * * * *; pure detectStalls predicate (conditions A/B/C) reused verbatim from the tested deploy/onprem/monitor/datapath-watch.ts; alert sink decided (S311) = Sentry fallback via logger.error bridge (MONITOR_ALERT_WEBHOOK_URL deliberately unset). Thresholds have defaults (STALL_THRESHOLD_MINUTES=30).
  • recordPipelineRun (lib/pipeline/record-run.ts) already Sentry.captureMessage-alerts on failed + completed_with_errors (verified in source).
  • Kill-switch — runbook B3 documents it (Coolify control stop / UI / scheduled-task disable). Crash-loop backstop documented same section.
  • Migration #106 widened pipeline_runs_status_check (in_progress/cancelled) — live both envs.
  • ID-83 / bl-221 boot-never-walks burn gate + bearer-gated POST /walk — DONE S311.

So most “observability” ID-61 rows are already-covered; the genuine ID-61 work is (i) the 3 named code sub-parts, (ii) two small persistence/audit fold-ins, (iii) the operational pre-decisions that are still open as runbook OQs.

Proposed subtask table

Section titled “Proposed subtask table”
#Proposed subtaskPRE-reingest?Already-covered-byOne-line rationale
61.1Prompt-cache passthrough for cocoindex extractors (T13.1) — wire cache_control: ephemeral into the Path-A/Path-B extractor LLM callsYApp draft path done (lib/ai/draft.ts ×4, classify.py:745); gap is real — scripts/cocoindex_pipeline/prompts.py:27 explicitly defers cache passthrough to “Wave 4”Live cost gap on every full-corpus run; cheapest to land before T7 burns the whole corpus at full token price
61.2Tracked-changes DOCX regression test — fixture-based, asserts w:ins/w:del handling doesn’t leak revision markup into extracted markdownYPartial: form_extractors/docx.py:50 exercises one acceptance fixturePhew corpus is .docx; a tracked-changes regression would corrupt content on T7 — must be proven before the real run
61.3bid→procurement terminology rename audit — inventory + decide which of the ~134 bid* TS refs rename now vs stay (form_type-gated)NCollapse-list §3 has the ratified rename map; ~134 live refs remainCosmetic/clarity; renames are mechanical and carry no re-ingest data risk → safe to follow cutover (avoids churning the spine)
61.4bl-165 — Pydantic error-class persistence (Option B+D) — wire classify_pydantic_error → pipeline_runs.result; PII-redaction half already LANDED S299Y (low cost)Option-D PII redaction already in flow.py; Option-B persist NOT wiredMakes the first T7 run’s failures classifiable in the ledger rather than opaque — worth having for the headline run; ~45-60 min
61.5bl-169 — error_class / monitoring runbook re-target Cloud Run → Coolify (doc-only)NSuperseded framing only; {66.15}/{64.8} are the live surfacesDocumentation hygiene; the live monitor already exists, so this is a post-hoc doc correction
61.6bl-176 — cocoindex operation-value vocabulary case-align (dormant)Nneeds_research; no live consumer breaks todayDormant naming nit; resolve opportunistically, never gates a run
61.7bl-70 — bids.ts / bid-writing.md defer-rename (form_type-shape gated)Nneeds_research; gated on form_type probeBlocked on a shape probe; rolls up under 61.3’s audit; post-cutover
61.8Operational pre-decisions sign-off (register, not code) — ratify: (a) /walk cadence (OQ-1, e.g. nightly 0 2 * * *); (b) datapath-watch */15 + threshold defaults accepted; (c) alert routing = Sentry-only confirmed; (d) kill-switch drill + crash-loop backstop reviewed once liveY (decision) / N (drill)Cadence = OQ-1 OPEN; thresholds/alert-sink already DECIDED S311; kill-switch documented (runbook B3)These are the actual “pre-decisions” T13 names; most are decided — the open one is /walk cadence + a one-time go-live drill. Cheap, but the cadence decision should precede T7 so the first run is scheduled, not ad-hoc

fold-in already CLOSED (do NOT re-create): bl-166 PipelineRunStatus cast — closed S309 via bl-224 (PipelineRunRow/PipelineRunRowSchema now admit in_progress). The task description still lists it; spec-time should strike it, not subtask it. Any residual as PipelineRunStatus narrowing cast (if one survives) is a sub-minute hygiene line, not a subtask.

already-covered, do NOT build as ID-61 monitors: datapath stall monitor ({66.15}); failed/completed_with_errors Sentry alerting (recordPipelineRun); kill-switch + crash-loop backstop (runbook B3); pipeline-runs UI panel (components/intelligence/pipeline-runs-panel.tsx); coverage-alerts cron. The pipeline_failures table and Q4.12 cost-dashboards are [RATIFIED-DO-NOT-BUILD] (collapse-list §5.4) — explicitly out of scope.

Count + PRE-reingest split

Section titled “Count + PRE-reingest split”
  • 8 proposed subtasks (61.1–61.8).
  • PRE-reingest (must land before ID-45 T7): 4 → 61.1 (prompt-cache cost), 61.2 (tracked-changes DOCX correctness), 61.4 (error-class persistence for the headline run), 61.8 (the cadence pre-decision; the kill-switch drill half is post-go-live).
  • POST / can-follow: 4 → 61.3 (terminology audit), 61.5 (doc re-target), 61.6 (dormant case-align), 61.7 (defer-rename, blocked on probe).
  • Strike, don’t subtask: 1 → bl-166 (already closed S309).

Strictly, only 61.2 (DOCX correctness) is a hard blocker of a clean T7 — a tracked-changes leak would corrupt content. 61.1 and 61.4 are “strongly want before the expensive headline run” (cost + observability), and 61.8(a) is a cheap decision. 61.3/61.5/61.6/61.7 carry zero re-ingest data risk.

(b) bl-45 — verdict

Section titled “(b) bl-45 — verdict”

Item resolved by the id: bl-45 = “Convert remaining SECDEF functions to SECURITY INVOKER

  • RLS” (database tech-debt, NOT pipeline).

Verdict: PROGRESSABLE NOW — unblocked, ready, ~70 min.

  • status:ready, dependencies:[], all research complete (full analysis cited in the item: .planning/.research/s37-housekeeping/ops-43-1-best-practice-investigation.md).
  • Concrete close-out path is already specified in the item: one batch-5 migration — ALTER FUNCTION public.detect_reupload(text, uuid, text) SECURITY INVOKER; + ALTER FUNCTION public.find_exact_duplicates(text, uuid) SECURITY INVOKER; (both have qual=true SELECT policies so no RLS-policy change needed) + comment hygiene at lib/mcp/tools/content.ts:416-418 + regression via the two existing tests (__tests__/api/source-documents.test.ts, __tests__/api/items-batch.test.ts:862).
  • The other 2 of the 4 remaining functions (cleanup_filtered_articles, get_user_role) are documented keep-SECDEF end-states (flip is a no-op / would break RLS recursion) — so the item is genuinely a 2-function flip.

Only caveat (not a blocker): detect_reupload is itself listed [RATIFIED-RETIRE] in the collapse-list (§3.3, cocoindex source-key supersedes it). If retirement lands first the flip on that one function is moot — but find_exact_duplicates still flips, and retirement is not scheduled pre-cutover, so the item stands as ready today. Worth a one-line note when picked up.

⚠️ If Liam meant a different “bl-45” (a pipeline/observability item), it is not in product-backlog.json under id 45 — re-point needed. Nothing pipeline-adjacent resolves to that id.

(c) ID-45 (T7 full-corpus reingest) — gate summary

Section titled “(c) ID-45 (T7 full-corpus reingest) — gate summary”

ID-45 = “T7 — Phew full-corpus reingest via cocoindex (all content types)”, status:spec_needed, priority:high, deps [28,41,36,69]. Deps 28/41/36 done; dep 69 (ID-69 client-corpus / AI-readiness onboarding) is in_progress — a live blocker on the formal dependency edge.

What must precede T7 (the cutover SPINE, per v1-completion-sequence.md §1/§3):

  1. ID-80 Path-B form-write productionise + live-smoke (in_progress) — the one remaining write-path gate; a form file must ingest cleanly (watch form_type→form_types.key CHECK + form_template_fields uuid5 integrity).
  2. Non-prod target wiring (OQ-64-8) — a fresh Supabase Preview branch wired into the Coolify cocoindex env at re-ingest time (Task 82 proved the git-integrated preview-branch migration replay). Note cocoindex-write-model.md §5: the on-prem app is wired to PROD today; staging Coolify is the non-prod path.
  3. {64.7} retention export/re-link (pending) — needs Liam’s data-retention list; resolves OQ-64-7 (workspaces) + OQ-64-9 (content_history).
  4. {64.8} readiness surface + types regen + cutover runbook (pending, LAST in ID-64) — depends on all 64.x; the cutover gen types is the clean regen window.
  5. (cleanest-window, soft) Lane-A schema cleanup — bl-189 ({64.12}/{64.13}, drop 5 telemetry + 4 orphan content_items cols) should land before the Preview-branch wiring so the fresh corpus writes into the cleaned schema (zero backfill). Not a hard blocker.
  6. Two-stage runbook prerequisites (docs/runbooks/two-stage-re-ingestion-runbook.md) — reference-data seed step ({82.4}) REQUIRED on every fresh branch (else guides FK-violates), migration squash, DOCX pre-processing checklist, source artefacts staged. Plus the known ingest-root reconcile (docs/client-documentation-base→-corpus, 0 vs 33 files — an ID-45 subtask per S279).

One-liner: ID-45/T7 is gated on the cutover spine finishing first — ID-80 Path-B smoke → a fresh Preview-branch non-prod target → {64.7} retention + {64.8} readiness/types/runbook (plus dep ID-69 closing) — so ID-61’s PRE-reingest items (61.2 DOCX correctness, 61.1 prompt-cache cost, 61.4 error-class persist, 61.8a cadence) must be ordered into that pre-T7 window, ahead of S5.

Ordering recommendation (where ID-61 PRE items slot)

Section titled “Ordering recommendation (where ID-61 PRE items slot)”
ID-80 Path-B smoke (S1) ──┐
bl-189 schema cleanup ────┤  (cleanest BEFORE Preview-branch)
61.1 prompt-cache  ───────┤  PRE-reingest (cost)
61.2 DOCX regression ─────┤  PRE-reingest (correctness — hard blocker)
61.4 error-class persist ─┤  PRE-reingest (observability for the run)
61.8a /walk cadence ──────┘  PRE-reingest (decision)
        │
        ▼
Preview-branch target (S2) → {64.7} retention (S3) → {64.8} readiness/types (S4)
        │
        ▼
ID-45 / T7 full-corpus re-ingest (S5)  →  cut over to prod
        │
        ▼  (post-cutover, no re-run cost)
61.3 terminology · 61.5 doc · 61.6 case-align · 61.7 defer-rename · 61.8b kill-switch drill

End of analysis.