Phase 0.2.5 — Build-not-wired investigation
Phase 0.2.5 — Build-not-wired investigation
Section titled “Phase 0.2.5 — Build-not-wired investigation”Audit date: 2026-05-06
Branch: content-items-investigation (audit on main against current code; same commit base as Phase 0.1 reports — verified report dates 2026-05-06)
Scope: Verify Phase 0.1 “build-not-wired” seeds; broader scan for parsed-but-unused / never-overridden / never-hit patterns. Cited code is on main HEAD as of audit date.
Confidence reminder: anything <90% is moved to Open Questions. No fixes — documentation only.
Confirmed seed findings
Section titled “Confirmed seed findings”Finding B-1: EP2 markdown-batch — auto_supersede parsed-but-unused
Section titled “Finding B-1: EP2 markdown-batch — auto_supersede parsed-but-unused”- Source seed: Phase 0.1
0.1-ts-ep2-markdown-batch.md§“Per-batch admin overrides”, Drift §3 #7, Open question #6. - Verified: YES.
- Schema:
lib/ingest/markdown-batch-schema.ts:39definesauto_supersede: z.boolean().optional()insideBatchWideOptionsSchema(.strict()). - Route:
app/api/ingest/markdown/route.ts:260runsparseBody(BatchOptionsSchema, parsedOptions)and forwards options verbatim into the orchestrator. - Consumer: orchestrator
lib/ingest/markdown-orchestrator.ts— a grep forauto_supersede,autoSupersede, orsetSupersessionreturns ZERO matches in the orchestrator and ZERO matches anywhere underlib/ingest/except the schema declaration itself + the route comment forwarding it. - Net: caller can pass
{batch: {auto_supersede: true}}admin-only, the route accepts it, the orchestrator never reads it. The result envelope (results_summary.superseded[]) is unconditionally empty (orchestrator line ~342, per 0.1 report).
- Schema:
- Severity: P2 (hygiene — neither breaks ingest nor leaves a data gap; admins are silently misled into thinking auto-supersede is wired).
- Fix options (do not act on these — recorded only):
- Reject the field at schema level (
z.never()or strip) — least-effort path until §1.18 Python-parity follow-on lands. - Wire it: orchestrator imports
setSupersession(from EP8 sisterkb_pipeline/supersede.py— TS port needed), invoked when filename heuristic detects draft↔final pair AND admin caller AND flag set.
- Reject the field at schema level (
- Re-ingest impact: none (auto-supersede operates on prior rows; if not wired, nothing supersedes, but ingest itself completes correctly).
Finding B-2: EP2 markdown-batch — tag parsed-but-unused
Section titled “Finding B-2: EP2 markdown-batch — tag parsed-but-unused”- Source seed: Phase 0.1
0.1-ts-ep2-markdown-batch.mdcontent_items field map row “user_tags”, Drift §3 #7. - Verified: YES.
- Schema:
lib/ingest/markdown-batch-schema.ts:41definestag: z.string().optional()(“Mirror of Python —tag”). - Orchestrator: grep for
\btag\binlib/ingest/markdown-orchestrator.tsreturns three hits — line 81cleanMdxTagsimport, line 211 comment “Clean MDX tags”, line 861tags: { pipeline: PIPELINE_NAME, status }(Sentry tags). NONE consumeoptions.batch.tag. - The Python parity expectation is that
--tagwould append touser_tags(per 0.1 report row), but the TS path leavesuser_tagsNOT SET on EP2 inserts.
- Schema:
- Severity: P2 (hygiene; same class as B-1).
- Fix options:
- Drop from schema until parity work lands.
- Wire it: orchestrator’s INSERT payload (line 635–648) adds
user_tags: tag ? [tag] : undefined(admin-or-editor; no role gate per Python).
- Re-ingest impact: none.
Finding B-3: EP2 markdown-batch — layer never set on TS path
Section titled “Finding B-3: EP2 markdown-batch — layer never set on TS path”- Source seed: Phase 0.1
0.1-ts-ep2-markdown-batch.mdcontent_items Governance row, Drift §3 #3, Open question #5. - Verified: YES.
data-entry-points.md§12 Quick Comparison Matrix claims “Layer inference: Yes” for EP2.- Reality: orchestrator never invokes
inferLayer.classifyContentdoes NOT writelayer(verifiedclassify.ts:1426–1438UPDATE shape — it sets primary/secondary domain, ai_keywords, summary, suggested_title, classification_confidence, classification_reasoning, classified_at, updated_by, conditional embedding/metadata. NOlayer.) - Comparable TS paths that DO write layer:
app/api/items/batch/route.ts:369–372,app/api/upload/route.ts:824–827,app/api/ingest/url/route.ts:381–384,app/api/items/route.ts(via post-create flow). EP2 is the outlier.
- Severity: P1 (degraded behaviour — every markdown-batch row leaves
layer=NULL, which breaks downstream layer-aware filtering / dashboards / reorient flow). - Fix: Add a post-classify
inferLayerUPDATE to the orchestrator’simportOneFile(mirror EP3 file-upload pattern atroute.ts:809–832but withingestionSource:'upload'). Same enum domain —inferLayeralready accepts'upload'. - Re-ingest impact: YES — re-ingest fixes the gap for every markdown-batch row. Pre-launch this is preferable to a backfill script.
Finding B-4: EP2 markdown-batch — embedding double-write
Section titled “Finding B-4: EP2 markdown-batch — embedding double-write”- Source seed: Phase 0.1
0.1-ts-ep2-markdown-batch.mdEmbedding row, Drift §3 #2, Open question #4. - Verified: YES.
- First write:
classifyContentregenerates the embedding (post-classification) using${suggested_title}\n\n${plainText}and UPDATEscontent_items.embedding(classify.ts:1452–1470). - Second write: orchestrator
importOneFilethen computes${titleResult.title}\n\n${cleanedBody}and UPDATEscontent_items.embeddingagain (markdown-orchestrator.ts:668–678). - Both writes succeed; the orchestrator’s overwrites the classifier’s. Different prefix (
suggested_titlevstitle) means the LRU cache (embed.ts:65–88, 500 entries × 1h) does NOT short-circuit — second write incurs a fresh OpenAI call.
- First write:
- Severity: P2 (cost + risk — extra OpenAI call per file; mid-write failure leaves the classifier-derived embedding in place which is acceptable but undocumented).
- Fix: Remove the orchestrator’s redundant embedding step (lines 668–691). The classifier already writes the embedding inside
classifyContent; the orchestrator step is leftover from pre-classify-regen behaviour. EP3 file-upload has the same pattern (route line 666–680 then classify regen overwrites) — same fix applies. - Re-ingest impact: none (data identical; only cost differs).
Finding B-5: EP2 markdown-batch — maxDuration=60 vs doc-claimed 300
Section titled “Finding B-5: EP2 markdown-batch — maxDuration=60 vs doc-claimed 300”- Source seed: Phase 0.1
0.1-ts-ep2-markdown-batch.mdDrift §3 #4. - Verified: YES.
- Code:
app/api/ingest/markdown/route.ts:86:export const maxDuration = 60;. - Doc:
docs/reference/data-entry-points.md§12 line ~1015 (per 0.1 report) claimsmaxDuration=300. - The 60s value is correct per S226 D-7 ratification; doc is stale.
- Code:
- Severity: P2 (doc drift; functional behaviour correct).
- Fix: Update §12 of
data-entry-points.mdto reflectmaxDuration=60and the queue-worker handoff post-S226 (orchestrator runs in cron workerlib/queue/dispatch.ts:355–418, not inline). - Re-ingest impact: none.
Finding B-6: Batch path Q&A — inferLayer ingestionSource forced to 'upload'
Section titled “Finding B-6: Batch path Q&A — inferLayer ingestionSource forced to 'upload'”- Source seed: Phase 0.1
0.1-ts-batch-creation.md§10 Open question Q4 + content_items field map row “layer”. - Verified: YES.
app/api/items/batch/route.ts:354–367:inferLayer({contentType:'q_a_pair', contentLength: ..., ingestionSource:'upload', hasBrief/Detail/Reference:false, isBidDiscovered:false, title:item.title}).lib/layer-inference.ts:81–211: Rule 2 (the “bid Q&A → bid_detail” fast-path) matches ONLY wheningestionSource === 'bid_library' && contentType === 'q_a_pair'. WithingestionSource:'upload', Rule 2 NEVER fires for batch-route Q&A items.- Falls through to Rule 5 — content-length heuristic. Q&A under 500 chars →
sales_brief(low confidence); Q&A ≥500 chars →bid_detail(low confidence). All inferred atconfidence: 'low'. - The batch path is the dominant source of new Q&A pairs from the UI tab (
upload-tab-content.tsx:439); writes ~0 prod rows today (per 0.1 report —'upload_autosplit'ingest_source, distinct from 440 production CLI-imported Q&As). - Compare:
scripts/import_bid_library.py:300(Python Q&A path) DOES passingestion_source='bid_library'→ Rule 2 fires →bid_detailhigh confidence.
- Severity: P1 (degraded — Q&A items via batch path get a low-confidence layer suggestion instead of the canonical
bid_detailRule 2 high-confidence verdict). - Fix options:
- Add a literal
'bid_library'value toLayerInferenceInput.ingestionSourceenum (already accepts it — no change needed) and pass'bid_library'from the batch route. Cleanest. - Map
'upload_autosplit'→'bid_library'in the inference call (route layer translates). - Add Rule 2-equivalent rule to
inferLayermatchingingestionSource === 'upload'ANDcontentType === 'q_a_pair'. Riskier — would change behaviour for non-Q&A-batch upload paths.
- Add a literal
- Re-ingest impact: YES — re-ingest of any batch-created Q&A items lifts them from
sales_brief(low) tobid_detail(high). Pre-launch volume on this path is small but non-zero (per 0.1 report).
Finding B-7: Q&A docx CLI — brief/detail/reference written by Path 3 only (data-entry-points.md §8 missing)
Section titled “Finding B-7: Q&A docx CLI — brief/detail/reference written by Path 3 only (data-entry-points.md §8 missing)”- Source seed: Phase 0.1
0.1-qa-docx-import.md§3 + §8 reference doc drift; Phase 0.1 brief mentions data-entry-points.md §8 missing this behaviour. - Verified: YES.
- Writer:
scripts/kb_pipeline/post_insert.py:334(update_content_item(item_id, depth_result)) called fromprogressive_depth.generate_progressive_depth()(scripts/kb_pipeline/progressive_depth.py:204). - Gating: only when caller passes
generate_progressive_depth_flag=True. The ONLY caller that passes True isscripts/import_bid_library.py:736. Python URL ingest (scripts/ingest.py→kb_pipeline/pipeline.py) and Python markdown ingest both default to False → never write brief/detail/reference (per 0.1 report Path 3 §3 confirmation). - AI primary path:
claude-haiku-4-5; deterministic fallback (first-paragraph-of-answer / answer_standard+answer_advanced concatenation / question_text). Brief/detail/reference are STORED back tocontent_itemsvia UPDATE post-insert. data-entry-points.md§8 Quick Comparison Matrix says “AI summary: No (truncated answer only)” but does NOT list the progressive-depth Anthropic call OR the brief/detail/reference writes. Matrix Content history row also says “No” but trigger does write v1 (per S207 WP-A4 / migration20260422060118).
- Writer:
- Severity: P2 (doc drift; the writes themselves are correct and intentional per spec; the gap is observability — anyone reading §8 underestimates the path’s behaviour).
- Fix: Update §8 to list:
- Progressive-depth Anthropic Haiku call (lines + cite).
- Brief/detail/reference UPDATE via
post_insert.py:334. - Content history v1 row written by DB trigger (not app).
- Q&A ingest_source =
'qa_import'(S207 WP-A4 promoted column). - Layer =
bid_detailfrom Rule 2 (infer_layerpre-insert).
- Re-ingest impact: none (writes are happening; doc gap only).
Finding B-8: Cron Python URL — Cloud Run Jobs run python3 scripts/ingest.py with NO URLs (smoke-test no-op)
Section titled “Finding B-8: Cron Python URL — Cloud Run Jobs run python3 scripts/ingest.py with NO URLs (smoke-test no-op)”- Source seed: Phase 0.1
0.1-python-url-ingest.mdVerdict + §“Entry point” lines 13–15. - Verified: YES.
- Build manifest:
cloudrun/cloudbuild.yaml:32:--env=GOOGLE_ENTRYPOINT=python3 scripts/ingest.pybaked at image build forkh-pipeline-slim. NO arguments after the script name. - Job manifest:
cloudrun/jobs/prod-phew.yaml(and the three sister manifestsprod-kpf,staging-phew,staging-kpf) declare ONLYimage:+env:(NEXT_PUBLIC_CLIENT_ID, KH_REQUEST_ID) — NOcommand:orargs:overrides for the container. - Script behaviour:
scripts/ingest.py:111–113:if not urls: parser.print_help(); sys.exit(1). With zero URLs and no--fileflag, the script prints help and exits 1 unconditionally. - Net: any Cloud Scheduler / GitHub Action / manual
gcloud run jobs executeinvocation runs the container, fails fast on no-URLs, exits 1. The 0.1 report confirms “No tested-end-to-end ingest cron has been wired (Phase 2 territory per §5.3)” and that the deploy runbook treats first-invocation as a smoke test.
- Build manifest:
- Severity: P1 (degraded — the path is provisioned, deployed, and image-published, but is incapable of doing useful work without args. “Build-not-wired” canonical example: image and IAM ready, business logic not).
- Fix: what production-readiness needs to wire. Options:
- URL feed source — Cloud Run Job reads URLs from a Cloud Storage object (pre-curated batch); job manifest adds
args: [--file, gs://kh-prod-ingest/urls.txt]and IAM role for the Storage object. - Pub/Sub subscriber — Cloud Scheduler publishes ingest events to Pub/Sub; Cloud Run Job (or Cloud Function trigger) consumes single-URL messages. Requires a wrapper script (
scripts/ingest_pubsub.py) that reads message body → callsprocess_url. - HTTP-triggered ingest service — replace Job with a Cloud Run Service that exposes a POST
/ingestendpoint; Cloud Scheduler hits the endpoint with a JSON URL payload. Diverges from “Job” model; more flexible. - Per-tenant URL list at deploy-time — treat each manifest as carrying its tenant’s URL set via
args:. Simplest but scales poorly (manifest churn on URL list change).
- URL feed source — Cloud Run Job reads URLs from a Cloud Storage object (pre-curated batch); job manifest adds
- Re-ingest impact: This finding is orthogonal to re-ingest. The PYTHON URL ingest CLI is fine for hand-driven runs (Liam’s primary trigger today). Cron readiness is a Phase 2 item per the 0.1 report’s read of
cloud-run-phase-1-handover.md. - Cross-link: production-readiness track is wiring proper cron — primer in
docs/tracks/production-readiness.mdand runbookdocs/runbooks/cloud-run-phase-1-handover.md.
Finding B-9: EP3 file upload — source_documents INSERT silently swallowed (try/catch + .data destructure-only)
Section titled “Finding B-9: EP3 file upload — source_documents INSERT silently swallowed (try/catch + .data destructure-only)”- Source seed: Phase 0.1
0.1-ts-file-upload.md§3 + §11 Q1. - Verified: YES, with a tighter failure mode than the seed flagged.
- Code:
app/api/upload/route.ts:410–447. The INSERT is wrapped intry { ... } catch (srcDocErr) { logger.error(...); }(line 444–447 — non-fatal). 0.1 report flagged this catch. - Additional finding (not in 0.1): the INSERT result is destructured with ONLY
data: sourceDocon line 417 — the supabase-jserrorfield is NEVER checked. supabase-js does NOT throw on REST errors; it returns{data: null, error: PostgrestError}. So a 4xx (e.g. constraint violation, RLS denial, schema mismatch) →data=null,error=present, the catch block does NOT fire (no exception thrown), andif (sourceDoc)at line 436 is false →sourceDocumentIdstays null silently with NOT EVEN AN ERROR LOG. - Net: TWO independent silent-failure modes — (a) thrown exception caught and only
logger.error’d, (b) PostgrestError on REST never caught at all and not even logged. - Knock-on: subsequent
if (sourceDocumentId)gates skip the source_documents UPDATEs (extracted_text fill, status=‘processed’), the diff/impact analysis (re-upload path), and thecontent_items.source_document_idFK link. Thecontent_itemsrow exists withsource_document_id=NULLand zero observability into why.
- Code:
- Severity: P0 (silent data gap — every successful upload is at risk of birthing an orphaned content_items row with no source_documents lineage; impossible to detect from logs alone unless caller stares at row count post-hoc).
- Fix: Two changes:
- Destructure
errorand check it:const { data: sourceDoc, error: srcDocError } = await .... Onerror,logger.error({err: srcDocError, op: 'upload.source_documents.insert'}, 'INSERT failed')and bubble. - Decide policy: should
source_documentsfailure be FATAL (rollback the content_items row, return 500) or BEST-EFFORT (current behaviour, but at least logged)? Spec silence — needs Liam call.
- Destructure
- Re-ingest impact: YES if there are existing content_items with
ingest_source='upload'andsource_document_id IS NULLdue to silent fail. Pre-launch row count audit (per 0.1 §11 Q1) would resolve. If counts are ~0 (path is shipped-but-unused in prod), the orphan risk is theoretical not realised.
Broader scan findings
Section titled “Broader scan findings”Pattern A — Zod parsed-but-unused
Section titled “Pattern A — Zod parsed-but-unused”| Finding | File:line | Severity |
|---|---|---|
BatchWideOptionsSchema.auto_supersede (Seed B-1 above) | lib/ingest/markdown-batch-schema.ts:39 | P2 |
BatchWideOptionsSchema.tag (Seed B-2 above) | lib/ingest/markdown-batch-schema.ts:41 | P2 |
No additional parsed-but-unused fields found in spot-checked routes (app/api/items/route.ts, app/api/items/batch/route.ts, app/api/upload/route.ts, app/api/ingest/url/route.ts, app/api/ingest/markdown/route.ts). Each destructured field is referenced downstream. Full sweep across all 87 parseBody callers not run (out-of-scope time-box).
Note for parent session: A future build-not-wired sweep could bun run knip against schema field types — but knip operates at file/export level not field level; field-level dead detection would require static analysis (e.g. ts-morph) walking the parsed shape against the symbol-table.
Pattern B — Recent migrations with no writer
Section titled “Pattern B — Recent migrations with no writer”| Migration | Column added | Writer status |
|---|---|---|
20260427103256_add_review_cadence_columns.sql | next_review_date (date) | WRITTEN — lib/governance/cadence-renewal.ts (compute), app/api/governance/review/route.ts:144 (apply on approve), lib/mcp/tools/governance.ts:1182 (apply on approve via MCP) |
20260427103256_add_review_cadence_columns.sql | review_cadence_days (integer) | NEVER WRITTEN — zero callers in app/, lib/, scripts/. Only readers (cron quality-score, governance review fetch, MCP governance fetch). Migration comment says “backfill ships in §5.4 / Plan T6” — backfill is the only planned writer and is not yet shipped. No app write surface (POST /api/items, POST /api/items/[id] PATCH, governance-review approve flow) exposes a review_cadence_days field. |
20260421172733_add_dedup_status_to_content_items.sql | dedup_status | WRITTEN by all six ingest paths (see 0.1 reports) |
20260421222059_add_superseded_by_to_content_items.sql | superseded_by | WRITTEN by scripts/kb_pipeline/supersede.py:140–145 (Q&A docx auto-supersede); not by TS paths |
20260427125412_add_publication_status_column.sql | publication_status | WRITTEN — manual creation, batch, EP2/EP3, MCP create |
20260428174512_add_ingest_source_to_content_items.sql | ingest_source | WRITTEN by all ingest paths (the 0.1 report set) |
20260502233917_s221_w1_queue_infra_d1_d2_d3.sql | processing_queue.idempotency_key | OUT-OF-SCOPE for this audit (queue infra column, not content_items) |
Pattern B finding 1 — review_cadence_days (P1 build-not-wired). The column exists, has a CHECK constraint (1–1095 days), is read by quality-score / governance-review routes, but has zero writers. Migration comment is honest (backfill ships in §5.4 / Plan T6), but no current path lets a user or admin SET a cadence on a row. Until §5.4 / T6 ships, every row’s review_cadence_days IS NULL, and cadence-renewal.ts:32 returns no update on approve (current next_review_date is preserved unchanged). Severity P1 — column is a no-op until backfill + an admin SET-cadence surface lands. Re-ingest impact: none (re-ingest doesn’t help; this needs an admin UI/API path or backfill SQL).
Pattern C — Default flags never overridden
Section titled “Pattern C — Default flags never overridden”| Helper | Default flag | Finding | Severity |
|---|---|---|---|
classifyContent({validate?: boolean}) | validate: false | validate: true is NEVER passed by any caller. Verified by `grep -rE “validate:\s*(true | false)” app lib across the repo — zero hits. Pass 2 entity validation (classify.ts:1567gate, thevalidateEntities()LLM call at line 971, thepayment_gateway_product_anchorPass 2 prompt augmentation, the validated_entities filter pipeline) is **dead code at runtime**. The Python—entitiesflag (Q&A docx CLI) does invoke the corresponding Python Pass 2 inclassify.py:721`, but the TS port has nobody enabling it. |
classifyContent({force: boolean}) | n/a (required) | All callers pass force: true (markdown-orchestrator line 663, items/route line 508, items/batch line 323, upload/route line 1117, items/[id]/route line 597). The if (force === false) { early-return-on-classified-at } branch at classify.ts:1115–1135 is never hit in production. Confirmed by grep — zero hits for force: false. The [id]/classify route DOES pass force from request body; that route is the one place a viewer can set false (per app/api/items/[id]/classify/route.ts:58). So the branch IS reachable from one user-facing surface; reclassified as P3 (live but obscure). | P3 |
run_post_insert(... infer_layer_flag=False, generate_progressive_depth_flag=False, ...) | both False | infer_layer_flag=False is the only value passed — Python URL/markdown/Q&A all set False on this flag (Q&A pre-sets layer in INSERT instead, per 0.1 report). The flag and its branch (post_insert.py:295–301) are unreachable. generate_progressive_depth_flag=True IS passed by Q&A only (import_bid_library.py:736). So this helper has dead-flag (infer_layer_flag) PLUS a flag with one caller (generate_progressive_depth_flag). | P2 — clean up the dead flag; document the single-caller flag in data-entry-points.md §8. |
checkForDuplicates(supabase, contentText, embedding?, options?) | embedding undefined skips near-dup branch | EP3 file-upload calls with embedding=undefined (route line 528–533) because dedup runs BEFORE embedding generation. Result: near-duplicate detection NEVER runs on upload path. EP4 URL ingest and EP3 manual creation both pass embeddingArray. Asymmetric coverage. | P2 — re-order EP3 upload to embed before dedup, OR document the asymmetry. |
Pattern D — Branches never hit
Section titled “Pattern D — Branches never hit”| Branch | Status | Severity |
|---|---|---|
classify.ts:1567 Pass 2 entity validation gate (if (params.validate && ...)) | NEVER HIT in production (Pattern C). Reachable in tests only. | P1 (code dead + sunk cost on Anthropic Opus 4.6 prompt + tests + maintenance) |
lib/dedup.ts:170-184 near-duplicate branch | UNREACHABLE on EP3 file upload (Pattern C); reachable on EP4 URL ingest, EP3 manual creation. | P2 (asymmetric) |
post_insert.py:295-301 infer_layer_flag branch | NEVER HIT in production (Pattern C). | P2 |
data-entry-points.md §12 claim “Layer inference: Yes” for EP2 | DOC ASSERTS A BRANCH THE CODE DOES NOT HAVE (Seed B-3). | P1 doc drift |
data-entry-points.md §3 step 14 ordering (“Embed → Dedup”) | DOC INVERTS THE ACTUAL ORDER (EP3 upload runs dedup first; per 0.1 report). Drift only — no live behaviour change. | P3 doc |
Severity summary
Section titled “Severity summary”(Counts include broader-scan findings.)
| Severity | Count | Tally |
|---|---|---|
| P0 (silent data gap) | 1 | B-9 |
| P1 (degraded behaviour) | 5 | B-3 (EP2 layer NULL), B-6 (batch Q&A wrong layer), B-8 (cron no-args no-op), Pattern B review_cadence_days no writer, Pattern D Pass 2 dead branch |
| P2 (hygiene / observability / cost) | 7 | B-1 (auto_supersede unused), B-2 (tag unused), B-4 (embedding double-write), B-5 (maxDuration doc drift), B-7 (data-entry-points §8 missing brief/detail/reference), Pattern C infer_layer_flag dead, Pattern C upload pre-embed dedup asymmetry |
| P3 (low) | 2 | Pattern C force:false only via [id]/classify, Pattern D §3 dedup-vs-embed step ordering |
Re-ingest readiness gate
Section titled “Re-ingest readiness gate”For each P0/P1, decision matrix on whether the gap blocks re-ingest:
| Finding | Severity | Must-fix-before-reingest? | Reasoning |
|---|---|---|---|
| B-9 EP3 source_documents silent fail | P0 | YES — strongly recommended. | Re-ingest will write more rows via this path. If the silent-fail mode is live in prod (currently 0 prod source_documents per 0.1 §11 Q1, but the path is exercised), every silent-failed upload = orphan row. Pre-launch is the cheapest moment to flip the catch to fail-loud. |
| B-3 EP2 markdown layer NULL | P1 | NO — but YES if you want layer-aware filters working post-reingest. | Re-ingest WITHOUT fix: every markdown row lands with layer=NULL again, identical to today. Re-ingest WITH fix (~30min change to orchestrator): every markdown row gets a correct layer suggestion. Liam’s call — depends whether layer-aware UI is planned for launch or post-launch. |
| B-6 batch Q&A wrong-layer | P1 | YES if you want Q&A items to land in bid_detail automatically. | Same logic as B-3 — re-ingest without fix preserves the low-confidence sales_brief / bid_detail length-derived suggestion. Fix is two-line: change ingestionSource:'upload' to 'bid_library' at app/api/items/batch/route.ts:361. Re-ingest then produces high-confidence Rule 2 hits. |
| B-8 cron no-args no-op | P1 | NO — orthogonal to re-ingest. | Production-readiness track wires the cron. Hand-driven python3 scripts/ingest.py <url> works today. Re-ingest of existing content uses CLI, not cron. |
Pattern B review_cadence_days | P1 | NO | Column has no writer regardless of re-ingest. Backfill ships in §5.4 Plan T6. Independent of re-ingest decision. |
| Pattern D Pass 2 dead branch | P1 | NO | Reachability is a code-cleanup decision, not an ingest decision. Either ship validate:true from one path (likely cron classification-quality) or delete the dead 600+ lines of Pass 2 code. |
| B-1 auto_supersede unused (P2) | P2 | NO | Hygiene only. Same as Pattern B — needs a writer, not a re-ingest. |
| B-2 tag unused (P2) | P2 | NO | Hygiene. |
| B-4 embedding double-write (P2) | P2 | NO — but a one-line fix saves one OpenAI call per markdown file at re-ingest. ROI tiny but positive if re-ingest >100 files. | |
| B-5 maxDuration doc drift (P2) | P2 | NO | Doc only. |
| B-7 §8 doc gap (P2) | P2 | NO | Doc only. |
Open questions for parent session
Section titled “Open questions for parent session”-
EP3 source_documents silent-fail policy — should the catch be made fatal (rollback content_items, 500), best-effort-with-loud-log, or remain best-effort silent? Spec silence per 0.1 §11 Q1. Cross-cuts with Phase 0.2.6 swallow-catch audit.
-
Cron URL ingest delivery model (B-8) — four options enumerated. Production-readiness track owns the choice; flag for cross-track sync.
-
Pass 2 entity validation (Pattern C / D) — keep + enable from cron
classification-quality(the obvious caller, already does targeted re-classify for low-confidence rows), or delete? Cost: each Pass 2 = 1 Opus 4.6 call ≈ 2k tokens. Quality: perdocs/audits/two-pass-cost-quality-measurement.md§7 (referenced in classify.ts comments), payment-gateway product type-flips dropped from Pass 1 stochastic to anchored-Pass-2 deterministic — but that gain is realised only when validate IS enabled. Ironic. -
review_cadence_dayswriter surface — when §5.4 Plan T6 backfills, is there an admin UI/API path planned to SET cadence per-item, or per-content-type policy? Worth confirming the column is not orphaned long-term. -
Q&A batch layer fix (B-6) — confidence on the
'bid_library'value flip — 88% (theinferLayerenum already accepts it; semantic intent matches perimport_bid_library.py:300; the only risk is downstream consumers readingmetadata.ingestion_sourceSTRING which would still be'upload_autosplit'). Below 90% so flagging — please confirm semantic intent before flipping. -
EP2 embedding double-write removal (B-4) — confidence on safe-to-remove: 90%. The classifier’s regenerated embedding (using
suggested_title) is canonically what we want. The orchestrator’s overwrite uses raw filename-derivedtitlewhich is generally lower quality. Removing the overwrite leaves the better embedding. But — the orchestrator runs AFTER classify, so a classify failure would leave NO embedding. The orchestrator’s overwrite is a safety net. Trade-off: keep the safety net (current state, double cost) or remove it (single cost, risk = no embedding on classify failure, recoverable via backfill scripts). Liam’s call. -
Are
0.2a/0.2boutputs available to cross-reference Pattern B (review_cadence_daysis the standout but I had no inventory to compare against)? If so, should this report’s Pattern B section be expanded?