Skip to content

ID-63 {63.2} PRODUCT — Ontology / controlled-vocabulary (CV) integration-correctness gate for re-ingest

ID-63 {63.2} PRODUCT — Ontology / controlled-vocabulary (CV) integration-correctness gate for re-ingest

Section titled “ID-63 {63.2} PRODUCT — Ontology / controlled-vocabulary (CV) integration-correctness gate for re-ingest”
  • Task: ID-63 — Ontology / controlled-vocabulary (CV) integration verification across the canonical pipeline.
  • Subtask: {63.2} PRODUCT (this document). Numbered, testable Behaviour invariants.
  • Status: PRODUCT — defines the desired behaviour + invariants. Does NOT prescribe implementation (that is {63.3} TECH).
  • Author: fresh task-planner instance (Q-PLANNER-2 — NOT the author of {63.1}), opus thinking:max, isolated worktree off main @ 11cd62e5.
  • Date: 29/05/2026.
  • Language: UK English throughout (colour, organise, behaviour, DD/MM/YYYY).
  • Predecessor: docs/specs/id-63-ontology-cv-integration/RESEARCH.md ({63.1}) — the authoritative finding-set. Read in full; this PRODUCT resolves OQ-63-1, OQ-63-1a, OQ-63-2, OQ-63-3, OQ-63-4, OQ-63-5, OQ-63-6 against the S279 Liam decisions and the empirical verification in §Verification below.

Reading guide. §Summary frames the gate. §Problem records the code-intelligence orientation + the integration-correctness gap. §Scope bounds the Task. §Behaviour is the spec — numbered invariants Inv-N, each acceptance-verifiable. §Form-type drop-vs-grow resolution settles OQ-63-1a. §Verification is the pre-ratification empirical block. §Carried notes records out-of-scope dispositions. §Open questions surfaces the one NEW question.


ID-63 is a pre-re-ingest integration-correctness gate, not a feature. Its consumers are (a) the operator who will run the full-corpus T7 re-ingest, and (b) the code/tests that must stay in lockstep so a re-ingest does not systematically mis-classify the corpus. The gate guarantees that the canonical re-ingest classifier (scripts/cocoindex_pipeline/extraction.py, driven by flow.py) emits only taxonomy-valid content_type, form_type, and (per the proposed Inv-7) primary_domain / primary_subtopic values, and that the controlled-vocabulary lockstep across markdown register ↔ live DB ↔ snapshot ↔ prompt is enforced by parity guards that assert canonical behaviour, not a frozen legacy implementation list. The headline defect this gate closes is a “test-green-but-integration-wrong” drift: the Q&A-form extraction prompt instructs the model to emit three form_type values the validator rejects, and a passing test locks that stale contract in place.


Code-intelligence orientation (cited verbatim — re-run for {63.2} per Q-PLANNER-2; not inherited from {63.1})

Section titled “Code-intelligence orientation (cited verbatim — re-run for {63.2} per Q-PLANNER-2; not inherited from {63.1})”

Per the .gitnexus/CLAUDE.md “Always Do” block, orientation was run before drafting. TS symbols via gitnexus (repo /Users/liamj/Documents/development/knowledge-hub); Python / markdown / SQL via grep (outside the TS index per .ast-dataflow/CLAUDE.md).

  • gitnexus_query({query: 'classifyContent'}) — top execution flows are proc_245_get (app/api/cron/classification-quality/route.ts:GET 43-330) and proc_29_post (app/api/items/[id]/classify/route.ts:POST 27-125). Definitions returned: Function:lib/ai/classify.ts:classifyContent (1104-1832, module Ai), Function:lib/ai/classify.ts:coerceSubtopic (698-703), Interface:lib/ai/classify.ts:ClassificationResult (705-727). This is the app-side classify path, NOT the re-ingest path.
  • gitnexus_query({query: 'controlled vocabulary'})processes returned the layer-schema / draft-stream / classify-quality flows (lib/validation/layer-schemas.ts, app/api/layers/*); no single indexed “CV-loading” execution flow surfaced. CV loading is distributed across build-time TS scripts, a Pydantic module-load read (Python), and live-DB queries (TS classify).
  • gitnexus_query({query: 'ontology'})processes: []. Definitions: Function:lib/ontology/loader.ts:loadOntologyCVs (42-75), Function:lib/ontology/schemas.ts:OntologyCVSchema (105-137), Function:scripts/generate-content-type-values.ts:main (45-84), Function:scripts/ontology-sync/parse-flow.py:{process_ontology_file,parse_cv_frontmatter,validate_cv_against_yaml}, Class:scripts/tests/test_cocoindex_extraction.py:TestFormTypeParity (738-788). No TS execution flow runs through the ontology loader — confirming the markdown register and the runtime classifier are decoupled.
  • gitnexus_context({name: 'classifyContent', file_path: 'lib/ai/classify.ts'}) — uid Function:lib/ai/classify.ts:classifyContent. Incoming callers: scripts/reclassify-items.ts:main, scripts/batch-reclassify-keywords.ts:main, lib/intelligence/pipeline.ts:storeAsContentItem, lib/ingest/markdown-orchestrator.ts:importOneFile, app/api/cron/classification-quality/route.ts:GET, app/api/items/[id]/classify/route.ts:POST (6 direct callers). Outgoing calls include lib/ai/classify.ts:validateDomain, lib/ai/classify.ts:coerceSubtopic, lib/supabase/safe.ts:sb. processes: []. Verdict: HIGH (broad caller fan-out across app ingest + cron + intelligence + MCP surfaces). The TS path calls validateDomain + coerceSubtopic — i.e. it enforces domain/subtopic against the live taxonomy; the cocoindex path does not (the asymmetry that motivates Inv-7).

The integration-correctness gap (why test-green ≠ correct)

Section titled “The integration-correctness gap (why test-green ≠ correct)”

There are three classifier implementations with three different CV wirings (RESEARCH §4.2): lib/ai/classify.ts (TS, app/UI ingest — live-DB taxonomy + validateDomain/coerceSubtopic); scripts/kb_pipeline/classify.py (legacy CLI — warns-not-rejects, RESEARCH §4.2 #2); and scripts/cocoindex_pipeline/extraction.py (the canonical re-ingest classifier — hard-validates content_type + form_type against the snapshot, but leaves primary_domain / primary_subtopic as unconstrained free-text). The cocoindex re-ingest classifier therefore applies the weakest taxonomy constraint of the three on the domain/subtopic dimension. Re-ingesting the corpus now would (a) lose framework / DPS / G-Cloud q_a_form extractions to invalid_enum, and (b) populate content_items.primary_domain / primary_subtopic with values that need not match any live taxonomy_domains / taxonomy_subtopics slug (RESEARCH §8). Both are pre-re-ingest blockers under the ID-64 readiness gate-set.

The drift is locked by a passing test: scripts/tests/test_cocoindex_prompts.py::test_q_a_form_enumerates_form_types (lines 112-130) asserts the legacy 11-value form_type set is present, with the docstring “should list the 11 canonical form_type values” — encoding a stale, partly-invalid contract as GREEN (a docs/reference/test-philosophy.md violation: the test asserts a legacy implementation list, not canonical behaviour). This is the archetypal “test-green-but-integration-wrong” gap ID-63 exists to find.


In scope (this Task’s deliverable surface): the controlled-vocabulary integration-correctness contract on the cocoindex re-ingest path (scripts/cocoindex_pipeline/extraction.py + prompts.py) and its triple-source lockstep guards (markdown register ↔ live DB ↔ snapshot ↔ prompt). Per OQ-63-5, scripts/cocoindex_pipeline/extraction.py is declared the canonical re-ingest classifier; the app-side lib/ai/classify.ts path is canonical for interactive/UI ingest and is out of scope here (its existing validateDomain/coerceSubtopic behaviour is referenced only as the parity baseline for Inv-7, not modified). scripts/kb_pipeline/classify.py is treated as legacy; its retirement is a follow-up disposition (see §Carried notes), not a deliverable of this Task.

Out of scope: requirement_type ingest enforcement (Path-C / T10, unbuilt — nil ingest-time risk, §Carried notes); the larger re-ingest-classifier redesign (handed to the T7 / ID-45 re-ingest spec); ID-9 docubot coupling (kept orthogonal per OQ-63-6); ID-149 / ID-173 closure (Curator-routed). Inv-7 (domain/subtopic) is in scope as a PRODUCT-MAY-PROPOSE invariant with a recommended enforcement mode (soft-warn); the TECH author and Liam confirm the mechanism at the TECH gate.


Form-type drop-vs-grow resolution (OQ-63-1a — RESOLVED: SHRINK the prompt to 8; framework/dps/gcloud are legitimately retired from form_type)

Section titled “Form-type drop-vs-grow resolution (OQ-63-1a — RESOLVED: SHRINK the prompt to 8; framework/dps/gcloud are legitimately retired from form_type)”

The form_type vocabulary source-of-truth is the live form_types Postgres table, regenerated into scripts/tests/fixtures/taxonomy_snapshot.json:form_types by bun run sync:taxonomy, with docs/ontology/26-form-type.md frontmatter baseline_values and {52.6}’s Python _VALID_FORM_TYPES (scripts/cocoindex_pipeline/extraction.py:127) as the markdown and Python consumers. All three currently agree on exactly 8 keys: bid, checklist, itt, pqq, questionnaire, rfp, sales_proposal_template, tender (empirically verified, §Verification).

framework, dps, and gcloud were deliberately migrated out of form_type, not lost. The “Planned 3-tier taxonomy split (RATIFIED-S243 Item 19)” recorded in 26-form-type.md:110-128 — and shipped by the T2 combined-PR migration (20260520120828_t2_combined_pr_intel_shape_b_form_type_split.sql, S246 staging / S247 prod) — split the conflated vocabulary into three Layer-1 CVs:

TierCV (file)ValuesStatus
1 — true form typesform_type (26-form-type.md)bid, rfp, pqq, itt, tender, checklist, questionnaire, sales_proposal_template (8)active; frontmatter trimmed
2 — procurement vehiclesprocurement_vehicle (30-procurement-vehicle.md)framework, dps, dynamic_procurement (3)APPLIED-S246
3 — vehicle instancesprocurement_vehicle_instance (31-procurement-vehicle-instance.md)g_cloud, dos (2)APPLIED-S246

Rationale (Liam-surfaced, S243 Item 19): G-Cloud is a framework instance, not a form; framework and dps are procurement vehicles, not forms. The conflation was inherited from the live template_requirements.template_type 10-value CHECK and cleaned up structurally by T2. Therefore the correct fix direction for OQ-63-1 is to shrink Q_A_FORM_PROMPT’s form_type enumeration (and the locking test’s expectation) to the 8 canonical keysframework / dps / gcloud must be absent from the form_type prompt and the form_type lock. They remain available to the pipeline via their sibling CVs (procurement_vehicle / procurement_vehicle_instance), which are out of the q_a_form form_type contract’s scope. There is no case for growing form_type back to 11.

NEW finding (surfaced, not silently fixed): 26-form-type.md carries residual prose drift the frontmatter trim did not reach — the human-readable ## Baseline values table (lines 66-78), the ## Definition paragraph (line 49), and the Q-OQR1 note (line 51) still list the legacy 11 (including framework/DPS/G-Cloud). The parity test reads only frontmatter baseline_values, so this prose drift is unguarded. Inv-8 covers prose-table ↔ frontmatter parity to close it. This is the one NEW question’s subject (see §Open questions).


Each invariant is a standalone, acceptance-verifiable statement. “The canonical re-ingest classifier” means scripts/cocoindex_pipeline/extraction.py driven by flow.py. “The snapshot” means scripts/tests/fixtures/taxonomy_snapshot.json, regenerated from the live DB by bun run sync:taxonomy. Tests are run with bun run test (TS) / python3 -m pytest (Python) per CLAUDE.md.

A. Form-type prompt/validator/lock alignment (OQ-63-1 + OQ-63-1a — blocker)

Section titled “A. Form-type prompt/validator/lock alignment (OQ-63-1 + OQ-63-1a — blocker)”
  1. Inv-1 — Q_A_FORM_PROMPT enumerates exactly the canonical form_type set. The form_type values enumerated in scripts/cocoindex_pipeline/prompts.py:Q_A_FORM_PROMPT (both the FIELD CONSTRAINTS “MUST be ONE of:” line and any GUIDANCE repetition) MUST equal the snapshot form_types[].key set (currently the 8 keys bid, checklist, itt, pqq, questionnaire, rfp, sales_proposal_template, tender). The retired values framework, dps, gcloud MUST NOT appear in Q_A_FORM_PROMPT as form_type values.

    • Acceptance: every form_type value named in Q_A_FORM_PROMPT is a member of _VALID_FORM_TYPES (which loads from the snapshot); and framework / dps / gcloud are not present in the prompt’s form_type enumeration. (framework may still occur in ENTITY_MENTION_PROMPT’s entity_type list — that is a different vocabulary and is unaffected.)
  2. Inv-2 — no prompt-named form_type is rejected by the validator. The set of form_type values the prompt instructs the model to emit MUST be a subset of the set the validator accepts. After Inv-1 holds, the difference {prompt-named form_types} − _VALID_FORM_TYPES MUST be empty (it is currently {dps, framework, gcloud} — see §Verification). A q_a_form extraction for a genuine framework / DPS / G-Cloud procurement form MUST NOT fail form_type validation as invalid_enum solely because the prompt steered the model toward a retired value.

    • Acceptance: a test asserting {form_types named in Q_A_FORM_PROMPT} ⊆ _VALID_FORM_TYPES passes; equivalently the rejected-subset is empty.
  3. Inv-3 — the form_type prompt lock asserts canonical behaviour, not a frozen list. The test that guards Q_A_FORM_PROMPT’s form_type enumeration (today scripts/tests/test_cocoindex_prompts.py::TestPromptsEnumeratesEnums::test_q_a_form_enumerates_form_types, lines 112-130) MUST derive its expected set from the snapshot (the single source of truth), not from a hardcoded literal. Its docstring MUST NOT claim a fixed count (“11 canonical”) that can silently drift from the snapshot. When the snapshot’s form_types set changes (a value added or removed via sync:taxonomy), this test MUST track it without a manual edit to a hardcoded list.

    • Acceptance: the test loads the expected form_type set from scripts/tests/fixtures/taxonomy_snapshot.json; mutating the snapshot’s form_types (in a fixture/temp copy) changes the test’s expectation correspondingly; and the test passes against the current 8-key snapshot. The pre-existing assertion of the literal 11-value set is gone.

B. Content-type prompt parity guard (OQ-63-2 — regression guard; current state already aligned)

Section titled “B. Content-type prompt parity guard (OQ-63-2 — regression guard; current state already aligned)”
  1. Inv-4 — CLASSIFICATION_PROMPT enumerates exactly the snapshot content_type set. The content_type values enumerated in scripts/cocoindex_pipeline/prompts.py:CLASSIFICATION_PROMPT (the FIELD CONSTRAINTS “MUST be ONE of the following canonical values:” line) MUST equal the snapshot content_types set (currently the 15 values listed in §Verification). The current prompt is already aligned (empirically prompt == snapshot, §Verification); this invariant is a regression guard that fails loudly if a future content_type is added to the snapshot but not the prompt (or vice versa).
    • Acceptance: a test asserting {content_types named in CLASSIFICATION_PROMPT} == snapshot.content_types passes today and fails when either side drifts. Both directions (in-prompt-not-snapshot, in-snapshot-not-prompt) are reported.

C. Domain / subtopic enforcement on the re-ingest path (OQ-63-3 — PRODUCT-MAY-PROPOSE; blocker-grade for re-ingest correctness)

Section titled “C. Domain / subtopic enforcement on the re-ingest path (OQ-63-3 — PRODUCT-MAY-PROPOSE; blocker-grade for re-ingest correctness)”
  1. Inv-5 — content_type stays hard-validated on the re-ingest path (no regression). The canonical re-ingest classifier MUST continue to reject any content_type not in the snapshot content_types set, surfaced as the invalid_enum error class (current ClassificationExtraction._validate_content_type, extraction.py:267-278). This invariant guards against any change introduced by the domain/subtopic work weakening the existing content_type contract.

    • Acceptance: an out-of-vocabulary content_type (e.g. "junk") raises a Pydantic ValidationError that classify_pydantic_error() maps to invalid_enum.
  2. Inv-6 — the re-ingest path’s domain/subtopic vocabulary is sourced from the live taxonomy. Whatever enforcement mode Inv-7 lands, the set of valid primary_domain / primary_subtopic values used by the re-ingest path MUST derive from the live taxonomy_domains / taxonomy_subtopics (via the snapshot’s domains / subtopics arrays, which sync:taxonomy already regenerates — present in the snapshot today). The re-ingest path MUST NOT hardcode a domain/subtopic list that can drift from the live taxonomy.

    • Acceptance: the valid-domain / valid-subtopic set the re-ingest path checks against is loaded from the snapshot’s domains[].name / subtopics[].name (or an equivalent live-taxonomy-backed source), not a literal in prompts.py / extraction.py.
  3. Inv-7 — the re-ingest path surfaces out-of-taxonomy domain/subtopic values (PROPOSED: soft-warn + coverage signal, not silent coercion, not hard-reject). On the canonical re-ingest path, a primary_domain or primary_subtopic value emitted by the model that does not match a live taxonomy slug MUST be surfaced rather than silently accepted as free-text. The recommended mode is soft-warn: the row is still written (the extraction is not dropped), but the out-of-taxonomy value is recorded/counted so a corpus-wide re-ingest produces a measurable “domain/subtopic taxonomy-miss” signal the operator can inspect — rather than the current behaviour where unmatched values pass through invisibly.

    • Rationale for soft-warn over the alternatives: the legacy kb_pipeline path warns-not-rejects (RESEARCH §4.2 #2); the TS app path silently coerces to the closest slug and falls back to validDomains[0] (validateDomain, classify.ts:620-629) — coercion to a possibly-wrong domain is itself a known smell; hard-reject would drop extractions corpus-wide for a soft dimension. Soft-warn + coverage preserves data while making drift visible. Hard-reject vs soft-warn is Liam’s call at the TECH gate — this PRODUCT proposes soft-warn and the TECH author records the ratified mode.
    • Acceptance: an extraction whose primary_domain is not a live taxonomy slug is written AND raises a structured warning / increments a taxonomy-miss counter (the exact surface is a TECH decision); under the proposed mode it is NOT dropped and NOT silently coerced. (If Liam ratifies hard-reject instead, acceptance flips to: the extraction fails validation with a domain-specific error class.)

D. Triple-source lockstep guards for the latent surfaces (OQ-63-4 + OQ-63-6)

Section titled “D. Triple-source lockstep guards for the latent surfaces (OQ-63-4 + OQ-63-6)”
  1. Inv-8 — 26-form-type.md is internally consistent (prose ↔ frontmatter) AND in lockstep with the snapshot. The form_type vocabulary MUST be consistent across all three of 26-form-type.md frontmatter baseline_values, the file’s human-readable ## Baseline values table, and the snapshot form_types[].key set. The existing frontmatter ↔ snapshot guard (__tests__/lib/ontology/form-type-parity.test.ts) MUST be extended (or a sibling guard added) so the prose ## Baseline values table is also held to the same 8-key set — closing the residual prose drift surfaced in §Form-type drop-vs-grow resolution (the prose table currently lists 11).

    • Acceptance: a test fails when the ## Baseline values markdown table in 26-form-type.md lists a form_type not in the frontmatter baseline_values (or vice versa); after the prose table is corrected to 8, frontmatter, prose table, and snapshot all agree.
  2. Inv-9 — requirement_type markdown register is guarded against its live DB CHECK. The requirement_type vocabulary MUST be held in parity across docs/ontology/12-requirement-type.md baseline_values (7 keys: policy, statement, evidence, data, narrative, declaration, reference) and the live form_template_requirements.requirement_type CHECK constraint (form_template_requirements_requirement_type_check, exactly those 7 values — verified §Verification). A parity test MUST guard this both ways, even though the consuming Path-C / T10 is unbuilt, so future drift is caught before Path-C ships. This entails wiring requirement_type into a snapshot key (the snapshot currently has no requirement_type entry) and extending the parity guard (mirroring the content_type / form_type pattern; markdown-parity.test.ts’s SNAPSHOT_KEY_BY_CV_NAME currently omits it).

    • Acceptance: the snapshot exposes the requirement_type value set; a test asserts 12-requirement-type.md baseline_values keys == snapshot requirement_type set == live DB CHECK values; the test fails when any side drifts. The markdown ↔ DB CHECK direction is genuinely covered (not skipped by the editable_via filter).
  3. Inv-10 — lockstep is enforced by in-repo test/CI guards, decoupled from ID-9 docubot. All CV-lockstep invariants (Inv-1 … Inv-9) MUST be enforceable by bun run test / python3 -m pytest in the existing CI test suite — they MUST NOT depend on the ID-9 Astro/docubot docs-publishing path. No invariant in this spec couples CV-loading correctness to the docs-site sync. (Per OQ-63-6: ID-9 publishes markdown to the docs site; it does not regenerate the snapshot, the DB CHECKs, or the prompts.)

    • Acceptance: the new/extended guards run in the standard test suites and gate PRs via the existing CI jobs; none imports or invokes the docs-site sync; removing the docs-site sync entirely would not affect any Inv-1 … Inv-9 guard.

E. Canonical-classifier declaration (OQ-63-5)

Section titled “E. Canonical-classifier declaration (OQ-63-5)”
  1. Inv-11 — the canonical re-ingest classifier is unambiguous and singular. scripts/cocoindex_pipeline/extraction.py (driven by flow.py, writing content_items / q_a_extractions via mount_table_target) is the canonical classifier for the corpus re-ingest. The CV-correctness invariants in this spec apply to that path. The app-side lib/ai/classify.ts path remains canonical for interactive/UI ingest and is unmodified by this Task; scripts/kb_pipeline/classify.py is legacy (retirement is a follow-up disposition, not gated here). No second classifier may be introduced into the re-ingest write path without re-opening this contract.
    • Acceptance: the spec/TECH names exactly one re-ingest classifier; the invariants reference only scripts/cocoindex_pipeline/extraction.py + prompts.py for re-ingest CV correctness; no change in this Task alters lib/ai/classify.ts behaviour.

F. Global quality bars (apply to every implementation Subtask)

Section titled “F. Global quality bars (apply to every implementation Subtask)”
  1. Inv-12 — KH quality bars hold for any code/test/DDL this Task produces. UK English throughout (colour, organise, behaviour, DD/MM/YYYY). Any new Supabase read uses sb() / tryQuery() from @/lib/supabase/safe (no silent failures); no barrel re-exports (direct file imports); tests verify real behaviour not implementation per docs/reference/test-philosophy.md (directly load-bearing for Inv-3, which corrects a test that asserted an implementation list). If any CHECK / table change is proposed (none is required by the in-scope invariants), DDL goes via the Supabase CLI only (supabase migration new + db push) and any new PL/pgSQL carries SET search_path = public, extensions. Tests are invoked with bun run test (not bun test) / python3 -m pytest.
    • Acceptance: a reviewer can confirm each produced artefact against these bars; the form_type lock test (Inv-3) reads as canonical-behaviour prose, not a frozen literal.

Verification (pre-ratification empirical block — OQ-3 / Q-EX2 forcing function)

Section titled “Verification (pre-ratification empirical block — OQ-3 / Q-EX2 forcing function)”

External + internal symbols cited in this PRODUCT were checked against the installed pins in this worktree (worktree-agent-a6304465fe04a1899, off main @ 11cd62e5). The Q-EX2 precedent (cocoindex 1.0.0 removed ExtractByLlm; drift propagated unchecked from RESEARCH → TECH → PLAN → Executor) is the failure mode this block prevents.

DatePinSymbol / artefact checkedResult
29/05/2026cocoindex[postgres]==1.0.3scripts/cocoindex_pipeline/extraction.py imports cocoindex as coco, @coco.fn(memo=True) extractors; module imports cleanlyPRESENT (module imports + validators load; ExtractByLlm known-ABSENT and avoided per {52.1})
29/05/2026anthropic==0.79.0ANTHROPIC_MODEL = "claude-opus-4-6" (extraction.py:65); anthropic.{InternalServerError,RateLimitError,APIConnectionError}PRESENT
29/05/2026pydantic==2.12.5FormMetadata._validate_form_type (extraction.py:179-190), ClassificationExtraction._validate_content_type (extraction.py:267-278), field_validator strict + extra='forbid'PRESENT — _VALID_FORM_TYPES/_VALID_CONTENT_TYPES load from snapshot at import
29/05/2026tenacity==9.1.4_anthropic_retry (AsyncRetrying, stop_after_attempt, wait_exponential)PRESENT (referenced only as context; unaffected by this spec)
29/05/2026zod ^4.4.3lib/ontology/schemas.ts:OntologyCVSchema (.strict(), .superRefine)PRESENT
29/05/2026gray-matter ^4.0.3lib/ontology/loader.ts:loadOntologyCVs frontmatter parsePRESENT

Ground-truth checks (this worktree):

  • Snapshot inventory (scripts/tests/fixtures/taxonomy_snapshot.json): content_types = 15 (article, blog, capability, case_study, certification, compliance, document, methodology, note, other, pdf, policy, product_description, q_a_pair, research); form_types[].key = 8 (bid, checklist, itt, pqq, questionnaire, rfp, sales_proposal_template, tender); no requirement_type key present (grep -c requirement_type → 0).
  • R1 byte-diff (empirical import-and-compare): Q_A_FORM_PROMPT names 11 form_types (bid, rfp, pqq, itt, tender, framework, dps, gcloud, checklist, questionnaire, sales_proposal_template); extraction.py:_VALID_FORM_TYPES accepts 8; {prompt-named} − {validator-accepted} = {dps, framework, gcloud} → confirms R1 (SIGNATURE_DRIFT between the prompt contract and the validator). Direction: SHRINK prompt to 8 (OQ-63-1a).
  • OQ-63-2 byte-diff (empirical): CLASSIFICATION_PROMPT’s enumerated content_type set == snapshot content_types exactly (both directions empty) → currently aligned; Inv-4 is a regression guard, not a fix. (PRESENT — no drift.)
  • R2 lock state: python3 -m pytest scripts/tests/test_cocoindex_prompts.py19 passed — confirming the drifted 11-value contract is locked GREEN by test_q_a_form_enumerates_form_types (the test Inv-3 corrects).
  • Form-type 3-tier split shipped: 30-procurement-vehicle.md frontmatter = framework, dps, dynamic_procurement (status APPLIED-S246); 31-procurement-vehicle-instance.md frontmatter = g_cloud, dos (APPLIED-S246); 26-form-type.md frontmatter baseline_values = the 8 true forms. The retired form_types are present in the sibling CVs → “legitimately retired from form_type”, not dropped. NEW: 26-form-type.md prose ## Baseline values table (lines 66-78) + Definition (49) + Q-OQR1 note (51) still list 11 (residual prose drift; Inv-8).
  • requirement_type DB CHECK (live migration): form_template_requirements_requirement_type_check enforces exactly ('policy','statement','evidence','data','narrative','declaration','reference') (migration …102457:4167, renamed …120828:242) — matches 12-requirement-type.md baseline_values (7) exactly today; no parity test guards it (Inv-9). Behaviour PRESENT/aligned, guard ABSENT.
  • TS-path asymmetry (basis for Inv-6/Inv-7): lib/ai/classify.ts reads live taxonomy_domains (1150) + taxonomy_subtopics (1159) and calls validateDomain (1393, silent coerce → validDomains[0] fallback) + coerceSubtopic (1387, empty→null). The cocoindex path’s ClassificationExtraction.primary_domain: str (extraction.py:262) has no validator → free-text. Confirms R3.

No ABSENT or SIGNATURE_DRIFT against the installed pins blocks ratification. The one SIGNATURE_DRIFT found is the prompt ↔ validator form_type mismatch (R1) — which is the in-repo defect this Task’s Inv-1/Inv-2/Inv-3 exist to fix, not an external-API drift requiring a version-pin change. Ratification of this PRODUCT proceeds.


Carried notes (out of scope — recorded dispositions, not invariants)

Section titled “Carried notes (out of scope — recorded dispositions, not invariants)”
  • requirement_type is Path-C / T10 (UNBUILT). It is authored by the unbuilt AI-cataloguing skill (ID-52.14) and consumed by T10 matching; it is never read by the ingest classifier (RESEARCH §4.1 framing correction). Nil ingest-time risk. Inv-9 adds only the parity guard (cheap latent-drift closure), not ingest enforcement.
  • ID-149 + ID-173 are effectively RESOLVED by {52.5} / {52.6} (markdown-parity + form-type-parity GREEN). The Orchestrator routes them to the Curator for close-as-resolved verification — not a PRODUCT invariant, and not a Planner action. Caveat: these are test-green resolutions; ID-63’s premise is that test-green ≠ integration-correct (which is why the Inv set above goes beyond them).
  • ID-9 (Astro / docubot) is orthogonal. It publishes docs/ontology/*.md to the docs site; it does not regenerate the snapshot, the DB CHECKs, or the prompts. Inv-10 explicitly keeps lockstep enforcement in the in-repo test/CI guard set, decoupled from ID-9.
  • scripts/kb_pipeline/classify.py retirement (OQ-63-5 / R5) and scripts/ontology-sync/parse-flow.py disposition (OQ-63-7 / R7) are follow-up dispositions for the TECH/Curator pass, not gated by this PRODUCT. If parse-flow.py is dead, flag for Knip/Curator removal; if live, assert validator agreement with OntologyCVSchema.

  • NEW — 26-form-type.md prose-table remediation ownership. Inv-8 requires the residual prose drift in 26-form-type.md (the ## Baseline values table + Definition + Q-OQR1 note still listing the legacy 11) to be corrected to the canonical 8 and guarded. The behaviour (prose ↔ frontmatter ↔ snapshot lockstep) is settled by Inv-8; the only open point is whether the TECH author corrects the prose in this Task (recommended — it is a one-file edit fully inside the CV-correctness remit and the guard would otherwise fail) or whether the prose correction is handed to the docubot/docs-IA owner. Recommended default: correct it in this Task as part of satisfying Inv-8 (the guard and the fix ship together), with the docs-site re-publish following naturally via the orthogonal ID-9 path. Liam/Orchestrator to confirm at the TECH gate.

All OQ-63-N from {63.1} are resolved by the decisions above: OQ-63-1 → Inv-1/2/3 (fix in-Task); OQ-63-1a → SHRINK to 8 (§Form-type drop-vs-grow resolution); OQ-63-2 → Inv-4 (regression guard; already aligned); OQ-63-3 → Inv-6/Inv-7 (soft-warn PROPOSED, Liam confirms mode at TECH gate); OQ-63-4 → Inv-9; OQ-63-5 → Inv-11 + §Scope; OQ-63-6 → Inv-10; OQ-63-8 → §Scope (blocker-clearing slice + R3/R5 hand-off). OQ-63-7 (parse-flow.py) → §Carried notes (TECH/Curator disposition).