ID-132 Frontmatter Wave — routing hints (bl-456) + A19 confidence (bl-477)

ID-132 Frontmatter Wave — bl-456 routing hints + bl-477 A19 confidence

Status: DRAFT (spec-authoring, S480). TECH+PLAN-lite tier — designs mostly ratified upstream; this doc owns the A19 producer-drafted confidence-setting rule, the precise shared touchpoint map, the validator posture, and the two-subtask decomposition. Parent: ID-132 (in_progress) — “OKF L-concepts layer + 2-pass producer”. Realises: bl-456 (ready), bl-477 (ready). Both are members of the bundle-shaping gate set that must land BEFORE the {132.35} BI-18 re-proof ({132.38} + bl-456 + bl-457 + bl-458/{132.40} + bl-477 — doctrine okf-platform/bundle-doctrine.md line 77). Governing rulings (in-force, not re-litigated): DR-079 (bundle doctrine — A19 ratified for BOTH bundle classes, 2026-07-16), DR-019 (OKF superset posture; per-bundle-class closed types), DR-016/DR-027/DR-054/DR-060. A19 value vocabulary strong/partial/no-content/ needs-SME is the id-71 lane-a affordance, already ratified — this wave does not re-derive it.

Context — code-intelligence orientation (verbatim)

Orientation was run against the real symbols each invariant modifies (not feature vocabulary), per the Planner code-intel discipline. Outputs cited verbatim so the Checker can verify the step.

gitnexus_query({query: 'concept frontmatter emitter bundle writer build_concept_frontmatter', repo: 'canonical'}) returned zero processes (the producer frontmatter path is not a graph-indexed execution flow) and these definition hits (verbatim): the emitter Function:scripts/cocoindex_pipeline/producer/frontmatter.py, the bundle writer Function:scripts/cocoindex_pipeline/producer/bundle_writer.py:declare_concept (startLine 295), regenerate_indexes (startLine 417), the Pass-1 draft Function:scripts/cocoindex_pipeline/producer/enrich.py:enrich_concept (startLine 593) + Class:...enrich.py:ConceptDraft (startLine 161), and the Pass-2 Function:scripts/cocoindex_pipeline/producer/web_pass.py:run_web_pass (startLine 765).

gitnexus_context({name: 'build_concept_frontmatter', repo: 'canonical'}) returned uid Function:scripts/cocoindex_pipeline/producer/frontmatter.py:build_concept_frontmatter (startLine 135, endLine 191) with these incoming callers (verbatim): enrich.py: enrich_concept, frontmatter.py:emit_concept_frontmatter, web_pass.py: _parse_reference_concept, web_pass.py:run_web_pass, plus tests test_producer_bundle_writer.py:_fm and test_producer_web_pass.py:_product_draft; outgoing calls: frontmatter.py:_normalise_timestamp, resource_uri.py:is_canonical_resource_uri, resource_uri.py:contains_record_pointer. This is the complete producer call-graph the confidence-derivation must be wired into: three production call sites (enrich_concept L650, _parse_reference_concept L634, run_web_pass L863).

Grep fallback / cross-file (Python + TS not both graph-indexed): grep -rln '"purpose"|audience|routing' scripts/cocoindex_pipeline/producer/ → empty, confirming bl-456’s grep-confirmed “not yet present”. The TS mirror is lib/ontology/concept-schema.ts (ConceptFrontmatterSchema, CONCEPT_TYPE_VALUES). The validator is scripts/cocoindex_pipeline/producer/validator.py (check_concept, _as_mapping, check_required_keys). CONFORMANCE.md is a hand-authored, per-bundle-root reserved file (bundle_writer.py:172 “the hand-authored bundle-root CONFORMANCE.md”; CONFORMANCE_FILENAME is only a reserved-filename constant — NOT a producer-generated template), so the bl-477 “CONFORMANCE.md” touchpoint is documentation, not a code generator (see OQ-3).

Problem & scope

Two OPTIONAL per-concept frontmatter fields, sharing the same emitter / validator / TS-mirror touchpoints, must land in one wave (bl-477 note: “same emitter/validator/TS-mirror touchpoints, land in the same wave”):

1. bl-456 — routing hints. OKF SPEC 4.1 (upstream PR #189) adds three OPTIONAL concept-frontmatter fields — purpose, task, audience — as routing hints, never gates: consumers tolerate their absence. Populated by the Pass-1 agent.
2. bl-477 — A19 confidence. A per-concept confidence field carrying the ratified A19 vocabulary so consumers can down-weight weakly-grounded concepts. Applies to client business AND system baseline bundle classes.

Both are bundle-shaping: changing what the producer writes into every concept .md, gated before the cold full-corpus BI-18 re-proof ({132.35}) so the shape is paid once.

Out of scope: the system-baseline (Path-2, bl-465) confidence-SETTING signal (eval round-trip / human sign-off — a separate parallel lane; the ONLY coupling to this wave is the shared field + vocabulary, doctrine line 72); any DB migration or Supabase type regen (frontmatter is file-level — there is no schema change).

Design — A19 producer-drafted confidence-setting rule (the open-design deliverable)

The doctrine’s open item (bundle-doctrine.md line 88) is: does the A19 vocabulary fit producer-drafted client concepts, and what sets it? Design below; genuine owner-discretion knobs are flagged as numbered OQs with recommendations, not blocked on.

Principle: deterministic, never model-authored

confidence is computed deterministically by the producer at frontmatter-assembly time from draft-time grounding signals — it is never asked of the model. This mirrors the existing producer discipline that resource: is only ever builder output, never model-authored (BI-6/BI-10), and that “format alone is not proof of provenance” (BI-17, enrich.py:618). A model self-assessing its own confidence is unreliable and un-auditable; a deterministic rule over the resolved (resource, citations) is unit-testable and the Checker can verify it exactly.

The signals available at draft time

From enrich_concept (enrich.py:646-658) and both web_pass call sites, at the moment build_concept_frontmatter is called each concept already has:

• resource — its primary record anchor, or None. _resource_from_raw (enrich.py:213) returns a per-row canonical://{source_documents,reference_items}/<uuid> anchor, a q_a_pairs query anchor, or None. The per-row form (distinguishable via the existing is_valid_concept_resource_uri / _PER_ROW_RESOURCE_RE) is the durable client-row grounding; the q_a_pairs query form is thinner; None is unanchored.
• citations — the validated, provenance-checked citation set (_Pass1Envelope. citations; BI-17 guarantees non-empty for a record-backed concept — Pass1DraftError fails loudly on empty). Entries are EITHER canonical:// record anchors OR bare concept cross-link paths; is_valid_concept_resource_uri(c) separates them. Only distinct record anchors count as independent corroboration (a cross-link is not fresh record grounding).

The rule — derive_concept_confidence

A single shared helper (call it derive_concept_confidence, home: frontmatter.py, the same “single shared implementation, all call sites” pattern the codebase already applies to render_citations_trailer / detect_citation_shrink):

derive_concept_confidence(*, resource: str | None, citations: Sequence[str]) -> str
  record_anchors = { c for c in citations if is_valid_concept_resource_uri(c) }
  strong  iff  resource is a PER-ROW anchor (source_documents|reference_items)
               AND len(record_anchors) >= 2        # anchored to a durable client row + corroborated
  else    partial

• strong — anchored to a durable Canonical row (per-row resource:) AND corroborated by ≥2 distinct record anchors. The producer’s highest achievable grounding.
• partial — everything else the Path-1 producer writes: a single record anchor, a q_a_pairs-query-only anchor, resource=None, or a web-enriched references/<slug>.md concept (whose resource is a gated web reference_items anchor — honest “grounded but thin” relative to client records). This is the correct, honest default: a producer-drafted client concept is never not grounded (BI-17), but most are single/thin-source.
• no-content and needs-SME are carried by the FIELD and VALIDATED when present, but are NOT auto-set by the Path-1 producer in this wave (OQ-2): the producer never writes a bodyless concept (Pass1DraftError fails loudly on empty citations, so no-content has no producer path), and no deterministic draft-time signal distinguishes “an expert must verify this” from “thinly grounded” (so needs-SME is a human/SME-review or system-baseline value, not a producer default).

Applied at all three call sites (both passes)

confidence is derived at every build_concept_frontmatter call that produces a client or reference concept, from the (resource, citations) that site already holds:

• enrich.py:650 (Pass-1) — derive_concept_confidence(resource=resource, citations=envelope.citations).
• web_pass.py:634 (_parse_reference_concept, Pass-2 reference concepts) — resource=citations[0] (a gated web anchor, not per-row) → resolves to partial.
• web_pass.py:863 (run_web_pass) — recomputed from the FINAL enriched (resource, citations), so a Pass-1 partial concept that gains a per-row anchor + a second record citation during enrichment can legitimately become strong (never downgraded silently — the rule is monotonic in grounding).

The producer ALWAYS emits confidence; the validator treats it as OPTIONAL

Every producer-written concept carries confidence (strong|partial). But the validator and TS schema must remain optional-tolerant (never gate on absence) per OKF SPEC — because (a) OKF consumers must tolerate absence, and (b) hand-authored, older, and system-baseline-pre-eval bundles may omit it. Optional at read, always-present at Path-1 write.

Shared frontmatter contract extension

Field set (all OPTIONAL)

Field	Type	Set by	Validated
purpose	free string	Pass-1 agent (bl-456)	BI-10 stray-pointer only (no positive shape check)
task	free string	Pass-1 agent (bl-456)	BI-10 stray-pointer only
audience	free string	Pass-1 agent (bl-456)	BI-10 stray-pointer only
confidence	enum strong|partial|no-content|needs-SME	producer (deterministic, bl-477)	enum-membership WHEN present; absence tolerated

Emitter (frontmatter.py)

• ConceptFrontmatter (frozen dataclass) gains four optional fields defaulting to None: purpose, task, audience, confidence.
• build_concept_frontmatter + emit_concept_frontmatter gain four optional kwargs, threaded through. confidence, when supplied, is asserted to be in the A19 set (defence in depth alongside the validator); routing hints get the same BI-10 contains_record_pointer guard the existing string fields get.
• render_concept_frontmatter emits each new field on its own line only when not None, in a FIXED order for deterministic output (BI-18 memo/diff stability): after description, before resource — purpose, task, audience, confidence, then existing resource/tags. Values rendered via the existing _yaml_scalar (safe for the hyphenated no-content/needs-SME — neither matches _YAML_* ambiguity patterns anchored on $).

Validator (validator.py)

• _as_mapping (lines 307-321) must carry the four new dataclass fields into the mapping — otherwise they silently drop out of every downstream check. Load-bearing: the BI-10 check_no_stray_pointer scan iterates frontmatter.items(), so once _as_mapping carries the routing hints they are automatically BI-10-guarded (no separate hint check needed).
• Add _CONFIDENCE_VALUES = frozenset({"strong","partial","no-content","needs-SME"}) + check_confidence(value) — returns [] when the key is absent OR the value is in the set; returns a violation only for a PRESENT-but-invalid value. Wire into check_concept (guarded if "confidence" in fm). Routing hints get NO positive check (optional free strings; absence and any value tolerated bar BI-10).
• Absence of any of the four is never an error (OKF SPEC optional-tolerant posture).

TS mirror (lib/ontology/concept-schema.ts)

• ConceptFrontmatterSchema gains purpose/task/audience as z.string().optional() and confidence as z.enum(CONFIDENCE_VALUES).optional().
• Export CONFIDENCE_VALUES as a const array (mirror of the Python frozenset; the S448 CONCEPT_TYPE_VALUES single-const-array precedent) so a future vocabulary change is one-place. Cross-language parity is by convention (not import), same as CONCEPT_TYPE_VALUES ↔ ALLOWED_CONCEPT_TYPES.

  The cited PRECEDENT is retired (id-427 {427.13}, 2026-08-10) — the DECISION it supports is not. Both CONCEPT_TYPE_VALUES (lib/ontology/concept-schema.ts) and ALLOWED_CONCEPT_TYPES (producer/validator.py) were DELETED by {427.6} under DR-141: concept type is an open label, so there is no vocabulary left for the two sides to hold parity on, and ConceptFrontmatterSchema.type is z.string().min(1). CONFIDENCE_VALUES itself STANDS and is still the right shape — confidence is a genuinely closed four-value vocabulary (strong/partial/no-content/needs-SME, A19/bl-477), which is precisely the property type turned out not to have. The by-convention-not-import parity posture also stands; producer/validator.py’s _CONFIDENCE_VALUES documents its own deliberate duplication.

Pass-1 population surface (enrich.py + prompts.py)

• Routing hints are model-authored (bl-456 “Pass-1 agent populates”). The Pass-1 terminal JSON envelope contract (prompts.py PASS1_INSTRUCTION_PROMPT, keys documented at lines 71-75) is extended to INVITE three OPTIONAL keys purpose/task/audience; the envelope parse (_Pass1Envelope / _parse_pass1_response) reads them if present and tolerates their absence (they are NOT added to _REQUIRED_ENVELOPE_KEYS). Absent → the field is omitted from the emitted frontmatter entirely (not emitted empty).
• confidence is NOT a model key — it is computed by derive_concept_confidence and passed by enrich_concept itself.

Empirical-verification note

No new external-library API surface is introduced. The TS changes reuse Zod patterns already present in the same file (z.enum(...), z.string().optional() — concept-schema.ts:95-106); the Python changes touch only internal Canonical symbols (ConceptFrontmatter, build_concept_frontmatter, check_concept) and stdlib. Per the empirical-verification scope (external-library symbols only), no import-and-call check is warranted for this wave.

Test strategy (behaviour-first — bun run test / python3 -m pytest scripts/tests/)

Behaviour, not implementation (reference/test-philosophy.md). New tests extend the existing test_producer_frontmatter.py, test_producer_validator.py, and a TS spec beside concept-schema.ts; population tested via test_producer_enrich* / test_producer_web_pass.py.

• Emitter: frontmatter renders purpose/task/audience/confidence lines only when supplied; omits them entirely when None; fixed field order is stable; no-content / needs-SME render unquoted and safely.
• Confidence rule: per-row anchor + ≥2 record-anchor citations → strong; single record anchor → partial; q_a_pairs-query anchor → partial; resource=None → partial; a reference concept (web anchor) → partial. Cross-link-only “second citation” does NOT lift to strong (only record anchors corroborate).
• Validator: absence of all four fields → no violation; confidence: "banana" → one violation; each A19 value accepted; a routing-hint string carrying a canonical:// uri → BI-10 violation (proves _as_mapping carries the field).
• TS parity: parseConceptFrontmatter accepts a concept with the four fields, accepts one without them, rejects a bad confidence enum via ZodError.
• Population: a Pass-1 draft with ≥2 record anchors emits confidence: strong; routing hints threaded when the model supplies them and absent-tolerant when it does not; a Pass-2 reference concept emits confidence: partial.

Open questions (owner ratification — recommendations included, none blocking)

• OQ-1 (confidence strong threshold). The corroboration bar is “per-row resource: anchor AND ≥2 distinct record-anchor citations.” Recommendation: adopt as default (2). The only tunable knob; keep it as a named module constant so a later change is one-line.
• OQ-2 (reserved values). no-content and needs-SME are carried + validated but NOT auto-set by the Path-1 producer. Recommendation: confirm reserved — no-content for system-baseline/stub scaffolding, needs-SME for an explicit human/SME-review lane. If ratified this is a small DR addendum to DR-079 (returned as a DR-intent, not written in-branch).
• OQ-3 (CONFORMANCE.md touchpoint). bl-477 lists “CONFORMANCE.md” but it is a hand-authored per-bundle-root file, not an in-repo generator. Recommendation: document the two fields in this spec + the docs-site frontmatter-contract reference (okf-platform/semantic-linter-gate.md or bundle-navigation-contract.md), and add them to the per-bundle CONFORMANCE.md at bundle-authoring time; do NOT block the code wave on a bundle-artefact doc.
• OQ-4 (system-baseline setting rule). Out of scope here (bl-465 lane). Flagged only to record the coupling boundary: this wave ships the shared field/vocabulary/validator; bl-465 defines Path-2’s own eval/sign-off → confidence mapping.

bl-457 overlap report (for wave sequencing)

bl-457 (IRI projection, ready, priority could) is a bundle-shaping change in the SAME gate set, and its owner-ratified decision (2026-07-15) explicitly gives “the producer bundle-writer + frontmatter emitter … a deterministic IRI-minting projection step.” The exact shared files:

File	This wave (bl-456/bl-477)	bl-457 (IRI projection)	Conflict?
producer/frontmatter.py	+4 optional fields on ConceptFrontmatter + emitter kwargs + render order	adds @context/IRI projection to the emitted frontmatter block	YES — same dataclass + render_concept_frontmatter
producer/bundle_writer.py	untouched by this wave (CONFORMANCE.md is doc-only)	gains the deterministic IRI-minting projection step	No (this wave does not touch it)
lib/ontology/concept-schema.ts	+4 optional fields on ConceptFrontmatterSchema	likely adds @context/IRI shape to the same schema	YES — same Zod object
producer/enrich.py, web_pass.py, prompts.py	population wiring	not a primary bl-457 site	Low

Sequencing recommendation: land THIS wave (bl-456/bl-477) FIRST — it is should priority, smaller, purely additive optional fields, and needs no IRI-namespace governance. bl-457 (could) carries the heavier, ratification-gated IRI-namespace design (base vs per-client namespaces, versioning, published-IRI irreversibility) and will re-touch the same ConceptFrontmatter dataclass + render_concept_frontmatter + ConceptFrontmatterSchema; sequencing it AFTER this wave means it rebases onto a settled field set rather than colliding mid-flight. Do NOT dispatch both waves against frontmatter.py / concept-schema.ts concurrently.