Skip to content

S535 — Taxonomy grounding: what we classify, what DR-121 actually ruled, and what still stands on the retiring axis

Investigation session (no changes made). Sources: S528 census lanes 1–4 + owner feedback, S529 A1/A2/B reports + D1/D2 board, DR-121/123/124/125/126/127, id-416/417/419/422 task files, and two fresh requirement-first sweeps at canonical HEAD 86fe88a5d (code-consumer census; ratified-docs sweep). Full agent reports are in the S535 session transcript; load-bearing anchors are cited inline.

No ratified source gives subject taxonomy (domains/subtopics) a positive role in the target model. platform-direction.md and entity-glossary.md contain zero occurrences of “taxonomy”, “domain”, or “subtopic”. What exists is four negative rulings:

  1. DR-125 — the (primary_domain, primary_subtopic) topic grain is a transitional fallback whose requirement EXPIRED at first bundle publication (fallback deleted S531, 48821e6d8; scope_tag is the sole topic grain since).
  2. DR-126 — guides need a membership predicate, “but nothing requires it to be primary_domain/primary_subtopic specifically” (weakens lane 1’s last KEEP anchor).
  3. s529-a1 §4.1 — “connectors are coming so we’ll need the taxonomy more” is “unsupported by the ratified model — the classes point the other way”.
  4. The bundle’s shipped ontology.json carries entity_types (12) + relationship_types (10) + concept_types (5). Subject domains/subtopics are not in it.

The implied positive shape (R6 + the Talisman Layer-3 framing in phase-b-prerequisite-1-onthology-pipeline.md): a client-overlay controlled vocabulary co-located with the client bundle — taxonomy as a hierarchy-encoding discipline over CVs, a layer of the ontology, not a driver of app behaviour. That is the owner’s stated position (S528 owner-feedback doc; GROUNDING §4), and nothing found contradicts it. But no doc yet states it — carried as open question Q-A below.

2. What we classify — the axes, disentangled

Section titled “2. What we classify — the axes, disentangled”
AxisQuestion it answersRatified sourceEnforcement todayStatus
content_typewhat a record IS (structural)DR-050; ontology/04; entity-glossaryHARD-reject at extraction.py field_validatorLIVE (set membership = open Q-B)
form_typewhat a form instance ISontology/26; entity-glossaryHARD-reject (Python); FK form_types.keyLIVE (CV file needs REWRITE — still carries bid)
scope_tagwhat a q_a_pair APPLIES TO (relevance/safety)entity-glossary:17; DR-125none yet — implementation is id-422LIVE, primary grain
layer_vocabularyWHO reads it (audience)ontology/03; B.1 owner decisionvalidate_layer_key trigger on reference_items.layerLIVE vocabulary; binding contested (filter vs nesting; DR-126: nesting has no substrate on the answer grain)
L-raw/L-records/L-conceptsWHERE data lives (OKF storage)ontology/36structuralLIVE (re-home to reference/ per lane 3)
entity/relationship/concept typesthe shipped ontologyDR-019/DR-027/DR-082; ontology-sync.mdproducer/validator.py → bundle ontology.jsonLIVE
domains/subtopicswhat content is ABOUT (subject)none positive — DR-125 expiry + three negatives abovezero FK/CHECK/trigger anywhereretiring axis; residual surface in §5

The historical “classification” feature classified content_items (dropped ID-131 M6) onto the subject axis for search filtering, coverage tracking, and the plugin — all IMS-era requirements. Lane 1 §8’s honest summary: if search filtering and domain-keyed governance go, “the only surviving requirement consumer of those values is guides section routing — which needs a vocabulary, not a per-document LLM classification pass”, making the classification stage itself a retire-candidate (held on the unruled questions below).

3. DR-121 — split verdict: location right, composition never ratified

Section titled “3. DR-121 — split verdict: location right, composition never ratified”

The mechanical claim is true and still load-bearing. extraction.py loads FOUR arrays fail-loud at module import (content_types, form_types, domains, subtopics); an id-416 Dockerfile that excludes the file ships an image whose pipeline cannot boot. The move-before-narrowing ordering stands.

But its justification covers only half the file. The DR’s Context cites one production read: the content_types/form_types HARD gate. The other arrays travel only because they are in the same file:

  • domains/subtopics production purpose = the Inv-6/7 soft-warn validator (extraction.py:536-582 — “OBSERVABILITY-ONLY: the row is written UNCHANGED”). Terminus measured this session: TaxonomyMissCounterflow.py:912 payload["taxonomyMisses"]pipeline_runs.result jsonb → rendered ONLY as a collapsible raw-JSON dump in components/provenance/pipeline-failure-drawer.tsx:120. Write-only telemetry; nothing parses or acts on it. The snapshot’s domains never reach the LLM — prompts.py hardcodes its domain examples inline.
  • platforms + requirement_type: zero production consumers. platforms is already RETIRE-verdicted (lane 1 §7) and held only because its CV file routes into unapproved B3.
  • Lane 3 already split the file by upstream (domains/subtopics/form_types DB-read; content_types/platforms/requirement_type register-read), and R6 splits it by ownership (core base CV vs client overlay). One production runtime file cannot honour both.

Recommended reading of DR-121 (finding, not a ruling): it is a file-location ruling under a deploy deadline. It did not ratify the file’s composition, and executing it as-is would re-bless the domains/subtopics bundle as “pipeline runtime data” in the same month the subject grain was expired. The composition decision (slim the snapshot to the structural registries; decide the Inv-7 telemetry’s fate explicitly) should be put to the owner before or with the id-416 move — see Q3.

4. Why this keeps getting re-confused — the mechanism

Section titled “4. Why this keeps getting re-confused — the mechanism”
  1. The name. taxonomy_snapshot.json, sync:taxonomy, “canonical taxonomy” all label a mixed bag in which the load-bearing halves are the structural registries.
  2. The columns. source_documents carries 8 classification columns, primary_domain / primary_subtopic NOT NULL DEFAULT 'unclassified' — a deliberate “mirror content_items 1:1” type-port, not a re-derived model. A session that starts from the schema infers relevance from existence. (The owner named this exact failure mode as the reason for the next-session column cleanup.)
  3. Live docs still teach taxonomy-as-driver. The worst offenders, all published and none banner’d: runbooks/taxonomy-change-runbook.md (“classifying all content”, “the DATABASE is the single source of truth” — direct conflict with ontology/README.md’s “the register is THE single human source of truth”); reference/classification-architecture.md (content_items-era pipeline, two absent scripts, in a directory advertised “safe to cite”); reference/entity-type-taxonomy-spec.md (self-declared “Authoritative”); the product-functionality/ family (documents the deleted /api/taxonomy/* + components as live); ontology/01/02 §Consumers (every listed consumer dead; editable_via: admin_ui with no admin surface). Model to copy: reference/classification-prompt.md’s historical banner.
  4. Unruled questions keep machinery standing — see §7.

5. What still stands on the retiring axis (measured at HEAD)

Section titled “5. What still stands on the retiring axis (measured at HEAD)”
  • hybrid_search filter_domain/filter_subtopic args + MCP find scope enum + lib/mcp/tools/review.ts:333 filter_domains (all downstream of unruled Q1).
  • TaxonomyProvider in the ROOT layout + 7 UI consumers (library filter facet, batch-create pickers, governance section, guide/section dialogs, owner-management, domain badges). Filter/picker half presupposes the retired axis; domain-badge.tsx stays needed only while primary_domain is populated and displayed.
  • Writers: lib/ai/classify.ts (two paths; validates against live DB tables, floors misses to 'unclassified' — the id-419 fix, present) and flow.py (soft-warn only). TRAP: classify.ts skips validation entirely when taxonomy_domains returns zero rows (validDomainSlugs.length > 0 guard) — dropping the tables without touching classify.ts silently disables the id-419 floor and restores raw-LLM writes into a NOT NULL column.
  • sync:taxonomy chain (4 scripts incl. build:plugin) — every output has a live reader, three of four DB-sourced; the plugin is the main consumer and the owner has already ruled the plugin “wouldn’t be a justification for retaining the taxonomy concepts in their current form”.
  • __tests__/validation/taxonomy-consistency.test.ts — hard-pins ≥7 domains / ≥30 subtopics + the hierarchy: an active CI brake on any vocabulary narrowing. The plugin-parity test (A5, held) runs only in the opt-in private-docs lane.
  • supabase/seed.sql §4a/4b — 7 domains + 34 subtopics, deliberately kept as the sole reference baseline (id-417 C6/B2). Note the domain rows embed key_signal classifier-prompt prose. Internally consistent while the tables exist; downstream of the table-fate ruling, not evidence for it.
  • Guides columns (guides.domain_filter, guide_sections.subtopic_filter etc.) — lane 1’s strongest KEEP anchor, since weakened by DR-126.
  • supabase/seed.sql — see §5. Correct while the tables live; carries the vocabulary AND classifier guidance into every reset.
  • supabase/platform/001_canonical_source_tables.sqlstale artefact, delete-candidate. id-95 ({95.12} Option B, “platform control-plane 7-table source DB”) is archived; the model was superseded by the four-DB topology in which Platform prod is a FULL canonical deployment (migrations push to both Platform DBs — e.g. 20260805115054 applied to both). Zero repo references; its baseline-seed step was deferred at S420 pending owner row-selection and never executed; its header rule “NEVER supabase db push to the platform DB” contradicts current standard practice. Note reports/ast-dataflow-schema-audit.md:37 cited it as authority — the exact stale-artefact-read-as-current failure this session investigated.

7. Open questions — S535 owner rulings recorded

Section titled “7. Open questions — S535 owner rulings recorded”
  1. Lane-1 §9 Q1 — owner delegated the design call to the session lead; the recommendation (below, §7a) is: retire domain/subtopic filtering; the driving axes are scope, semantics, and concept membership. Formal DR to be minted when the cleanup task is specced.
  2. B3 — RULED (recorded in tasks/id-417.md, S535 entry): DROP 05, 07, 11, 14, 15, 19, 30, 31; 13-ingest-source REWRITE (connectors-aware, R8); 17-diff-mode RETAIN (unified diff surface = id-117); 27-form-format is an incorrect reference (likely form_instances.mime_type) — correct it. Almost all surviving CVs need rewriting (lane 3 §3). The platforms trim unblocks.
  3. Snapshot — owner asked directly whether it is required at all. Answer: no (§7a below). DR-121’s move dissolves rather than executes; supersede it when the retirement lands.
  4. Q-A — answered by §7a’s target shape; a positive statement should land in entity-glossary.md/platform-direction.md when ruled into a DR.
  5. Q-B / OQ5 — RULED: 04-content-type.md needs FULL rework (IMS baseline values
    • pre-BI-3 raw-type/concept_type mappings conflated in one file); fold the value-set re-derivation into the content-type cleanup, and keep the CHECK withheld until then.

7a. The target shape — RATIFIED (owner, S535) — wants a DR at cleanup-spec time

Section titled “7a. The target shape — RATIFIED (owner, S535) — wants a DR at cleanup-spec time”

Owner, S535: “what you’ve outlined for OQ1 is what I was unable to articulate — ratified.” Carry-through explicitly required into the search surfaces and id-71’s tool/registry rebuild.

Principle: “aboutness” is derived, display-level metadata; nothing is DRIVEN by a platform-global subject vocabulary. A source document or reference item can still show a domain — but the platform’s driving axes are:

  1. Scope (scope_tag) — the only filter-shaped mechanism retrieval needs: exclusion over a small closed boundary set (client feedback Item 3’s safety requirement; DR-125; id-422).
  2. Semantics — embeddings + ai_keywords + entity mentions/relationships, already extracted by the pipeline. Aboutness at query time is ranking, not an exclusion predicate (lane 1 §1.6: subject narrowing over an open space is recall-hostile by construction).
  3. Concept membership — topics in the OKF bundle (scope_tag-keyed since S531) and guides as human-authored concept clusters (DR-126). If users need subject browsing, it is a curation-loop product, not an ingest-time stamp.

Consequences: the per-document LLM subject-classification stage retires with its column pair’s drivers; the 7×34 platform-global vocabulary retires as a platform-core concept; if a client-facing subject vocabulary is ever needed (guides membership display, badges), it returns under R6 as a client-overlay CV in the bundle repo — curated, display-only, never NOT NULL, never minting identity, never a pipeline gate. This scales with the connector roadmap (R8): more heterogeneous inlets make a fixed subject vocabulary worse, while embeddings/entities/concepts are inlet-agnostic — s529-a1 §4.1 already ruled the “connectors need taxonomy more” argument unsupported. The six application types need none of the retiring machinery: procurement/sales retrieval is semantic + scope; intelligence feeds key on company profile + sector config, not corpus classification; guides are curated skeletons.

The snapshot dies with it. Of its six arrays: form_types gates a value the walk discards (ID-136); domains/subtopics feed a write-only counter; platforms/ requirement_type have no consumers; and content_types — the one real gate — is hardcoded in the generator script (generate-taxonomy-snapshot.ts:91-127), so “DB-derived mirror” was never true of it: it is a script constant laundered through a JSON file. Retirement sequence (next session): inline the content-type gate’s set as a documented transitional constant in extraction.py (pending the Q-B rework), delete the other three validators + the miss-counter telemetry, delete snapshot + generator + parity tests + fixture registrations, shrink sync:taxonomy to the plugin-only legs (pending the plugin’s own review), and mark DR-121 superseded — id-416’s Dockerfile then has no data file to carry.

8. Owner follow-up (same session): the three “structural registry” arrays interrogated

Section titled “8. Owner follow-up (same session): the three “structural registry” arrays interrogated”

The owner challenged the retained half too. Measured results:

form_types — real enforcement is the DB FK; the snapshot gate guards a discarded value. The app creation flow writes form_instances.form_type FK-validated against the form_types table (form_instances_form_type_fkey; picker + lib/validation/schemas.ts carry compile-time copies). The pipeline’s hard gate (extraction.py:359) validates the LLM-guessed form_type on extract_qa_form — but the corpus walk stopped writing form_instances at ID-136 (flow.py:4251-4259; sole surviving writer is the app upload path), and form_type appears nowhere else in flow.py: the gated value is discarded. Its only production effect is failing an item when the LLM guesses outside the 7. The owner’s model (seed core rows at client deployment, client-extensible thereafter) matches the DB design already present (provenance core/client/recommended column; seed.sql §2·0b) — the missing half is that no app CRUD surface for form_types exists (read-only picker). The snapshot mirror is the IMS-era residue.

content_type — three vocabularies share one word, and the app’s runtime need is the one DR-050 says should not be this column. (1) The snapshot/register 7 (article, blog, pdf, note, research, document, other) — mixes FORMAT (pdf, document) with editorial kind (article, blog), IMS-ingest-shaped, hard-gated in the pipeline, no DB CHECK anywhere. (2) The plugin/classification-skill set (q_a_pair, case_study, policy, methodology, certification, capability, document) — a different, richer pre-BI-3 set still shipped in lib/ai/skills/classification.md and the plugin. (3) The runtime kind discriminators the app actually branches on — content_type === 'q_a_pair' (template-coverage.ts:315), === 'reference_item' (draft-stream/route.ts:383) — which are GRAIN kinds, the axis DR-050 rules belongs to owner_kind (“content_type … is never overloaded as a grain key” — these sites are the leftover violation). hybrid_search’s filter_kind filters on arm kind (document/answer/reference), NOT on content_type. Net: the 7-value editorial set has no behavioural consumer in the app; its production role is the pipeline gate + guides content_type_filter. The owner’s doubt about the values is lane 3’s blocked Q-B verbatim — the CHECK was deliberately withheld pending exactly this ruling.

requirement_type — live column, dead snapshot array, and the pipeline never touches it. The table was RENAMED form_template_requirementsform_requirement_templates at {145.6} W1c (the platform/001 file carries the old name — more staleness). The commonforms-based extraction lane (ID-145 {145.11}: form_extractors/ detect FIELDS → form_instance_fields) does not write requirement_type. Its only writers are the deliberately human-gated Path-C catalogue promotion (catalogue/from-instance.ts — LLM classifies each catalogued question into the 7-value CHECK set with per-type semantic guidance) and scripts/catalogue-standard-sq.ts. Its live reader is the coverage engine (template-coverage.ts — per-type content-length thresholds, declaration special-case). The snapshot’s requirement_type array has zero production consumers (one parity-test assertion). So: the column is a live, post-IMS, form-templating-domain concern whose vocabulary is a catalogue-design question; the snapshot copy is dead weight and strengthens the DR-121 composition finding — of the six arrays, only content_types has a correctly-shaped production requirement on the snapshot itself, and even its values + column-home are owner-doubted (Q-B).

9. Feed-forward: the source_documents / reference_items column session

Section titled “9. Feed-forward: the source_documents / reference_items column session”

Column-fate map, with the rulings each depends on:

Column(s)Writers todayReaders todayFate hangs on
sd.primary_domain/primary_subtopicclassify.ts ×2, flow.pyMCP resources/review, domain-badge, l_records (SELECT-only passenger — zero reads of the fetched keys)Q1 + Q4; sentinel default is the measured over-exclusion origin
sd.secondary_domain/secondary_subtopicflow.py accepts; TS skill asks, Python prompt doesn’tnone foundQ1 (lane 1 §5: no consumer if filtering retires)
sd.ai_keywords, summary, suggested_title, classification_confidence, classification_reasoning, classified_atclassify.tsNOT censused this session — flag for that sessionclassification-stage question (Q1)
sd.pipeline_run_id/workspace_idalready RETIRE-ruled (id-402 carry), unhomed DDL rider
ri.primary_domain/primary_subtopicreference_ingest split-brain (id-419 S531 note: classified values to ri, ‘unclassified’ to paired sd)ri surfacesQ1/Q4 + id-364/DR-124 lineage
ri.layervalidated (trigger)guides B.1/B.6KEEP per lane 1 §7 “do not sweep up”
ri.source_document_idreference_ingest mints 1:1 sd rowprovenance FKDR-124: a reference item does NOT mint a synthetic sd — the minting path itself is the thing to unwind

Traps for that session: the classify.ts validation-skip (above); the taxonomy-consistency test brake; the opt-in plugin-parity lane; DR-121/id-416 ordering; seed.sql §4 and the §2 fixtures must move in lockstep with any DDL.