S535 — Taxonomy grounding: what we classify, what DR-121 actually ruled, and what still stands on the retiring axis
Investigation session (no changes made). Sources: S528 census lanes 1–4 + owner feedback,
S529 A1/A2/B reports + D1/D2 board, DR-121/123/124/125/126/127, id-416/417/419/422 task
files, and two fresh requirement-first sweeps at canonical HEAD 86fe88a5d (code-consumer
census; ratified-docs sweep). Full agent reports are in the S535 session transcript;
load-bearing anchors are cited inline.
1. The headline
Section titled “1. The headline”No ratified source gives subject taxonomy (domains/subtopics) a positive role in the
target model. platform-direction.md and entity-glossary.md contain zero occurrences
of “taxonomy”, “domain”, or “subtopic”. What exists is four negative rulings:
- DR-125 — the
(primary_domain, primary_subtopic)topic grain is a transitional fallback whose requirement EXPIRED at first bundle publication (fallback deleted S531,48821e6d8;scope_tagis the sole topic grain since). - DR-126 — guides need a membership predicate, “but nothing requires it to be
primary_domain/primary_subtopicspecifically” (weakens lane 1’s last KEEP anchor). - s529-a1 §4.1 — “connectors are coming so we’ll need the taxonomy more” is “unsupported by the ratified model — the classes point the other way”.
- The bundle’s shipped
ontology.jsoncarries entity_types (12) + relationship_types (10) + concept_types (5). Subject domains/subtopics are not in it.
The implied positive shape (R6 + the Talisman Layer-3 framing in
phase-b-prerequisite-1-onthology-pipeline.md): a client-overlay controlled
vocabulary co-located with the client bundle — taxonomy as a hierarchy-encoding
discipline over CVs, a layer of the ontology, not a driver of app behaviour. That is
the owner’s stated position (S528 owner-feedback doc; GROUNDING §4), and nothing found
contradicts it. But no doc yet states it — carried as open question Q-A below.
2. What we classify — the axes, disentangled
Section titled “2. What we classify — the axes, disentangled”| Axis | Question it answers | Ratified source | Enforcement today | Status |
|---|---|---|---|---|
content_type | what a record IS (structural) | DR-050; ontology/04; entity-glossary | HARD-reject at extraction.py field_validator | LIVE (set membership = open Q-B) |
form_type | what a form instance IS | ontology/26; entity-glossary | HARD-reject (Python); FK form_types.key | LIVE (CV file needs REWRITE — still carries bid) |
scope_tag | what a q_a_pair APPLIES TO (relevance/safety) | entity-glossary:17; DR-125 | none yet — implementation is id-422 | LIVE, primary grain |
layer_vocabulary | WHO reads it (audience) | ontology/03; B.1 owner decision | validate_layer_key trigger on reference_items.layer | LIVE vocabulary; binding contested (filter vs nesting; DR-126: nesting has no substrate on the answer grain) |
| L-raw/L-records/L-concepts | WHERE data lives (OKF storage) | ontology/36 | structural | LIVE (re-home to reference/ per lane 3) |
| entity/relationship/concept types | the shipped ontology | DR-019/DR-027/DR-082; ontology-sync.md | producer/validator.py → bundle ontology.json | LIVE |
| domains/subtopics | what content is ABOUT (subject) | none positive — DR-125 expiry + three negatives above | zero FK/CHECK/trigger anywhere | retiring axis; residual surface in §5 |
The historical “classification” feature classified content_items (dropped ID-131 M6)
onto the subject axis for search filtering, coverage tracking, and the plugin — all
IMS-era requirements. Lane 1 §8’s honest summary: if search filtering and domain-keyed
governance go, “the only surviving requirement consumer of those values is guides section
routing — which needs a vocabulary, not a per-document LLM classification pass”, making
the classification stage itself a retire-candidate (held on the unruled questions below).
3. DR-121 — split verdict: location right, composition never ratified
Section titled “3. DR-121 — split verdict: location right, composition never ratified”The mechanical claim is true and still load-bearing. extraction.py loads FOUR
arrays fail-loud at module import (content_types, form_types, domains,
subtopics); an id-416 Dockerfile that excludes the file ships an image whose pipeline
cannot boot. The move-before-narrowing ordering stands.
But its justification covers only half the file. The DR’s Context cites one
production read: the content_types/form_types HARD gate. The other arrays travel
only because they are in the same file:
domains/subtopicsproduction purpose = the Inv-6/7 soft-warn validator (extraction.py:536-582— “OBSERVABILITY-ONLY: the row is written UNCHANGED”). Terminus measured this session:TaxonomyMissCounter→flow.py:912payload["taxonomyMisses"]→pipeline_runs.resultjsonb → rendered ONLY as a collapsible raw-JSON dump incomponents/provenance/pipeline-failure-drawer.tsx:120. Write-only telemetry; nothing parses or acts on it. The snapshot’s domains never reach the LLM —prompts.pyhardcodes its domain examples inline.platforms+requirement_type: zero production consumers.platformsis already RETIRE-verdicted (lane 1 §7) and held only because its CV file routes into unapproved B3.- Lane 3 already split the file by upstream (domains/subtopics/form_types DB-read; content_types/platforms/requirement_type register-read), and R6 splits it by ownership (core base CV vs client overlay). One production runtime file cannot honour both.
Recommended reading of DR-121 (finding, not a ruling): it is a file-location ruling under a deploy deadline. It did not ratify the file’s composition, and executing it as-is would re-bless the domains/subtopics bundle as “pipeline runtime data” in the same month the subject grain was expired. The composition decision (slim the snapshot to the structural registries; decide the Inv-7 telemetry’s fate explicitly) should be put to the owner before or with the id-416 move — see Q3.
4. Why this keeps getting re-confused — the mechanism
Section titled “4. Why this keeps getting re-confused — the mechanism”- The name.
taxonomy_snapshot.json,sync:taxonomy, “canonical taxonomy” all label a mixed bag in which the load-bearing halves are the structural registries. - The columns.
source_documentscarries 8 classification columns,primary_domain/primary_subtopicNOT NULL DEFAULT 'unclassified'— a deliberate “mirror content_items 1:1” type-port, not a re-derived model. A session that starts from the schema infers relevance from existence. (The owner named this exact failure mode as the reason for the next-session column cleanup.) - Live docs still teach taxonomy-as-driver. The worst offenders, all published and
none banner’d:
runbooks/taxonomy-change-runbook.md(“classifying all content”, “the DATABASE is the single source of truth” — direct conflict withontology/README.md’s “the register is THE single human source of truth”);reference/classification-architecture.md(content_items-era pipeline, two absent scripts, in a directory advertised “safe to cite”);reference/entity-type-taxonomy-spec.md(self-declared “Authoritative”); theproduct-functionality/family (documents the deleted/api/taxonomy/*+ components as live);ontology/01/02§Consumers (every listed consumer dead;editable_via: admin_uiwith no admin surface). Model to copy:reference/classification-prompt.md’s historical banner. - Unruled questions keep machinery standing — see §7.
5. What still stands on the retiring axis (measured at HEAD)
Section titled “5. What still stands on the retiring axis (measured at HEAD)”hybrid_searchfilter_domain/filter_subtopicargs + MCPfindscope enum +lib/mcp/tools/review.ts:333filter_domains(all downstream of unruled Q1).TaxonomyProviderin the ROOT layout + 7 UI consumers (library filter facet, batch-create pickers, governance section, guide/section dialogs, owner-management, domain badges). Filter/picker half presupposes the retired axis;domain-badge.tsxstays needed only whileprimary_domainis populated and displayed.- Writers:
lib/ai/classify.ts(two paths; validates against live DB tables, floors misses to'unclassified'— the id-419 fix, present) andflow.py(soft-warn only). TRAP:classify.tsskips validation entirely whentaxonomy_domainsreturns zero rows (validDomainSlugs.length > 0guard) — dropping the tables without touching classify.ts silently disables the id-419 floor and restores raw-LLM writes into a NOT NULL column. sync:taxonomychain (4 scripts incl.build:plugin) — every output has a live reader, three of four DB-sourced; the plugin is the main consumer and the owner has already ruled the plugin “wouldn’t be a justification for retaining the taxonomy concepts in their current form”.__tests__/validation/taxonomy-consistency.test.ts— hard-pins ≥7 domains / ≥30 subtopics + the hierarchy: an active CI brake on any vocabulary narrowing. The plugin-parity test (A5, held) runs only in the opt-in private-docs lane.supabase/seed.sql§4a/4b — 7 domains + 34 subtopics, deliberately kept as the sole reference baseline (id-417 C6/B2). Note the domain rows embedkey_signalclassifier-prompt prose. Internally consistent while the tables exist; downstream of the table-fate ruling, not evidence for it.- Guides columns (
guides.domain_filter,guide_sections.subtopic_filteretc.) — lane 1’s strongest KEEP anchor, since weakened by DR-126.
6. The two files the owner asked about
Section titled “6. The two files the owner asked about”supabase/seed.sql— see §5. Correct while the tables live; carries the vocabulary AND classifier guidance into every reset.supabase/platform/001_canonical_source_tables.sql— stale artefact, delete-candidate. id-95 ({95.12} Option B, “platform control-plane 7-table source DB”) isarchived; the model was superseded by the four-DB topology in which Platform prod is a FULL canonical deployment (migrations push to both Platform DBs — e.g.20260805115054applied to both). Zero repo references; its baseline-seed step was deferred at S420 pending owner row-selection and never executed; its header rule “NEVERsupabase db pushto the platform DB” contradicts current standard practice. Notereports/ast-dataflow-schema-audit.md:37cited it as authority — the exact stale-artefact-read-as-current failure this session investigated.
7. Open questions — S535 owner rulings recorded
Section titled “7. Open questions — S535 owner rulings recorded”- Lane-1 §9 Q1 — owner delegated the design call to the session lead; the recommendation (below, §7a) is: retire domain/subtopic filtering; the driving axes are scope, semantics, and concept membership. Formal DR to be minted when the cleanup task is specced.
- B3 — RULED (recorded in
tasks/id-417.md, S535 entry): DROP 05, 07, 11, 14, 15, 19, 30, 31;13-ingest-sourceREWRITE (connectors-aware, R8);17-diff-modeRETAIN (unified diff surface = id-117);27-form-formatis an incorrect reference (likelyform_instances.mime_type) — correct it. Almost all surviving CVs need rewriting (lane 3 §3). Theplatformstrim unblocks. - Snapshot — owner asked directly whether it is required at all. Answer: no (§7a below). DR-121’s move dissolves rather than executes; supersede it when the retirement lands.
- Q-A — answered by §7a’s target shape; a positive statement should land in
entity-glossary.md/platform-direction.mdwhen ruled into a DR. - Q-B / OQ5 — RULED:
04-content-type.mdneeds FULL rework (IMS baseline values- pre-BI-3 raw-type/concept_type mappings conflated in one file); fold the value-set re-derivation into the content-type cleanup, and keep the CHECK withheld until then.
7a. The target shape — RATIFIED (owner, S535) — wants a DR at cleanup-spec time
Section titled “7a. The target shape — RATIFIED (owner, S535) — wants a DR at cleanup-spec time”Owner, S535: “what you’ve outlined for OQ1 is what I was unable to articulate — ratified.” Carry-through explicitly required into the search surfaces and id-71’s tool/registry rebuild.
Principle: “aboutness” is derived, display-level metadata; nothing is DRIVEN by a platform-global subject vocabulary. A source document or reference item can still show a domain — but the platform’s driving axes are:
- Scope (
scope_tag) — the only filter-shaped mechanism retrieval needs: exclusion over a small closed boundary set (client feedback Item 3’s safety requirement; DR-125; id-422). - Semantics — embeddings +
ai_keywords+ entity mentions/relationships, already extracted by the pipeline. Aboutness at query time is ranking, not an exclusion predicate (lane 1 §1.6: subject narrowing over an open space is recall-hostile by construction). - Concept membership — topics in the OKF bundle (scope_tag-keyed since S531) and guides as human-authored concept clusters (DR-126). If users need subject browsing, it is a curation-loop product, not an ingest-time stamp.
Consequences: the per-document LLM subject-classification stage retires with its column pair’s drivers; the 7×34 platform-global vocabulary retires as a platform-core concept; if a client-facing subject vocabulary is ever needed (guides membership display, badges), it returns under R6 as a client-overlay CV in the bundle repo — curated, display-only, never NOT NULL, never minting identity, never a pipeline gate. This scales with the connector roadmap (R8): more heterogeneous inlets make a fixed subject vocabulary worse, while embeddings/entities/concepts are inlet-agnostic — s529-a1 §4.1 already ruled the “connectors need taxonomy more” argument unsupported. The six application types need none of the retiring machinery: procurement/sales retrieval is semantic + scope; intelligence feeds key on company profile + sector config, not corpus classification; guides are curated skeletons.
The snapshot dies with it. Of its six arrays: form_types gates a value the walk
discards (ID-136); domains/subtopics feed a write-only counter; platforms/
requirement_type have no consumers; and content_types — the one real gate — is
hardcoded in the generator script (generate-taxonomy-snapshot.ts:91-127), so
“DB-derived mirror” was never true of it: it is a script constant laundered through a
JSON file. Retirement sequence (next session): inline the content-type gate’s set as a
documented transitional constant in extraction.py (pending the Q-B rework), delete
the other three validators + the miss-counter telemetry, delete snapshot + generator +
parity tests + fixture registrations, shrink sync:taxonomy to the plugin-only legs
(pending the plugin’s own review), and mark DR-121 superseded — id-416’s Dockerfile
then has no data file to carry.
8. Owner follow-up (same session): the three “structural registry” arrays interrogated
Section titled “8. Owner follow-up (same session): the three “structural registry” arrays interrogated”The owner challenged the retained half too. Measured results:
form_types — real enforcement is the DB FK; the snapshot gate guards a discarded
value. The app creation flow writes form_instances.form_type FK-validated against the
form_types table (form_instances_form_type_fkey; picker + lib/validation/schemas.ts
carry compile-time copies). The pipeline’s hard gate (extraction.py:359) validates the
LLM-guessed form_type on extract_qa_form — but the corpus walk stopped writing
form_instances at ID-136 (flow.py:4251-4259; sole surviving writer is the app
upload path), and form_type appears nowhere else in flow.py: the gated value is
discarded. Its only production effect is failing an item when the LLM guesses
outside the 7. The owner’s model (seed core rows at client deployment, client-extensible
thereafter) matches the DB design already present (provenance core/client/recommended
column; seed.sql §2·0b) — the missing half is that no app CRUD surface for
form_types exists (read-only picker). The snapshot mirror is the IMS-era residue.
content_type — three vocabularies share one word, and the app’s runtime need is the
one DR-050 says should not be this column. (1) The snapshot/register 7
(article, blog, pdf, note, research, document, other) — mixes FORMAT (pdf,
document) with editorial kind (article, blog), IMS-ingest-shaped, hard-gated in
the pipeline, no DB CHECK anywhere. (2) The plugin/classification-skill set
(q_a_pair, case_study, policy, methodology, certification, capability, document) — a
different, richer pre-BI-3 set still shipped in lib/ai/skills/classification.md and the
plugin. (3) The runtime kind discriminators the app actually branches on —
content_type === 'q_a_pair' (template-coverage.ts:315),
=== 'reference_item' (draft-stream/route.ts:383) — which are GRAIN kinds, the axis
DR-050 rules belongs to owner_kind (“content_type … is never overloaded as a grain
key” — these sites are the leftover violation). hybrid_search’s filter_kind filters
on arm kind (document/answer/reference), NOT on content_type. Net: the 7-value
editorial set has no behavioural consumer in the app; its production role is the
pipeline gate + guides content_type_filter. The owner’s doubt about the values is
lane 3’s blocked Q-B verbatim — the CHECK was deliberately withheld pending exactly
this ruling.
requirement_type — live column, dead snapshot array, and the pipeline never touches
it. The table was RENAMED form_template_requirements → form_requirement_templates
at {145.6} W1c (the platform/001 file carries the old name — more staleness). The
commonforms-based extraction lane (ID-145 {145.11}: form_extractors/ detect FIELDS →
form_instance_fields) does not write requirement_type. Its only writers are the
deliberately human-gated Path-C catalogue promotion (catalogue/from-instance.ts — LLM
classifies each catalogued question into the 7-value CHECK set with per-type semantic
guidance) and scripts/catalogue-standard-sq.ts. Its live reader is the coverage engine
(template-coverage.ts — per-type content-length thresholds, declaration
special-case). The snapshot’s requirement_type array has zero production
consumers (one parity-test assertion). So: the column is a live, post-IMS,
form-templating-domain concern whose vocabulary is a catalogue-design question; the
snapshot copy is dead weight and strengthens the DR-121 composition finding — of the
six arrays, only content_types has a correctly-shaped production requirement on the
snapshot itself, and even its values + column-home are owner-doubted (Q-B).
9. Feed-forward: the source_documents / reference_items column session
Section titled “9. Feed-forward: the source_documents / reference_items column session”Column-fate map, with the rulings each depends on:
| Column(s) | Writers today | Readers today | Fate hangs on |
|---|---|---|---|
sd.primary_domain/primary_subtopic | classify.ts ×2, flow.py | MCP resources/review, domain-badge, l_records (SELECT-only passenger — zero reads of the fetched keys) | Q1 + Q4; sentinel default is the measured over-exclusion origin |
sd.secondary_domain/secondary_subtopic | flow.py accepts; TS skill asks, Python prompt doesn’t | none found | Q1 (lane 1 §5: no consumer if filtering retires) |
sd.ai_keywords, summary, suggested_title, classification_confidence, classification_reasoning, classified_at | classify.ts | NOT censused this session — flag for that session | classification-stage question (Q1) |
sd.pipeline_run_id/workspace_id | — | — | already RETIRE-ruled (id-402 carry), unhomed DDL rider |
ri.primary_domain/primary_subtopic | reference_ingest split-brain (id-419 S531 note: classified values to ri, ‘unclassified’ to paired sd) | ri surfaces | Q1/Q4 + id-364/DR-124 lineage |
ri.layer | validated (trigger) | guides B.1/B.6 | KEEP per lane 1 §7 “do not sweep up” |
ri.source_document_id | reference_ingest mints 1:1 sd row | provenance FK | DR-124: a reference item does NOT mint a synthetic sd — the minting path itself is the thing to unwind |
Traps for that session: the classify.ts validation-skip (above); the taxonomy-consistency test brake; the opt-in plugin-parity lane; DR-121/id-416 ordering; seed.sql §4 and the §2 fixtures must move in lockstep with any DDL.