Skip to content

OKF / Corpus-Structure Reframe — SYNTHESIS & RECONCILIATION

OKF / Corpus-Structure Reframe — SYNTHESIS & RECONCILIATION

Section titled “OKF / Corpus-Structure Reframe — SYNTHESIS & RECONCILIATION”

Audience: Product owner (non-developer). Plain-English verdicts first, file:line evidence inline. Inputs reconciled: owner feedback (corpus-structure-feedback.md), research doc (id127-corpus-structure-research.md), 7 grounded lenses (L1 Schema, L2 Pipeline, L3 OKF-spec, L4 Consumers/id-71-104, L5 Evolution/History, L6 Ontology+Workspace, L7 Platform-Direction+Storage). Date: 2026-06-27. Status: synthesis — reconciles the owner’s reframe against ground truth.


Your instinct is right on the big shape and wrong on a few specific attributions, and the research doc contains one load-bearing factual error that must be corrected before you ratify anything irreversible.

  1. You are right that the research doc conflates two different things. It treats the raw client-documentation corpus (what cocoindex reads) and the OKF concept bundle (what an ontology-governed producer would write) as one folder tree. They are two layers. The OKF bundle is a distilled, curated, derived layer that sits on top of the raw provenance substrate — it does not replace it. Every consumer lens (L3, L4, L7) confirms this. (§2)

  2. The research doc’s single most consequential recommendation (D5) rests on a false premise. It says “adopt OKF because OKF supplies a durable, location-independent frontmatter record id.” The OKF spec says the opposite: OKF identity is the file path (Concept ID = path minus .md). OKF is path-as-identity — the exact rel_path coupling the staff review wanted to escape. A durable id is achievable, but only as a Canonical-owned custom frontmatter extension — it is ours to define and defend, not a free OKF gift. The decision (durable id seed) is still correct; the justification is wrong, and the wrong justification would lead a builder to adopt OKF path-identity by default and reproduce the fragility. (§3, decision D5-CORRECTED)

  3. The one irreversible, pre-launch-free decision is the identity seed — and it sits on the ID-45 critical path, because the Phew re-ingest mints the first permanent content rows (flow.py:1924 verbatim: “prod has never completed a content write… Safe to change now”). This is the only thing that genuinely cannot wait. (§4, §5)

  4. Most of your structural hypotheses are “displace, don’t delete” / “rename is cosmetic” rather than clean wins. content_item is vestigial as a concept but the most-wired table in code (~191 TS call-sites + 109 Python). workspace removal contradicts ratified direction — but the content_item_workspaces M2M coupling under it is the genuine cheap cut. RAG is reduced, not removed. (§3)

  5. Recommendation: do not try to carry all of this inside ID-127. Open one new foundational Task (“Canonical Knowledge Layer / OKF substrate”) to own the architecture decisions; rescope ID-127 {127.4} to author OKF-native files on local-fs with the corrected durable-id seed (the irreversible win, at zero pipeline cost); spin the producer build out as bl-367. Ratify exactly three decisions this session; spike or defer the rest. (§4)


Verdict scale: SUPPORTED (ground truth backs you) / REFUTED (ground truth contradicts you) / NUANCED (partly right, important caveat) / NEEDS-DECISION (no ground-truth answer; a product call). Confidence reflects strength + agreement of the grounding lenses.

#Owner point (feedback line)VerdictConfDeciding evidence
FB1Reframe workspace/content_item to activity with its own ID (sales-proposal ID, form ID) — “unnecessary abstraction?” (L23)NUANCEDHIGH”workspace = activity-instance” is already the ratified design: workspaces.application_type_id NOT NULL is the spine and per-app tables are 1:1 satellites (squash:716, 8007). The glossary states “application … is the layer ABOVE workspace” and a form is “workspace-scoped via workspace_id” (platform-direction glossary). So the form/proposal already IS the activity ID; renaming workspaceactivity is cosmetic and fights ratified naming (07-collapse-list §11). The real simplification is dropping the content↔workspace coupling, not the workspace entity. (L1, L5, L6)
FB2qa_sidecar foundation already added; pre-launch we can still flip it (L25)SUPPORTEDHIGHCode is landed and tested: lib/q-a-pairs/sidecar-path.ts, emitCorpusSidecar, _ingest_qa_sidecar_branch, RouteKind=…'qa_sidecar', test_qa_sidecar_roundtrip.py (L5). It is itself a ratified reversal (S380, DEFERRED-v1.1→v1) — cheap to flip again. Critically it already realises “markdown file = canonical, DB rows = derived projection” — the working precedent for your whole OKF reframe (L5).
FB3content_items.source_url should have been dropped via id-118 (L26)REFUTED on the attribution, SUPPORTED on the intuitionHIGHid-118 = “Surface the reference reading UI in platform navigation” — a nav/IA task, not a schema drop (ledger task 118). source_url is still present (squash:271, beside the IMS-heritage platform varchar(30) column) and the drop never happened. BUT your underlying read is right: it is now duplicative — external-URL provenance was re-homed to reference_items via id-110 (you conflated 110 with 118). Caveat: it is still load-bearing on the manual/MCP authoring path (content tools, create-content UI, AI vision fetch — L1), so dropping it is a product decision about the manual “create item from URL” affordance, not a free drop. (L1, L5)
FB4reference_items ingested/extracted but never re-ingested/re-extracted (L28)SUPPORTEDHIGHreference_ingest is idempotent by design: uuid5 on source_url, UNIQUE(source_url), repeat URL “returns already_existed=true and writes NOTHING” (squash:4565-4624; L5). No re-extraction writer exists (L1, L2). Maps cleanly onto an OKF “evidence concept authored once.”
FB5”It walks every file” is wrong — only changed files get walked (L29)NUANCED (you are right on cost)HIGHCategory error, not a disagreement. Enumeration walks every file (walk_dir(live=True, recursive=True), flow.py:3497). Re-extraction (the LLM/Anthropic cost) is delta-only: @coco.fn(memo=True) content-hash memoisation skips unchanged files (extraction.py:196-213,757,797). Your cost intuition is correct; the research doc’s phrasing describes enumeration. (L2, L6, L7)
FB6Upload route is stale (pre-cocoindex); folder-drop newer; final approach TBD (L30)SUPPORTEDHIGHThree write directions, only two write content_items: /api/upload is a stale TS-native direct writer that bypasses cocoindex (own pdf/mammoth extraction, inserts content_items+source_documents+storage bytes); folder-drop is a thin feeder that stages bytes into COCOINDEX_SOURCE_PATH and nudges a walk; cocoindex is the canonical writer (L2). Consolidating onto the cocoindex feeder retires upload’s parallel extraction.
FB7Coolify is S3-compatible (L31)SUPPORTED (factual)MEDIUMRelevant as a git-remote/backup host, not as the pipeline source connector (L7). Feeds the substrate decision (D8 / new-decision-d), not anything immediate.
FB8Gate-keep authoritative docs aligns to OKF enrichment 2nd pass (L45)SUPPORTED + NEEDS-DECISIONHIGHThe reference_agent Pass 2 crawls only operator-provided seed URLs and follows links to enrich concept docs with citations — a near-exact match for your gate-kept-authoritative vision (L3). But today the pipeline ingests-ALL within the corpus dir; “authoritative-only” is folder-curation discipline, not a pipeline gate (L6). Whether to keep gate-keeping vs build an ingest-all-then-sort authority classifier is new-decision-c.
FB9Guides coverage+gaps model; one-record-many-views; intelligence feeds guides (L47)SUPPORTED (concept live), guides-as-app-type unbuiltHIGHThe coverage/gaps model is already live: where_are_we_exposed five-layer tool (data/quality/use_today/gaps/opportunities), coverage_analysis prompt, get_template_gaps, intelligence-guide-generator (L7). A guide’s section taxonomy effectively defines the target concept-schema for a domain — maps cleanly onto OKF concepts and favours authoritative-only ingestion. BUT product_guide is a reserved-seat satellite (0 prod rows); guides-as-application-type is unbuilt (L7).
FB10Ontology pipeline as semantic linter keeping allowed_types/allowed_relations in sync (L68)SUPPORTED (≈80% built); OKF target net-newHIGHAlready a semantic linter at the DB-row write boundary: closed Pydantic Literals (entity_type×12 = allowed_types; relationship×10 = allowed_relations) + snapshot-backed validators (content_type/form_type HARD-reject; domain/subtopic SOFT-WARN), with a markdown↔DB parity harness (L6). It gates DB rows, not an OKF directory — wiring it to gate .md frontmatter is net-new but reuses the existing gate. Fix first: the ontology markdown register is split across repos (canonical/docs/ontology is empty; the 29 CV files live in docs-site) — loader.ts points at the empty dir (L6).
FB11index.md (progressive disclosure) + log.md (change history) are valuable (L71-72)SUPPORTEDHIGHOKF’s only two reserved filenames; spec-valid for exactly these uses; MUST NOT be used for concept docs (L3).
FB12OKF reference impls (reference_agent, viewer, okf-skills) as lift-and-shift (L74-80)NUANCEDHIGHLifts: reference_agent Pass-2 enrichment loop, okf-skills /okf:validate + /okf:visualize (L3). Does NOT lift: reference_agent Pass-1 is BigQuery-only (Canonical has no BQ catalog) and the cross-linked-md producer is net-new custom-target glue — the cocoindex examples write either Neo4j (typed graph) or hierarchical md summaries, neither an interlinked OKF bundle (L2, L3).
FB13content_item_workspaces “probably isn’t needed”; “what exactly is a content_item”; replaced by OKF concepts; IMS-fork heritage (L82)see H1 + H2(M2M) belowAdjudicated as hypotheses H1/H2.

2. The CONFLATION — adjudicated (you are right)

Section titled “2. The CONFLATION — adjudicated (you are right)”

Your suspicion (feedback L41): “with the introduction of OKF as the key underlying knowledge framework, this is being conflated with the client documentation corpus.”

Verdict: SUPPORTED, HIGH confidence. This is the most important reframe in your feedback and the grounding confirms it three ways.

The research doc’s §3.2 “concrete example directory tree” shows .md concept files as the cocoindex INPUT corpus (platform-corpus/content/... rooted at COCOINDEX_SOURCE_PATH). But OKF concepts are properly the cocoindex OUTPUT — the distilled, ontology-governed artefacts a producer writes after extraction. The doc collapses these into one tree, which is why it reads as if “author OKF files = the corpus.”

Ground truth (L2, L7): every cocoindex write target today is postgres.mount_table_target (flow.py:3423-3491). The .md files are the pipeline’s raw input; Postgres rows are its output. Your flow inverts this (OKF bundle becomes the canonical output/store) — and that inversion is a different architecture, not a tidy-up of the research doc’s tree.

2.2 The two layers that must stay distinct

Section titled “2.2 The two layers that must stay distinct”
LayerWhat it isTodayUnder OKF
Raw provenance substrateThe client’s authoritative source files + ingested evidencesource_documents (bytes/storage) + reference_items (one-shot URL evidence)unchanged — stays the canonical source
Curated concept layerDistilled, cross-linked, ontology-typed knowledge agents read directlycontent_items + q_a_pairs rows (in Postgres)the OKF .md concept bundle (the new artefact)

L3 states it exactly: “OKF is best positioned as the distilled, ontology-governed, git-versioned canonical layer that agents read directly; the raw source_documents/reference_items remain the provenance substrate beneath it. This two-layer split is what makes H3 true where it is true.” L4 and L7 concur.

2.3 The deeper contradiction your flow surfaces (needs explicit ratification)

Section titled “2.3 The deeper contradiction your flow surfaces (needs explicit ratification)”

Your step 4 (“Host the files in a private git repo … creating the bulletproof single source of truth”) inverts the ratified vision. Verbatim ground truth:

  • platform-direction.md v1 shape: “Controlled local-fs canonical store. The client’s source files are canonical; KH is a materialised index plus curated derivations.
  • 01-vision.md §2.1: “External-folder canonical. The client’s source files are the canonical content store. KH is a materialised index plus curated derivations — not the storage layer.

So there is a genuine fork you must choose, and it is foundational:

  • (A) OKF bundle = DERIVATION (materialised index over still-canonical client source files — aligned with §2.1). Git gives versioning/rollback of the derived layer; the client’s source files remain canonical.
  • (B) OKF bundle = CANONICAL STORE (the inversion your step-4 implies — KH’s git tree becomes the source of truth, rewriting the “KH is not the storage layer” vision).

Recommendation: ratify (A), the two-layer derivation model. It resolves the conflation, keeps provenance/auditability (your #1 client need — feedback L39) anchored to the client’s real documents, makes the RAG-reduction claim true where it is true (curated layer), and does not require rewriting the vision. (B) is a bigger, riskier bet that throws away the “we don’t hold your data hostage / you keep your canonical files” positioning (principle 2, “helping users organise, not extracting their value”) — adopt it only with eyes open and a deliberate vision rewrite. The git working-tree can still be the substrate for the derived bundle under (A) — that is not the same as making it the client’s canonical store.

Net: OKF is the curated concept layer, not the corpus. Keep raw sources canonical; make the OKF bundle a governed derivation. The research doc’s tree should be split into “raw input corpus” (ID-127’s synthetic-corpus job) vs “OKF concept bundle” (the producer’s output).


H1 — Is content_item vestigial / supersedable by source_documents + reference_items + q_a_pairs?

Section titled “H1 — Is content_item vestigial / supersedable by source_documents + reference_items + q_a_pairs?”

Verdict: NUANCED — “displace, don’t delete.” Diagnosis HIGH confidence; the action is NEEDS-DECISION.

  • Vestigial as a CONCEPT (HIGH): content_item is the IMS personal-knowledge-base fork bucket — schema-corroborated by 12 social/bookmark heritage columns (starred, platform varchar(30), author_name, captured_date, thumbnail_url, user_tags, priority… squash:270-300; I confirmed source_url+platform adjacent at squash:270-272). It has been progressively hollowed out: Q&A → q_a_pairs (the answer_standard/answer_advanced columns now vestigial at squash:311-312), external URLs → reference_items (id-110). It is conspicuously absent from the canonical glossary (platform-direction defines tenant/workspace/form/q_a_pair — not content_item; L7). OKF concept files + the existing typed peer tables cover its substance — OKF removes the last reason to keep a generic “content” abstraction as the knowledge substrate (L3).
  • Load-bearing as IMPLEMENTATION (HIGH): it is the single most-wired corpus table — ~191 TS .from sites across ~40 files + 109 Python touches (L1); the cocoindex content-branch write target; the content_chunks FK parent; the citations.cited_content_item_id target (L4). H1 is a LARGE coordinated refactor, not a quiet drop.
  • No consumer OUTCOME contract is tied to the content_item CONCEPT (HIGH, L4): the contract is the verbs (two-step findgetcite) + response shape + RLS scoping + citations — not the table. Guides, q_a_pairs, reference_items already have independent consumer surfaces.
  • Three live roles must be RE-HOMED before removal (L5): (1) cocoindex content-branch write target; (2) the ID-75 “adopted knowledge vs evidence” semantic distinction (content_items = client-adopted, reference_items = evidence) — the one semantic OKF must explicitly preserve as a frontmatter type/layer; (3) the content_item_workspaces M2M anchor.

Action: treat content_item as a displacement candidate — re-home its three roles onto OKF concepts, then retire the table as a migration. Do not decide “drop” this session (EXPENSIVE, high-blast-radius). Freeze the verb contract, not the table. Tie the citation durable-target id to the OKF record id (same key — see D5-CORRECTED) so cite_content survives the displacement.

H2 — Is workspace an unnecessary abstraction (vs activity / application IDs)?

Section titled “H2 — Is workspace an unnecessary abstraction (vs activity / application IDs)?”

Verdict: NUANCED — workspace-table removal REFUTED; the M2M coupling is the real cut.

  • Strict “drop workspace” is REFUTED (HIGH, L6): platform direction ratified the single workspaces table + application_type_id discriminator (Q-OQR1-04 KISS) and rejected per-app top tables / dual-axis subtypes (04-workspace-types, 07-collapse-list §11). The glossary: workspace = “a container WITHIN one client’s database, bound N:1 to exactly one application_type.” Per-app tables are 1:1 satellites keyed by workspace_id (squash:8007) — the inverse of your “procurement_workspaces is the activity and workspaces is a redundant parent.”
  • Your reframe is partly ALREADY TRUE (HIGH, L5/L6): id-120 (S391, Liam-ratified) already establishes client = 1 DB = tenant; one workspace per application_type; the form/proposal is the activity instance with its own ID. “Workspace = the activity-instance container” is the design — your rename (workspaceactivity) is cosmetic and contradicts ratified naming.
  • The genuine cheap cut is content_item_workspaces (HIGH): it has zero ingest writers (deliberately — flow.py:3071, verbatim confirmed), exactly one operator-curation writer, and is already bypassed for the two concrete cases you cite — forms bind directly via form_templates.workspace_id NOT NULL; Q&A is corpus-level via scope_tag overlap with no junction (L6). Replacing “membership = M2M rows” with “relevance = OKF tags / scope_tag overlap at read time” (the q_a_pairs model, already live) is a near-zero-cost pre-launch swap and lets ID-69’s unbuilt v1.1 M2M-writer be cancelled rather than built.

Action: keep workspaces; drop the content_item_workspaces coupling in favour of query-time scope_tag overlap. Skip the rename.

H3 — Does OKF-bundle-in-git remove (lossy) RAG?

Section titled “H3 — Does OKF-bundle-in-git remove (lossy) RAG?”

Verdict: NUANCED — RAG is REDUCED, not REMOVED. HIGH confidence.

  • REDUCES (by design): for curated, stable, navigational knowledge (schemas, join paths, definitions, guides, canonical records) a cross-linked concept bundle replaces lossy chunk-RAG — an agent reads + updates concepts directly (L3). The platform already serves structured records (not raw chunk-RAG) via typed Postgres rows + MCP — so OKF-in-git changes the substrate, not the already-achieved structured-consumption principle (L7).
  • Does NOT REMOVE (HIGH, L3/L4): (1) the bundle-exceeds-context-window problem returns retrieval at scale — and no source gives the scale boundary (open question: does the SMB authoritative-doc corpus reliably fit?); (2) fuzzy semantic answer in <30s (the O1/James requirement) still needs a vector index — OKF ships none; (3) the raw long-tail (reference_items) still warrants retrieval; (4) OKF has no machine-readable relationship type, no RLS/tenancy model — your entity_mentions/scope_tag/RLS story stays richer and stays yours.
  • Consumer verdict (L4): HELPS more than it hurts. Keep a vector index over the OKF .md files (cocoindex already produces embeddings); find stays hybrid (semantic over the bundle); get traverses links + returns verbatim .md. Sell OKF to consumers as “grounded + provenanced retrieval,” not “no more search.”

Action: answer your own parenthetical (“reducing (removing?)”) as reducing. Do not drop the vector index. The RAG-removal bet is the biggest risk in the whole reframe (§5).

H4 — CocoIndex-as-OKF-producer + ontology semantic-linter?

Section titled “H4 — CocoIndex-as-OKF-producer + ontology semantic-linter?”

Verdict: SUPPORTED as the right architecture; NUANCED on cost (net-new, not a config tweak). Feasibility HIGH, effort MEDIUM.

  • Endorsed as idiomatic and correct (L3): CocoIndex producer → OKF md tree → ontology semantic linter is the right shape.
  • The linter is ~80% built (L6): reuse extraction.py’s field_validator + Pydantic Literal gate; add an OKF write target downstream of the validated extraction. Preserve the HARD-reject vs SOFT-WARN split (block bad type/relation; observe-only on domain). Do not re-implement vocabulary governance.
  • The producer is NET-NEW but de-risked by precedent (L2/L5): every target today is mount_table_target (Postgres); there is no file/markdown target. BUT the qa-sidecar already proves cocoindex can write a canonical .md file with DB rows as a derived projection in this exact pipeline (id-59) — id-59 is the literal template, not a blocker. Generalising the sidecar from Q&A-only to all concepts is the concrete path. Custom targets are buildable on cocoindex 1.0.7 (open question: confirm the stable custom-target API via the cocoindex skill).
  • Fix the source-of-truth fracture first (L6): consolidate the ontology markdown register (the 29 CV files live in docs-site; canonical/docs/ontology is empty; loader.ts points at the empty dir) so the producer and the parity harness read one register.

Action: build as a dedicated spike (bl-367), reusing the validator gate + the id-59 sidecar pattern + a new markdown custom-target. Adopt the reference_agent Pass-2 enrichment loop and okf-skills /okf:validate as accelerators.

H5 — Reject the “no pipeline-logic change” constraint?

Section titled “H5 — Reject the “no pipeline-logic change” constraint?”

Verdict: SUPPORTED — but re-purpose it as a task-scope fence, not a v1 lock. HIGH confidence.

  • It is NOT a v1 architectural lock (HIGH, L7): it is an ID-127 {127.4} scope fence (TECH.md:26-30). Pipeline-logic change is materially free pre-launchflow.py:1924 (verbatim, confirmed): “prod has never completed a content write… Safe to change now.”
  • Lineage backs you (L5): the pipeline has already absorbed three structural reversals pre-launch — reference re-home (id-110/112), diff re-centre (id-117), sidecar promotion (id-59). Changing the producer to write OKF is in character, not unprecedented risk.
  • Consumers are contract-neutral (L4): the substrate swap is invisible to find/get/cite and to headless goose if the verbs keep their shape — strongly favouring re-grounding the substrate now.

Action: reject it as an architectural/v1 lock; retain it as a deliberate fence so ID-127 {127.4} stays fast (scope_tags stay LLM-derived in v1). The real pipeline work (frontmatter parser, markdown target, durable-id seed, M2M-writer cancellation) lands in bl-367 / the new Task, sequenced against — not inside — the v1 cutover.


4. DECISION REGISTER — extending the research doc’s D1–D9

Section titled “4. DECISION REGISTER — extending the research doc’s D1–D9”

The research doc’s D1–D4, D6 stand (low-risk / already-ratified). D5, D7, D8, D9 need correction or re-anchoring given the OKF-spec ground truth, and six NEW foundational decisions (a–f) are added. Reversibility flags: IRREVERSIBLE (pre-launch-free, frozen at ID-45 GO) / EXPENSIVE (large coordinated change) / CHEAP (reversible pre-launch at low cost).

#Research doc saidCORRECTIONReversibility
D5-CORRECTEDSeed identity on “OKF frontmatter record id” because OKF supplies a durable, location-independent id.The premise is false. OKF is path-as-identity (Concept ID = path − .md; L3, verbatim from SPEC.md). Adopt a Canonical-owned custom frontmatter id (OKF’s open-key rule permits unknown keys) and own the uuid5 derivation across all three write paths (cocoindex Python rel_path seed, TS folder-drop destPath seed “INV-1”, upload). The decision stays D5(c)-shaped (durable id, not rel_path) but it is ours to define, and a builder must NOT default to OKF path-identity (that reproduces the fragility).IRREVERSIBLE / pre-launch-free. The one decision that genuinely cannot wait — on the ID-45 critical path.
D7-RECONFIRMEDAuthor OKF-native now (authoring discipline only).Still correct, with D5-CORRECTED’s caveat: the frontmatter id is a Canonical extension. Pipeline need not parse frontmatter to ship {127.4}.CHEAP (authoring discipline).
D8-RECONFIRMEDlocal-fs now; Option-E later.Correct. git-as-corpus does NOT contradict :41 (a git working-tree IS the local-fs folder cocoindex walks; commit/push/rollback is an additive operator layer). Bucket-as-source DOES contradict :41. Coolify-S3 is a git-remote/backup host, not the source connector.CHEAP iff identity is durable-id-seeded (D5).
D9-RE-ANCHOREDRatify corpus-structure + ingest-contract together (they share the seed).Still true, but the shared decision is the seed grain (D5-CORRECTED), not “OKF gives us the id.” Freezing a rel_path-seeded corpus while the seed is open bakes in a migration liability.
#DecisionRecommendationReversibility
(a)Keep / drop content_itemKEEP for v1; plan displacement post-v1. Re-home its 3 live roles (cocoindex write target; ID-75 adopted-vs-evidence semantic; M2M anchor) onto OKF concepts, then retire as a migration. Do not drop this session.EXPENSIVE (191 TS sites + 109 Python; content_chunks FK; citations target; hybrid_search). Cheaper pre-launch but still large.
(b)Keep / drop workspace abstractionKEEP workspaces (ratified activity-instance entity); DROP the content_item_workspaces M2M coupling in favour of query-time scope_tag overlap. Cancel ID-69’s unbuilt v1.1 M2M-writer. Skip the workspaceactivity rename (cosmetic, fights ratified naming).M2M drop: CHEAP (zero ingest writers, near-empty). Workspace-table removal: EXPENSIVE + contradicts ratified direction — don’t.
(c)Gate-keep authoritative-only vs ingest-all-then-sortGATE-KEEP (folder-curation discipline). Aligns with OKF enrichment Pass-2, the guides coverage baseline, and the SMB “no data-governance staff” reality. Ingest-all-then-sort would need a net-new authority-classifier the pipeline has no primitive for.CHEAP (operational discipline; reversible). A product call to ratify.
(d)Substrate = local-fs vs git-repo vs object-storagelocal-fs now (= :41), git working-tree as an additive operator layer for the derived bundle. Defer object-storage; ratify separately later.CHEAP / reversible once identity is durable-id-seeded — the flip becomes a zero-migration mechanism swap. Spike the git-checkout-mid-walk hazard (a branch switch could look like a mass file-change → full re-extract/re-key).
(e)OKF-bundle-in-git as canonical store + RAG-reductionRatify the TWO-LAYER DERIVATION model (option A, §2.3): OKF bundle = governed derivation over still-canonical raw sources; NOT the client’s canonical store. RAG: reduce for the curated layer, keep the vector index. This resolves the conflation and answers the vision §2.1 fork without a vision rewrite.Architecture choice: EXPENSIVE to reverse once built. The (A)-derivation vs (B)-canonical-store fork is foundational — ratify the stance now even though the build is later.
(f)CocoIndex-as-OKF-producer + ontology semantic-linterPURSUE as bl-367 spike. Reuse the extraction validator gate (linter ≈80% built) + the id-59 sidecar precedent + a net-new markdown custom-target. Preserve HARD-reject/SOFT-WARN. Fix the ontology-register repo split first.Decision to pursue: CHEAP (additive; DB targets stay). Build: EXPENSIVE (net-new custom target). Confirm cocoindex custom-target API via the cocoindex skill.

5.1 Reframe ID-127, or open a new foundational Task? — BOTH, split cleanly.

Section titled “5.1 Reframe ID-127, or open a new foundational Task? — BOTH, split cleanly.”

The owner’s reframe is bigger than ID-127 (it touches content_item displacement, the OKF producer, the storage substrate, the RAG model, and the vision’s canonical-store fork). Cramming it into {127.4} would either bloat a synthetic-corpus task into an architecture epic or smuggle irreversible decisions in under a “no-pipeline-change” envelope.

Recommendation:

  1. Open ONE new foundational Task — “Canonical Knowledge Layer / OKF substrate” — owning new decisions (a)–(f) + D5-CORRECTED + the §2.3 two-layer ratification. This is the home for the architecture, the bl-367 producer spike, and the content_item displacement plan.
  2. Rescope ID-127 {127.4} to: author OKF-native synthetic files on local-fs, with the Canonical custom frontmatter id seed (the irreversible win, zero pipeline cost), a forms-only .kh-workspace-map.json, and an explicit “no content folder implies a workspace” gate pinned in PLAN S4. Explicitly defer the producer/substrate/RAG work to the new Task. ID-127 authoring is not blocked by the full reframe — only by the seed decision.
  3. Keep bl-367 as the OKF producer/pipeline spike under the new Task.

5.2 Sequencing against goose G4 and the ID-45 v1 gate

Section titled “5.2 Sequencing against goose G4 and the ID-45 v1 gate”
  • ID-45 (Phew first-client re-ingest) is the v1 gate ({45.3}/{45.4} TECH/PLAN unwritten). The re-ingest mints the first permanent content rows, so the identity-seed grain HARDENS at ID-45 GO. D5-CORRECTED must precede ID-45’s first write — this pulls the seed decision onto the ID-45 critical path and likely re-opens {45.3}/{45.4} to encode the durable-id seed. This is the single sequencing constraint that bites.
  • goose G4 (current branch, headless agent) is NOT blocked (L4). Headless consumes the same remote MCP server (find/get/cite); the OKF substrate swap is invisible to the consumer surface as long as the verbs keep their shape. G4 proceeds independently and in parallel. The OKF reframe is a substrate concern, not a consumer-surface concern — do not gate G4 on it.
  • The new foundational Task runs in parallel with goose G4; its only hard handshake with ID-45 is the seed decision, which should be ratified before ID-45 authoring resumes.

5.3 The SMALLEST set to ratify THIS session (3 decisions)

Section titled “5.3 The SMALLEST set to ratify THIS session (3 decisions)”

Ratify exactly these three; everything else can be spiked or deferred:

  1. D5-CORRECTED — the durable identity seed. Adopt a Canonical-owned frontmatter id (not rel_path, not OKF path-identity), owned across all three write paths. IRREVERSIBLE, pre-launch- free, on the ID-45 critical path. This is the only decision that cannot wait.
  2. (e) — the two-layer derivation stance. Ratify that the OKF bundle is a governed derivation over still-canonical raw sources (not the client’s canonical store). This resolves the conflation and frames every downstream build. Cheap to state now, expensive to reverse after building.
  3. (b) — drop the content_item_workspaces coupling. Ratify query-time scope_tag overlap as the membership mechanism; cancel ID-69’s v1.1 M2M-writer. Low-risk, unblocks the corpus structure, and removes the weakest-defended abstraction.
  • (a) content_item displacement — spike the re-homing plan; do not decide “drop” (EXPENSIVE).
  • (f) OKF producer build — bl-367 spike; confirm cocoindex custom-target API; fix the ontology-register repo split first.
  • (d) object-storage substrate — defer; ratify separately once (e) and D5 are in.
  • (c) gate-keep vs ingest-all — product call; ratify soon, but not blocking the seed.
  • workspaceactivity rename — drop it (cosmetic, fights ratified naming).
  • Drop content_items.source_url — sequence WITH the displacement, not before (live manual-path consumers; ast-dataflow column-reads sweep first).

  1. The RAG-removal bet (HIGHEST). If the platform commits to “OKF removes RAG” and drops the vector index, it breaks the O1 <30s fuzzy-semantic answer requirement and long-tail reference_items retrieval. RAG is reduced, not removed — safe only for the curated concept layer that fits the agent context window, and no source gives the scale boundary (open question: do the SMB authoritative docs reliably fit?). Mitigation: keep a vector index over the OKF .md bundle; treat OKF as “grounded retrieval,” not “no search”; empirically size the first client’s authoritative corpus before betting.

  2. Dropping content_item (HIGH). High-blast-radius rewire — 191 TS sites, content_chunks FK, citations.cited_content_item_id, hybrid_search. A botched displacement breaks find/get/cite. The citation durable-target id must become the same key as the OKF record id (ties to D5) or cite_content loses its anchor. Mitigation: displace (re-home roles), don’t delete; freeze the verb contract not the table; do it pre-launch when no prod rows exist.

  3. The D5 false-premise trap (HIGH, subtle). Ratifying the seed on “OKF gives us a durable id” leads a builder to adopt OKF path-identity (the spec’s actual behaviour) and reproduce the exact rel_path fragility the whole exercise tries to escape. Mitigation: explicitly define the Canonical custom id extension and own its uuid5 derivation in lockstep across Python + TS (the seed lives in two languages — folder-drop destPath “INV-1” vs Python rel_path).

  4. RLS / tenancy gap (MEDIUM, hard-blocks multi-tenant). OKF has no tenancy/authorization model; every MCP tool is per-user RLS-scoped. If the git bundle becomes the authorization boundary, multi-tenant security breaks. Mitigation: keep the DB/API as the auth layer over the bundle; single-tenant Phew launch tolerates the gap (mempalace direct-pattern precedent), but never assume the git tree is the security boundary.

  5. The conflation re-entering the build (MEDIUM). If the team builds one tree that is both the raw corpus and the OKF bundle (the research doc’s collapse), the “self-updating wiki” round-trip becomes an operational hazard — a git checkout/pull mid-walk can look like a mass file-change event → full re-extract/re-key. Mitigation: ratify the two-layer model (e); keep raw input and derived bundle as distinct trees; spike the git-mid-walk behaviour (d).

  6. Vision inversion shipped silently (MEDIUM). Your step-4 “single source of truth” quietly rewrites “KH is not the storage layer” (vision §2.1) and the “we don’t extract your value / per-tenant corpus” positioning (principle 2). Mitigation: if you ever choose (e)-option-(B) over (A), do it as a deliberate, documented vision rewrite — not as a side-effect of a corpus task.


7. Evidence index (decisive file:line, verified this session)

Section titled “7. Evidence index (decisive file:line, verified this session)”
  • Vision/substrate (the conflation + step-4 inversion): platform-direction.md v1 shape “the client’s source files are canonical; KH is a materialised index plus curated derivations”; 01-vision.md §2.1 “External-folder canonical … not the storage layer.” Glossary defines tenant/workspace/form/q_a_pair — content_item absent. (verified)
  • OKF path-as-identity (corrects D5): L3 — SPEC.md “Concept ID = the path … with the .md suffix removed”; only type REQUIRED; links untyped; no tenancy model; v0.1 Draft.
  • Seed free now / IRREVERSIBLE: flow.py:1922-1924 (verified) “prod has never completed a content write… Safe to change now”; uuid5 on rel_path (flow.py:2110-2111).
  • M2M deliberately unpopulated: flow.py:3068-3076 (verified verbatim) — content branch is “CANONICAL, workspace-AGNOSTIC … M2M junction is DELIBERATELY not populated here.” Sole writer app/api/items/[id]/workspaces/route.ts:154,183.
  • content_item IMS heritage + hollowing: squash:270-300 (12 personal-KB cols incl source_url, platform — verified adjacent at 270-272); 311-312 vestigial answer_standard/advanced; ~191 TS
    • 109 Python touches (L1).
  • qa-sidecar precedent (H3/H4 template): lib/q-a-pairs/sidecar-path.ts, _ingest_qa_sidecar_branch, id-59 PRODUCT “RATIFIED REVERSAL S380 … sidecar file canonical, DB rows derived projections” (L5).
  • Delta re-extraction (FB5): extraction.py:196-213,757,797 memo=true (L2).
  • Three write directions (FB6): /api/upload direct writer vs folder-drop feeder vs cocoindex canonical (L2).
  • Ontology linter ≈80% built: extraction.py:378-391 (entity_type×12), :423-436 (relationship×10), :470-481 HARD-reject, :483-529 SOFT-WARN; register repo-split — loader.ts → empty docs/ontology (L6).
  • Coverage/gaps live (FB9): dashboard.ts:47-57 five-layer where_are_we_exposed; templates.ts:182 get_template_gaps (L7).
  • Consumer contract = verbs not table: search.ts (find→hybrid_search over content_items), content.ts:63,215 (get), procurement.ts:464-470 (cite_content → cited_content_item_id); id-104 AgentEvalContract kind:'agent_recipe', grounding_shape:'citations' (L4).
  • v1 gate / ID-45: v1-completion-sequence.md:17-33,66-68 (v1.0.0 cut; ID-45 only gate; {45.3}/{45.4} unwritten) (L7).