OKF / Corpus-Structure Reframe — SYNTHESIS & RECONCILIATION
OKF / Corpus-Structure Reframe — SYNTHESIS & RECONCILIATION
Section titled “OKF / Corpus-Structure Reframe — SYNTHESIS & RECONCILIATION”Audience: Product owner (non-developer). Plain-English verdicts first, file:line evidence inline.
Inputs reconciled: owner feedback (corpus-structure-feedback.md), research doc
(id127-corpus-structure-research.md), 7 grounded lenses (L1 Schema, L2 Pipeline, L3 OKF-spec,
L4 Consumers/id-71-104, L5 Evolution/History, L6 Ontology+Workspace, L7 Platform-Direction+Storage).
Date: 2026-06-27. Status: synthesis — reconciles the owner’s reframe against ground truth.
0. The headline (read this first)
Section titled “0. The headline (read this first)”Your instinct is right on the big shape and wrong on a few specific attributions, and the research doc contains one load-bearing factual error that must be corrected before you ratify anything irreversible.
-
You are right that the research doc conflates two different things. It treats the raw client-documentation corpus (what cocoindex reads) and the OKF concept bundle (what an ontology-governed producer would write) as one folder tree. They are two layers. The OKF bundle is a distilled, curated, derived layer that sits on top of the raw provenance substrate — it does not replace it. Every consumer lens (L3, L4, L7) confirms this. (§2)
-
The research doc’s single most consequential recommendation (D5) rests on a false premise. It says “adopt OKF because OKF supplies a durable, location-independent frontmatter record id.” The OKF spec says the opposite: OKF identity is the file path (
Concept ID = path minus .md). OKF is path-as-identity — the exact rel_path coupling the staff review wanted to escape. A durable id is achievable, but only as a Canonical-owned custom frontmatter extension — it is ours to define and defend, not a free OKF gift. The decision (durable id seed) is still correct; the justification is wrong, and the wrong justification would lead a builder to adopt OKF path-identity by default and reproduce the fragility. (§3, decision D5-CORRECTED) -
The one irreversible, pre-launch-free decision is the identity seed — and it sits on the ID-45 critical path, because the Phew re-ingest mints the first permanent content rows (
flow.py:1924verbatim: “prod has never completed a content write… Safe to change now”). This is the only thing that genuinely cannot wait. (§4, §5) -
Most of your structural hypotheses are “displace, don’t delete” / “rename is cosmetic” rather than clean wins.
content_itemis vestigial as a concept but the most-wired table in code (~191 TS call-sites + 109 Python).workspaceremoval contradicts ratified direction — but thecontent_item_workspacesM2M coupling under it is the genuine cheap cut. RAG is reduced, not removed. (§3) -
Recommendation: do not try to carry all of this inside ID-127. Open one new foundational Task (“Canonical Knowledge Layer / OKF substrate”) to own the architecture decisions; rescope ID-127 {127.4} to author OKF-native files on local-fs with the corrected durable-id seed (the irreversible win, at zero pipeline cost); spin the producer build out as bl-367. Ratify exactly three decisions this session; spike or defer the rest. (§4)
1. Position on each OWNER FEEDBACK point
Section titled “1. Position on each OWNER FEEDBACK point”Verdict scale: SUPPORTED (ground truth backs you) / REFUTED (ground truth contradicts you) / NUANCED (partly right, important caveat) / NEEDS-DECISION (no ground-truth answer; a product call). Confidence reflects strength + agreement of the grounding lenses.
| # | Owner point (feedback line) | Verdict | Conf | Deciding evidence |
|---|---|---|---|---|
| FB1 | Reframe workspace/content_item to activity with its own ID (sales-proposal ID, form ID) — “unnecessary abstraction?” (L23) | NUANCED | HIGH | ”workspace = activity-instance” is already the ratified design: workspaces.application_type_id NOT NULL is the spine and per-app tables are 1:1 satellites (squash:716, 8007). The glossary states “application … is the layer ABOVE workspace” and a form is “workspace-scoped via workspace_id” (platform-direction glossary). So the form/proposal already IS the activity ID; renaming workspace→activity is cosmetic and fights ratified naming (07-collapse-list §11). The real simplification is dropping the content↔workspace coupling, not the workspace entity. (L1, L5, L6) |
| FB2 | qa_sidecar foundation already added; pre-launch we can still flip it (L25) | SUPPORTED | HIGH | Code is landed and tested: lib/q-a-pairs/sidecar-path.ts, emitCorpusSidecar, _ingest_qa_sidecar_branch, RouteKind=…'qa_sidecar', test_qa_sidecar_roundtrip.py (L5). It is itself a ratified reversal (S380, DEFERRED-v1.1→v1) — cheap to flip again. Critically it already realises “markdown file = canonical, DB rows = derived projection” — the working precedent for your whole OKF reframe (L5). |
| FB3 | content_items.source_url should have been dropped via id-118 (L26) | REFUTED on the attribution, SUPPORTED on the intuition | HIGH | id-118 = “Surface the reference reading UI in platform navigation” — a nav/IA task, not a schema drop (ledger task 118). source_url is still present (squash:271, beside the IMS-heritage platform varchar(30) column) and the drop never happened. BUT your underlying read is right: it is now duplicative — external-URL provenance was re-homed to reference_items via id-110 (you conflated 110 with 118). Caveat: it is still load-bearing on the manual/MCP authoring path (content tools, create-content UI, AI vision fetch — L1), so dropping it is a product decision about the manual “create item from URL” affordance, not a free drop. (L1, L5) |
| FB4 | reference_items ingested/extracted but never re-ingested/re-extracted (L28) | SUPPORTED | HIGH | reference_ingest is idempotent by design: uuid5 on source_url, UNIQUE(source_url), repeat URL “returns already_existed=true and writes NOTHING” (squash:4565-4624; L5). No re-extraction writer exists (L1, L2). Maps cleanly onto an OKF “evidence concept authored once.” |
| FB5 | ”It walks every file” is wrong — only changed files get walked (L29) | NUANCED (you are right on cost) | HIGH | Category error, not a disagreement. Enumeration walks every file (walk_dir(live=True, recursive=True), flow.py:3497). Re-extraction (the LLM/Anthropic cost) is delta-only: @coco.fn(memo=True) content-hash memoisation skips unchanged files (extraction.py:196-213,757,797). Your cost intuition is correct; the research doc’s phrasing describes enumeration. (L2, L6, L7) |
| FB6 | Upload route is stale (pre-cocoindex); folder-drop newer; final approach TBD (L30) | SUPPORTED | HIGH | Three write directions, only two write content_items: /api/upload is a stale TS-native direct writer that bypasses cocoindex (own pdf/mammoth extraction, inserts content_items+source_documents+storage bytes); folder-drop is a thin feeder that stages bytes into COCOINDEX_SOURCE_PATH and nudges a walk; cocoindex is the canonical writer (L2). Consolidating onto the cocoindex feeder retires upload’s parallel extraction. |
| FB7 | Coolify is S3-compatible (L31) | SUPPORTED (factual) | MEDIUM | Relevant as a git-remote/backup host, not as the pipeline source connector (L7). Feeds the substrate decision (D8 / new-decision-d), not anything immediate. |
| FB8 | Gate-keep authoritative docs aligns to OKF enrichment 2nd pass (L45) | SUPPORTED + NEEDS-DECISION | HIGH | The reference_agent Pass 2 crawls only operator-provided seed URLs and follows links to enrich concept docs with citations — a near-exact match for your gate-kept-authoritative vision (L3). But today the pipeline ingests-ALL within the corpus dir; “authoritative-only” is folder-curation discipline, not a pipeline gate (L6). Whether to keep gate-keeping vs build an ingest-all-then-sort authority classifier is new-decision-c. |
| FB9 | Guides coverage+gaps model; one-record-many-views; intelligence feeds guides (L47) | SUPPORTED (concept live), guides-as-app-type unbuilt | HIGH | The coverage/gaps model is already live: where_are_we_exposed five-layer tool (data/quality/use_today/gaps/opportunities), coverage_analysis prompt, get_template_gaps, intelligence-guide-generator (L7). A guide’s section taxonomy effectively defines the target concept-schema for a domain — maps cleanly onto OKF concepts and favours authoritative-only ingestion. BUT product_guide is a reserved-seat satellite (0 prod rows); guides-as-application-type is unbuilt (L7). |
| FB10 | Ontology pipeline as semantic linter keeping allowed_types/allowed_relations in sync (L68) | SUPPORTED (≈80% built); OKF target net-new | HIGH | Already a semantic linter at the DB-row write boundary: closed Pydantic Literals (entity_type×12 = allowed_types; relationship×10 = allowed_relations) + snapshot-backed validators (content_type/form_type HARD-reject; domain/subtopic SOFT-WARN), with a markdown↔DB parity harness (L6). It gates DB rows, not an OKF directory — wiring it to gate .md frontmatter is net-new but reuses the existing gate. Fix first: the ontology markdown register is split across repos (canonical/docs/ontology is empty; the 29 CV files live in docs-site) — loader.ts points at the empty dir (L6). |
| FB11 | index.md (progressive disclosure) + log.md (change history) are valuable (L71-72) | SUPPORTED | HIGH | OKF’s only two reserved filenames; spec-valid for exactly these uses; MUST NOT be used for concept docs (L3). |
| FB12 | OKF reference impls (reference_agent, viewer, okf-skills) as lift-and-shift (L74-80) | NUANCED | HIGH | Lifts: reference_agent Pass-2 enrichment loop, okf-skills /okf:validate + /okf:visualize (L3). Does NOT lift: reference_agent Pass-1 is BigQuery-only (Canonical has no BQ catalog) and the cross-linked-md producer is net-new custom-target glue — the cocoindex examples write either Neo4j (typed graph) or hierarchical md summaries, neither an interlinked OKF bundle (L2, L3). |
| FB13 | content_item_workspaces “probably isn’t needed”; “what exactly is a content_item”; replaced by OKF concepts; IMS-fork heritage (L82) | see H1 + H2(M2M) below | — | Adjudicated as hypotheses H1/H2. |
2. The CONFLATION — adjudicated (you are right)
Section titled “2. The CONFLATION — adjudicated (you are right)”Your suspicion (feedback L41): “with the introduction of OKF as the key underlying knowledge framework, this is being conflated with the client documentation corpus.”
Verdict: SUPPORTED, HIGH confidence. This is the most important reframe in your feedback and the grounding confirms it three ways.
2.1 What the research doc conflates
Section titled “2.1 What the research doc conflates”The research doc’s §3.2 “concrete example directory tree” shows .md concept files as the
cocoindex INPUT corpus (platform-corpus/content/... rooted at COCOINDEX_SOURCE_PATH). But OKF
concepts are properly the cocoindex OUTPUT — the distilled, ontology-governed artefacts a
producer writes after extraction. The doc collapses these into one tree, which is why it reads
as if “author OKF files = the corpus.”
Ground truth (L2, L7): every cocoindex write target today is postgres.mount_table_target
(flow.py:3423-3491). The .md files are the pipeline’s raw input; Postgres rows are its output.
Your flow inverts this (OKF bundle becomes the canonical output/store) — and that inversion is a
different architecture, not a tidy-up of the research doc’s tree.
2.2 The two layers that must stay distinct
Section titled “2.2 The two layers that must stay distinct”| Layer | What it is | Today | Under OKF |
|---|---|---|---|
| Raw provenance substrate | The client’s authoritative source files + ingested evidence | source_documents (bytes/storage) + reference_items (one-shot URL evidence) | unchanged — stays the canonical source |
| Curated concept layer | Distilled, cross-linked, ontology-typed knowledge agents read directly | content_items + q_a_pairs rows (in Postgres) | the OKF .md concept bundle (the new artefact) |
L3 states it exactly: “OKF is best positioned as the distilled, ontology-governed, git-versioned
canonical layer that agents read directly; the raw source_documents/reference_items remain
the provenance substrate beneath it. This two-layer split is what makes H3 true where it is
true.” L4 and L7 concur.
2.3 The deeper contradiction your flow surfaces (needs explicit ratification)
Section titled “2.3 The deeper contradiction your flow surfaces (needs explicit ratification)”Your step 4 (“Host the files in a private git repo … creating the bulletproof single source of truth”) inverts the ratified vision. Verbatim ground truth:
platform-direction.mdv1 shape: “Controlled local-fs canonical store. The client’s source files are canonical; KH is a materialised index plus curated derivations.”01-vision.md §2.1: “External-folder canonical. The client’s source files are the canonical content store. KH is a materialised index plus curated derivations — not the storage layer.”
So there is a genuine fork you must choose, and it is foundational:
- (A) OKF bundle = DERIVATION (materialised index over still-canonical client source files — aligned with §2.1). Git gives versioning/rollback of the derived layer; the client’s source files remain canonical.
- (B) OKF bundle = CANONICAL STORE (the inversion your step-4 implies — KH’s git tree becomes the source of truth, rewriting the “KH is not the storage layer” vision).
Recommendation: ratify (A), the two-layer derivation model. It resolves the conflation, keeps provenance/auditability (your #1 client need — feedback L39) anchored to the client’s real documents, makes the RAG-reduction claim true where it is true (curated layer), and does not require rewriting the vision. (B) is a bigger, riskier bet that throws away the “we don’t hold your data hostage / you keep your canonical files” positioning (principle 2, “helping users organise, not extracting their value”) — adopt it only with eyes open and a deliberate vision rewrite. The git working-tree can still be the substrate for the derived bundle under (A) — that is not the same as making it the client’s canonical store.
Net: OKF is the curated concept layer, not the corpus. Keep raw sources canonical; make the OKF bundle a governed derivation. The research doc’s tree should be split into “raw input corpus” (ID-127’s synthetic-corpus job) vs “OKF concept bundle” (the producer’s output).
3. Position on the 5 HYPOTHESES (H1–H5)
Section titled “3. Position on the 5 HYPOTHESES (H1–H5)”H1 — Is content_item vestigial / supersedable by source_documents + reference_items + q_a_pairs?
Section titled “H1 — Is content_item vestigial / supersedable by source_documents + reference_items + q_a_pairs?”Verdict: NUANCED — “displace, don’t delete.” Diagnosis HIGH confidence; the action is NEEDS-DECISION.
- Vestigial as a CONCEPT (HIGH):
content_itemis the IMS personal-knowledge-base fork bucket — schema-corroborated by 12 social/bookmark heritage columns (starred,platform varchar(30),author_name,captured_date,thumbnail_url,user_tags,priority… squash:270-300; I confirmedsource_url+platformadjacent at squash:270-272). It has been progressively hollowed out: Q&A →q_a_pairs(theanswer_standard/answer_advancedcolumns now vestigial at squash:311-312), external URLs →reference_items(id-110). It is conspicuously absent from the canonical glossary (platform-direction defines tenant/workspace/form/q_a_pair — not content_item; L7). OKF concept files + the existing typed peer tables cover its substance — OKF removes the last reason to keep a generic “content” abstraction as the knowledge substrate (L3). - Load-bearing as IMPLEMENTATION (HIGH): it is the single most-wired corpus table — ~191 TS
.fromsites across ~40 files + 109 Python touches (L1); the cocoindex content-branch write target; thecontent_chunksFK parent; thecitations.cited_content_item_idtarget (L4). H1 is a LARGE coordinated refactor, not a quiet drop. - No consumer OUTCOME contract is tied to the content_item CONCEPT (HIGH, L4): the contract is
the verbs (two-step
find→get→cite) + response shape + RLS scoping + citations — not the table. Guides,q_a_pairs,reference_itemsalready have independent consumer surfaces. - Three live roles must be RE-HOMED before removal (L5): (1) cocoindex content-branch write
target; (2) the ID-75 “adopted knowledge vs evidence” semantic distinction (
content_items= client-adopted,reference_items= evidence) — the one semantic OKF must explicitly preserve as a frontmattertype/layer; (3) thecontent_item_workspacesM2M anchor.
Action: treat content_item as a displacement candidate — re-home its three roles onto OKF
concepts, then retire the table as a migration. Do not decide “drop” this session (EXPENSIVE,
high-blast-radius). Freeze the verb contract, not the table. Tie the citation durable-target id
to the OKF record id (same key — see D5-CORRECTED) so cite_content survives the displacement.
H2 — Is workspace an unnecessary abstraction (vs activity / application IDs)?
Section titled “H2 — Is workspace an unnecessary abstraction (vs activity / application IDs)?”Verdict: NUANCED — workspace-table removal REFUTED; the M2M coupling is the real cut.
- Strict “drop workspace” is REFUTED (HIGH, L6): platform direction ratified the single
workspacestable +application_type_iddiscriminator (Q-OQR1-04 KISS) and rejected per-app top tables / dual-axis subtypes (04-workspace-types, 07-collapse-list §11). The glossary: workspace = “a container WITHIN one client’s database, bound N:1 to exactly one application_type.” Per-app tables are 1:1 satellites keyed byworkspace_id(squash:8007) — the inverse of your “procurement_workspaces is the activity and workspaces is a redundant parent.” - Your reframe is partly ALREADY TRUE (HIGH, L5/L6): id-120 (S391, Liam-ratified) already
establishes client = 1 DB = tenant; one workspace per application_type; the form/proposal is the
activity instance with its own ID. “Workspace = the activity-instance container” is the design —
your rename (
workspace→activity) is cosmetic and contradicts ratified naming. - The genuine cheap cut is
content_item_workspaces(HIGH): it has zero ingest writers (deliberately — flow.py:3071, verbatim confirmed), exactly one operator-curation writer, and is already bypassed for the two concrete cases you cite — forms bind directly viaform_templates.workspace_id NOT NULL; Q&A is corpus-level viascope_tagoverlap with no junction (L6). Replacing “membership = M2M rows” with “relevance = OKF tags / scope_tag overlap at read time” (the q_a_pairs model, already live) is a near-zero-cost pre-launch swap and lets ID-69’s unbuilt v1.1 M2M-writer be cancelled rather than built.
Action: keep workspaces; drop the content_item_workspaces coupling in favour of
query-time scope_tag overlap. Skip the rename.
H3 — Does OKF-bundle-in-git remove (lossy) RAG?
Section titled “H3 — Does OKF-bundle-in-git remove (lossy) RAG?”Verdict: NUANCED — RAG is REDUCED, not REMOVED. HIGH confidence.
- REDUCES (by design): for curated, stable, navigational knowledge (schemas, join paths, definitions, guides, canonical records) a cross-linked concept bundle replaces lossy chunk-RAG — an agent reads + updates concepts directly (L3). The platform already serves structured records (not raw chunk-RAG) via typed Postgres rows + MCP — so OKF-in-git changes the substrate, not the already-achieved structured-consumption principle (L7).
- Does NOT REMOVE (HIGH, L3/L4): (1) the bundle-exceeds-context-window problem returns retrieval
at scale — and no source gives the scale boundary (open question: does the SMB authoritative-doc
corpus reliably fit?); (2) fuzzy semantic answer in <30s (the O1/James requirement) still needs a
vector index — OKF ships none; (3) the raw long-tail (
reference_items) still warrants retrieval; (4) OKF has no machine-readable relationship type, no RLS/tenancy model — yourentity_mentions/scope_tag/RLS story stays richer and stays yours. - Consumer verdict (L4): HELPS more than it hurts. Keep a vector index over the OKF
.mdfiles (cocoindex already produces embeddings);findstays hybrid (semantic over the bundle);gettraverses links + returns verbatim.md. Sell OKF to consumers as “grounded + provenanced retrieval,” not “no more search.”
Action: answer your own parenthetical (“reducing (removing?)”) as reducing. Do not drop the vector index. The RAG-removal bet is the biggest risk in the whole reframe (§5).
H4 — CocoIndex-as-OKF-producer + ontology semantic-linter?
Section titled “H4 — CocoIndex-as-OKF-producer + ontology semantic-linter?”Verdict: SUPPORTED as the right architecture; NUANCED on cost (net-new, not a config tweak). Feasibility HIGH, effort MEDIUM.
- Endorsed as idiomatic and correct (L3): CocoIndex producer → OKF md tree → ontology semantic linter is the right shape.
- The linter is ~80% built (L6): reuse
extraction.py’sfield_validator+ Pydantic Literal gate; add an OKF write target downstream of the validated extraction. Preserve the HARD-reject vs SOFT-WARN split (block badtype/relation; observe-only on domain). Do not re-implement vocabulary governance. - The producer is NET-NEW but de-risked by precedent (L2/L5): every target today is
mount_table_target(Postgres); there is no file/markdown target. BUT the qa-sidecar already proves cocoindex can write a canonical.mdfile with DB rows as a derived projection in this exact pipeline (id-59) — id-59 is the literal template, not a blocker. Generalising the sidecar from Q&A-only to all concepts is the concrete path. Custom targets are buildable on cocoindex 1.0.7 (open question: confirm the stable custom-target API via the cocoindex skill). - Fix the source-of-truth fracture first (L6): consolidate the ontology markdown register (the 29
CV files live in docs-site;
canonical/docs/ontologyis empty;loader.tspoints at the empty dir) so the producer and the parity harness read one register.
Action: build as a dedicated spike (bl-367), reusing the validator gate + the id-59 sidecar
pattern + a new markdown custom-target. Adopt the reference_agent Pass-2 enrichment loop and
okf-skills /okf:validate as accelerators.
H5 — Reject the “no pipeline-logic change” constraint?
Section titled “H5 — Reject the “no pipeline-logic change” constraint?”Verdict: SUPPORTED — but re-purpose it as a task-scope fence, not a v1 lock. HIGH confidence.
- It is NOT a v1 architectural lock (HIGH, L7): it is an ID-127 {127.4} scope fence
(TECH.md:26-30). Pipeline-logic change is materially free pre-launch —
flow.py:1924(verbatim, confirmed): “prod has never completed a content write… Safe to change now.” - Lineage backs you (L5): the pipeline has already absorbed three structural reversals pre-launch — reference re-home (id-110/112), diff re-centre (id-117), sidecar promotion (id-59). Changing the producer to write OKF is in character, not unprecedented risk.
- Consumers are contract-neutral (L4): the substrate swap is invisible to find/get/cite and to headless goose if the verbs keep their shape — strongly favouring re-grounding the substrate now.
Action: reject it as an architectural/v1 lock; retain it as a deliberate fence so ID-127 {127.4} stays fast (scope_tags stay LLM-derived in v1). The real pipeline work (frontmatter parser, markdown target, durable-id seed, M2M-writer cancellation) lands in bl-367 / the new Task, sequenced against — not inside — the v1 cutover.
4. DECISION REGISTER — extending the research doc’s D1–D9
Section titled “4. DECISION REGISTER — extending the research doc’s D1–D9”The research doc’s D1–D4, D6 stand (low-risk / already-ratified). D5, D7, D8, D9 need correction or re-anchoring given the OKF-spec ground truth, and six NEW foundational decisions (a–f) are added. Reversibility flags: IRREVERSIBLE (pre-launch-free, frozen at ID-45 GO) / EXPENSIVE (large coordinated change) / CHEAP (reversible pre-launch at low cost).
4.1 Corrections to existing D-register
Section titled “4.1 Corrections to existing D-register”| # | Research doc said | CORRECTION | Reversibility |
|---|---|---|---|
| D5-CORRECTED | Seed identity on “OKF frontmatter record id” because OKF supplies a durable, location-independent id. | The premise is false. OKF is path-as-identity (Concept ID = path − .md; L3, verbatim from SPEC.md). Adopt a Canonical-owned custom frontmatter id (OKF’s open-key rule permits unknown keys) and own the uuid5 derivation across all three write paths (cocoindex Python rel_path seed, TS folder-drop destPath seed “INV-1”, upload). The decision stays D5(c)-shaped (durable id, not rel_path) but it is ours to define, and a builder must NOT default to OKF path-identity (that reproduces the fragility). | IRREVERSIBLE / pre-launch-free. The one decision that genuinely cannot wait — on the ID-45 critical path. |
| D7-RECONFIRMED | Author OKF-native now (authoring discipline only). | Still correct, with D5-CORRECTED’s caveat: the frontmatter id is a Canonical extension. Pipeline need not parse frontmatter to ship {127.4}. | CHEAP (authoring discipline). |
| D8-RECONFIRMED | local-fs now; Option-E later. | Correct. git-as-corpus does NOT contradict :41 (a git working-tree IS the local-fs folder cocoindex walks; commit/push/rollback is an additive operator layer). Bucket-as-source DOES contradict :41. Coolify-S3 is a git-remote/backup host, not the source connector. | CHEAP iff identity is durable-id-seeded (D5). |
| D9-RE-ANCHORED | Ratify corpus-structure + ingest-contract together (they share the seed). | Still true, but the shared decision is the seed grain (D5-CORRECTED), not “OKF gives us the id.” Freezing a rel_path-seeded corpus while the seed is open bakes in a migration liability. | — |
4.2 NEW foundational decisions
Section titled “4.2 NEW foundational decisions”| # | Decision | Recommendation | Reversibility |
|---|---|---|---|
| (a) | Keep / drop content_item | KEEP for v1; plan displacement post-v1. Re-home its 3 live roles (cocoindex write target; ID-75 adopted-vs-evidence semantic; M2M anchor) onto OKF concepts, then retire as a migration. Do not drop this session. | EXPENSIVE (191 TS sites + 109 Python; content_chunks FK; citations target; hybrid_search). Cheaper pre-launch but still large. |
| (b) | Keep / drop workspace abstraction | KEEP workspaces (ratified activity-instance entity); DROP the content_item_workspaces M2M coupling in favour of query-time scope_tag overlap. Cancel ID-69’s unbuilt v1.1 M2M-writer. Skip the workspace→activity rename (cosmetic, fights ratified naming). | M2M drop: CHEAP (zero ingest writers, near-empty). Workspace-table removal: EXPENSIVE + contradicts ratified direction — don’t. |
| (c) | Gate-keep authoritative-only vs ingest-all-then-sort | GATE-KEEP (folder-curation discipline). Aligns with OKF enrichment Pass-2, the guides coverage baseline, and the SMB “no data-governance staff” reality. Ingest-all-then-sort would need a net-new authority-classifier the pipeline has no primitive for. | CHEAP (operational discipline; reversible). A product call to ratify. |
| (d) | Substrate = local-fs vs git-repo vs object-storage | local-fs now (= :41), git working-tree as an additive operator layer for the derived bundle. Defer object-storage; ratify separately later. | CHEAP / reversible once identity is durable-id-seeded — the flip becomes a zero-migration mechanism swap. Spike the git-checkout-mid-walk hazard (a branch switch could look like a mass file-change → full re-extract/re-key). |
| (e) | OKF-bundle-in-git as canonical store + RAG-reduction | Ratify the TWO-LAYER DERIVATION model (option A, §2.3): OKF bundle = governed derivation over still-canonical raw sources; NOT the client’s canonical store. RAG: reduce for the curated layer, keep the vector index. This resolves the conflation and answers the vision §2.1 fork without a vision rewrite. | Architecture choice: EXPENSIVE to reverse once built. The (A)-derivation vs (B)-canonical-store fork is foundational — ratify the stance now even though the build is later. |
| (f) | CocoIndex-as-OKF-producer + ontology semantic-linter | PURSUE as bl-367 spike. Reuse the extraction validator gate (linter ≈80% built) + the id-59 sidecar precedent + a net-new markdown custom-target. Preserve HARD-reject/SOFT-WARN. Fix the ontology-register repo split first. | Decision to pursue: CHEAP (additive; DB targets stay). Build: EXPENSIVE (net-new custom target). Confirm cocoindex custom-target API via the cocoindex skill. |
5. RECOMMENDED PATH FORWARD (decisive)
Section titled “5. RECOMMENDED PATH FORWARD (decisive)”5.1 Reframe ID-127, or open a new foundational Task? — BOTH, split cleanly.
Section titled “5.1 Reframe ID-127, or open a new foundational Task? — BOTH, split cleanly.”The owner’s reframe is bigger than ID-127 (it touches content_item displacement, the OKF producer, the storage substrate, the RAG model, and the vision’s canonical-store fork). Cramming it into {127.4} would either bloat a synthetic-corpus task into an architecture epic or smuggle irreversible decisions in under a “no-pipeline-change” envelope.
Recommendation:
- Open ONE new foundational Task — “Canonical Knowledge Layer / OKF substrate” — owning new decisions (a)–(f) + D5-CORRECTED + the §2.3 two-layer ratification. This is the home for the architecture, the bl-367 producer spike, and the content_item displacement plan.
- Rescope ID-127 {127.4} to: author OKF-native synthetic files on local-fs, with the
Canonical custom frontmatter
idseed (the irreversible win, zero pipeline cost), a forms-only.kh-workspace-map.json, and an explicit “no content folder implies a workspace” gate pinned in PLAN S4. Explicitly defer the producer/substrate/RAG work to the new Task. ID-127 authoring is not blocked by the full reframe — only by the seed decision. - Keep bl-367 as the OKF producer/pipeline spike under the new Task.
5.2 Sequencing against goose G4 and the ID-45 v1 gate
Section titled “5.2 Sequencing against goose G4 and the ID-45 v1 gate”- ID-45 (Phew first-client re-ingest) is the v1 gate ({45.3}/{45.4} TECH/PLAN unwritten). The re-ingest mints the first permanent content rows, so the identity-seed grain HARDENS at ID-45 GO. D5-CORRECTED must precede ID-45’s first write — this pulls the seed decision onto the ID-45 critical path and likely re-opens {45.3}/{45.4} to encode the durable-id seed. This is the single sequencing constraint that bites.
- goose G4 (current branch, headless agent) is NOT blocked (L4). Headless consumes the same remote MCP server (find/get/cite); the OKF substrate swap is invisible to the consumer surface as long as the verbs keep their shape. G4 proceeds independently and in parallel. The OKF reframe is a substrate concern, not a consumer-surface concern — do not gate G4 on it.
- The new foundational Task runs in parallel with goose G4; its only hard handshake with ID-45 is the seed decision, which should be ratified before ID-45 authoring resumes.
5.3 The SMALLEST set to ratify THIS session (3 decisions)
Section titled “5.3 The SMALLEST set to ratify THIS session (3 decisions)”Ratify exactly these three; everything else can be spiked or deferred:
- D5-CORRECTED — the durable identity seed. Adopt a Canonical-owned frontmatter
id(not rel_path, not OKF path-identity), owned across all three write paths. IRREVERSIBLE, pre-launch- free, on the ID-45 critical path. This is the only decision that cannot wait. - (e) — the two-layer derivation stance. Ratify that the OKF bundle is a governed derivation over still-canonical raw sources (not the client’s canonical store). This resolves the conflation and frames every downstream build. Cheap to state now, expensive to reverse after building.
- (b) — drop the
content_item_workspacescoupling. Ratify query-time scope_tag overlap as the membership mechanism; cancel ID-69’s v1.1 M2M-writer. Low-risk, unblocks the corpus structure, and removes the weakest-defended abstraction.
5.4 Spike or defer (NOT this session)
Section titled “5.4 Spike or defer (NOT this session)”- (a) content_item displacement — spike the re-homing plan; do not decide “drop” (EXPENSIVE).
- (f) OKF producer build — bl-367 spike; confirm cocoindex custom-target API; fix the ontology-register repo split first.
- (d) object-storage substrate — defer; ratify separately once (e) and D5 are in.
- (c) gate-keep vs ingest-all — product call; ratify soon, but not blocking the seed.
workspace→activityrename — drop it (cosmetic, fights ratified naming).- Drop
content_items.source_url— sequence WITH the displacement, not before (live manual-path consumers; ast-dataflow column-reads sweep first).
6. BIGGEST RISKS
Section titled “6. BIGGEST RISKS”-
The RAG-removal bet (HIGHEST). If the platform commits to “OKF removes RAG” and drops the vector index, it breaks the O1 <30s fuzzy-semantic answer requirement and long-tail
reference_itemsretrieval. RAG is reduced, not removed — safe only for the curated concept layer that fits the agent context window, and no source gives the scale boundary (open question: do the SMB authoritative docs reliably fit?). Mitigation: keep a vector index over the OKF.mdbundle; treat OKF as “grounded retrieval,” not “no search”; empirically size the first client’s authoritative corpus before betting. -
Dropping
content_item(HIGH). High-blast-radius rewire — 191 TS sites,content_chunksFK,citations.cited_content_item_id,hybrid_search. A botched displacement breaks find/get/cite. The citation durable-target id must become the same key as the OKF record id (ties to D5) orcite_contentloses its anchor. Mitigation: displace (re-home roles), don’t delete; freeze the verb contract not the table; do it pre-launch when no prod rows exist. -
The D5 false-premise trap (HIGH, subtle). Ratifying the seed on “OKF gives us a durable id” leads a builder to adopt OKF path-identity (the spec’s actual behaviour) and reproduce the exact rel_path fragility the whole exercise tries to escape. Mitigation: explicitly define the Canonical custom
idextension and own its uuid5 derivation in lockstep across Python + TS (the seed lives in two languages — folder-dropdestPath“INV-1” vs Pythonrel_path). -
RLS / tenancy gap (MEDIUM, hard-blocks multi-tenant). OKF has no tenancy/authorization model; every MCP tool is per-user RLS-scoped. If the git bundle becomes the authorization boundary, multi-tenant security breaks. Mitigation: keep the DB/API as the auth layer over the bundle; single-tenant Phew launch tolerates the gap (mempalace direct-pattern precedent), but never assume the git tree is the security boundary.
-
The conflation re-entering the build (MEDIUM). If the team builds one tree that is both the raw corpus and the OKF bundle (the research doc’s collapse), the “self-updating wiki” round-trip becomes an operational hazard — a
git checkout/pullmid-walk can look like a mass file-change event → full re-extract/re-key. Mitigation: ratify the two-layer model (e); keep raw input and derived bundle as distinct trees; spike the git-mid-walk behaviour (d). -
Vision inversion shipped silently (MEDIUM). Your step-4 “single source of truth” quietly rewrites “KH is not the storage layer” (vision §2.1) and the “we don’t extract your value / per-tenant corpus” positioning (principle 2). Mitigation: if you ever choose (e)-option-(B) over (A), do it as a deliberate, documented vision rewrite — not as a side-effect of a corpus task.
7. Evidence index (decisive file:line, verified this session)
Section titled “7. Evidence index (decisive file:line, verified this session)”- Vision/substrate (the conflation + step-4 inversion):
platform-direction.mdv1 shape “the client’s source files are canonical; KH is a materialised index plus curated derivations”;01-vision.md §2.1“External-folder canonical … not the storage layer.” Glossary defines tenant/workspace/form/q_a_pair — content_item absent. (verified) - OKF path-as-identity (corrects D5): L3 — SPEC.md “Concept ID = the path … with the .md suffix
removed”; only
typeREQUIRED; links untyped; no tenancy model; v0.1 Draft. - Seed free now / IRREVERSIBLE:
flow.py:1922-1924(verified) “prod has never completed a content write… Safe to change now”; uuid5 onrel_path(flow.py:2110-2111). - M2M deliberately unpopulated:
flow.py:3068-3076(verified verbatim) — content branch is “CANONICAL, workspace-AGNOSTIC … M2M junction is DELIBERATELY not populated here.” Sole writerapp/api/items/[id]/workspaces/route.ts:154,183. - content_item IMS heritage + hollowing: squash:270-300 (12 personal-KB cols incl
source_url,platform— verified adjacent at 270-272); 311-312 vestigialanswer_standard/advanced; ~191 TS- 109 Python touches (L1).
- qa-sidecar precedent (H3/H4 template):
lib/q-a-pairs/sidecar-path.ts,_ingest_qa_sidecar_branch, id-59 PRODUCT “RATIFIED REVERSAL S380 … sidecar file canonical, DB rows derived projections” (L5). - Delta re-extraction (FB5):
extraction.py:196-213,757,797memo=true (L2). - Three write directions (FB6):
/api/uploaddirect writer vs folder-drop feeder vs cocoindex canonical (L2). - Ontology linter ≈80% built:
extraction.py:378-391(entity_type×12),:423-436(relationship×10),:470-481HARD-reject,:483-529SOFT-WARN; register repo-split —loader.ts→ emptydocs/ontology(L6). - Coverage/gaps live (FB9):
dashboard.ts:47-57five-layerwhere_are_we_exposed;templates.ts:182get_template_gaps(L7). - Consumer contract = verbs not table:
search.ts(find→hybrid_search over content_items),content.ts:63,215(get),procurement.ts:464-470(cite_content → cited_content_item_id); id-104 AgentEvalContractkind:'agent_recipe',grounding_shape:'citations'(L4). - v1 gate / ID-45:
v1-completion-sequence.md:17-33,66-68(v1.0.0 cut; ID-45 only gate; {45.3}/{45.4} unwritten) (L7).