ID-69 — Client corpus & AI-readiness onboarding model: cross-workspace canonical ingest
ID-69 — Client corpus & AI-readiness onboarding model: cross-workspace canonical ingest
Section titled “ID-69 — Client corpus & AI-readiness onboarding model: cross-workspace canonical ingest”{69.3} TECH artefact. This is the technical-design phase of the spec-driven
lifecycle: it translates the ratified {69.2} PRODUCT behaviour invariants
(BI-1..BI-13) into an implementation plan grounded in the verified codebase.
It does not decompose into Subtasks (that is {69.4} PLAN). Every Proposed
change maps one-to-one to a numbered PRODUCT invariant.
- Task: ID-69 “Client corpus & AI-readiness onboarding model — cross-workspace canonical ingest”.
- Predecessors (read in full):
{69.1}RESEARCH.md (S290, ratified),{69.2}PRODUCT.md (S291, ratified — 13 invariants BI-1..BI-13). - Authored: S291 (31/05/2026), fresh Planner instance (Q-PLANNER-2: not
the
{69.2}author), agent worktree off local branchcanonical-pipeline-setup. - Status: Draft, pending ratification. Liam ratifies this before
{69.4}PLAN. - Language: UK English throughout.
Context
Section titled “Context”What is being built
Section titled “What is being built”PRODUCT BI-1..BI-13 require the canonical corpus to gain a working
content_item_workspaces M2M junction-write path (currently absent on the ingest
side — RESEARCH §4), via an idempotent upsert keyed on the composite
(content_item_id, workspace_id), such that the same junction-write contract
serves both the v1 operator-side curation surface
(app/api/items/[id]/workspaces) and a v1.1 ingest-side declarative writer
unchanged (BI-3/BI-4 — the load-bearing constraint). v1 ships the operator-side
driver (already present) + the contract definition + tests, plus a client-readable
file-source AI-readiness contract (BI-9/BI-10), and states the ID-69 → ID-45/T7
gating relationship (BI-13).
Current state — verified by direct read (S291, canonical-pipeline-setup HEAD)
Section titled “Current state — verified by direct read (S291, canonical-pipeline-setup HEAD)”Code-intelligence orientation (mandatory per .gitnexus/CLAUDE.md; cited
verbatim, run S291):
gitnexus_query({query: "content_item_workspaces workspace association canonical ingest", repo: "knowledge-hub"})returned no canonical-corpus→workspace execution flow. The top-ranked processes were unrelated (proc_6_get“GET → PermanentJobError”,proc_99_get“GET → LoadAliases”,proc_100_get“GET → HasGarbledKeywords”);process_symbolssurfacedingest_file(flow.py:1044-1435),app_main(flow.py:1469-1807),ensureWorkspaceLink(pipeline.ts:548-568) andstoreAsContentItem(pipeline.ts:576-734) as the nearest definitions. The absence of a traced cross-workspace-ingest flow IS the finding — consistent with RESEARCH §3.1.gitnexus_context({name: "resolve_workspace"})— caller count: 1 non-test (ingest_file,flow.py) + 8 unit tests intest_workspace_resolver.py;outgoing: {},processes: []. Top-3 affected flows: none traced (the symbol participates in no indexed process). Verdict LOW structural risk; blast radius of touching it = the Path-B form-write block only.gitnexus_context({name: "ingest_file"})—incoming: {}(entry symbol; no graph callers — invoked by the cocoindex runtime), caller count: 0 indexed;outgoing.callsincludesresolve_workspace,embed_content_text,extract_classification,extract_qa_form,extract_entity_mentions,convert_binary_to_markdown,extract_source_provenance. Top-3 affected flows: none traced. Verdict LOW structural risk (no upstream callers); HIGH product significance (it is the canonical write path).gitnexus_context({name: "storeAsContentItem"})— caller count: 1 (processFeedSource);outgoing.calls:sb,inferContentType,ensureWorkspaceLink,normaliseUrl,classifyContent. Top-3 affected flows: the intelligence feed pipeline only. Verdict LOW — the precedent to mirror in shape, not to modify.gitnexus_context({name: "ensureWorkspaceLink"})— caller count: 1 (storeAsContentItem);outgoing.calls:sb. Top-3 affected flows: the intelligence feed pipeline only. Verdict LOW — read-only reference; ID-69 touches no existing TS junction writer.- ast-dataflow
column-writes --table content_item_workspaces(per RESEARCH §3.3, re-confirmed by grep S291) → the only production writers arelib/intelligence/pipeline.ts:565(.insert { workspace_id, content_item_id }) andapp/api/items/[id]/workspaces/route.ts:147,170(.insert { content_item_id, workspace_id }); the rest are tests/fixtures; none inscripts/cocoindex_pipeline/**(grep over the package confirmed zero junction references inflow.py). The TS operator-side writer set is unchanged-and-reusable. - ast-dataflow
column-writes --table source_documents --column workspace_id(per RESEARCH) → no canonical-path writer (BI-2 baseline).
Verified branch state of the resolver (direct read, NOT as RESEARCH-on-main
characterised it):
The current resolver is single-valued by design —
scripts/cocoindex_pipeline/workspace_resolver.py (read in full, S291):
resolve_workspace(manifest, rel_path) -> UUID(L145-186): longest-prefix-wins, returns exactly oneUUID; no-match or equal-length-tie raisesResolutionFailure(L50-61, L170-184).WorkspaceMapping(L69-75): one{path_prefix: str, workspace_id: UUID}pair — a single workspace per prefix.WorkspaceManifest(L78-106):schema_version: int+mappings: list[WorkspaceMapping];model_config = extra="forbid", frozen=True; rejects duplicatepath_prefix(L94-105).load_workspace_manifest(path) -> WorkspaceManifest(L114-142); raisesManifestLoadError(L41-47) on missing/unparseable/schema-invalid.__all__(L189-196): the public surface is exactly the above.
There is NO plural resolve_workspaces, NO workspace_ids: [...]
list-per-prefix, and NO from_raw string-or-list normaliser on this branch
(grep over scripts/cocoindex_pipeline/ and scripts/tests/ returned zero hits).
The multi-workspace carrier is not yet built — it is the v1.1 extension this
TECH designs (Q1 below), not existing scaffolding.
Path A still writes no junction (the gap is real, verified S291).
ingest_file (flow.py:1107) computes identity content_item_id = uuid.uuid5(_KH_PIPELINE_DOC_NS, f"ci:{rel_path}") (:1240) and declares
source_documents (sd_target.declare_row, :1247), content_items
(ci_target.declare_row, :1266 — id/content_text/embedding/source_document_id/
op_id/primary_domain/primary_subtopic; no workspace), then content_chunks,
q_a_extractions, entity_mentions. No declare touches
content_item_workspaces (grep: zero occurrences in flow.py). The
docstring’s Stage-6 comment (:1233-1238) confirms the deterministic-PK
re-ingest is an UPSERT (declare_row re-stamps the existing row, op_id is a plain
field NOT in the PK). resolve_workspace is called at :1433 — after the
content_items declare and gated behind if manifest is None: return
(:1422-1424) — and its single result feeds only
form_templates.workspace_id (:1473, :1542), never the junction.
source_documents.declare_row (:1247-1257) does not set workspace_id (the
source_documents provenance row carries no workspace).
No junction test coverage. scripts/tests/test_workspace_resolver.py (16
single-valued-resolver references) and scripts/tests/test_cocoindex_flow_write_path.py
(0 content_item_workspaces references) confirm neither exercises the junction.
The junction behaviour and its tests are the implementation gap PLAN must close.
In-repo precedent (the idiom to mirror in shape). ensureWorkspaceLink
(lib/intelligence/pipeline.ts:549-569) pre-checks the composite key
(.select('content_item_id').eq('workspace_id', …).eq('content_item_id', …) .maybeSingle(), :554-563), returns on hit (idempotent no-op, :564), and
inserts otherwise (:565-568). storeAsContentItem (pipeline.ts:577-735)
wraps it: one content_items row per identity, then ensureWorkspaceLink per
workspace (:615, :734). The operator API
app/api/items/[id]/workspaces/route.ts POST is the stronger idempotency
reference — its action === 'assign' path inserts { content_item_id, workspace_id } (:170-173) and explicitly treats Postgres 23505
unique-violation as a benign 409 (:176-181), which is the concurrency-safe
form the contract should adopt.
Most relevant files
Section titled “Most relevant files”scripts/cocoindex_pipeline/flow.py:1107(ingest_file),:1239-1240(identityuuid5),:1266(content_itemsdeclare — no workspace),:1422-1424(manifest gate),:1433(resolve_workspacecall → Path-B only),:1473/:1542(form_templates.workspace_id),:1635(app_main).scripts/cocoindex_pipeline/workspace_resolver.py:145(resolve_workspace, single-valued),:69-106(WorkspaceMapping/WorkspaceManifest),:114(load_workspace_manifest).scripts/cocoindex_pipeline/adapters.py:74-77— raise-on-unsupported (must stay unchanged in v1, BI-9).lib/intelligence/pipeline.ts:549-569—ensureWorkspaceLinkidiom.app/api/items/[id]/workspaces/route.ts:60-211— v1 operator surface (POST gated admin/editor:65; assign insert:170-173;23505-benign:176-181; GET viaget_item_workspacesRPC:21-57; unassign delete:189-193; create+assign insert:145-150).supabase/migrations/20260416102457_pre_squash_reconciliation.sql:3511—content_item_workspacestable; composite PK(content_item_id, workspace_id)(legacy constraint namecontent_item_projects_pkey,:4318); FK cascades to both parents (:5361,:5366); RLSciw_insertadmin/editor (:5958),ciw_selectUSING (true)(:5962),ciw_update/ciw_deleteadmin-gated (:5966/:5954).scripts/tests/test_workspace_resolver.py,scripts/tests/test_cocoindex_flow_write_path.py— Python test homes.
Central design — the driver-agnostic junction-write contract (BI-3 / BI-4)
Section titled “Central design — the driver-agnostic junction-write contract (BI-3 / BI-4)”This is the binding constraint per the PRODUCT handoff. The contract is defined once, in prose here, and is what both drivers target.
The contract (single source of truth)
Section titled “The contract (single source of truth)”
linkContentItemToWorkspace(contentItemId, workspaceId)— an idempotent upsert againstcontent_item_workspaceskeyed on the composite PK(content_item_id, workspace_id). Semantics: insert the pair; treat Postgres23505unique-violation as benign success (the pair already exists, or a concurrent writer won the race). Equivalent and acceptable: pre-check the composite key and no-op on hit (theensureWorkspaceLinkidiom). The operation never raises on a pre-existing pair and never creates a duplicate row. It writes onlycontent_item_workspaces— neversource_documents.workspace_id, never acontent_itemsworkspace column (there is none).
The 23505-benign form (operator route, route.ts:176-181) is the canonical
expression because it is also concurrency-safe; the pre-check form
(ensureWorkspaceLink) is the equivalent serial-path idiom. Both are the SAME
contract: one row per (content_item_id, workspace_id), idempotently.
Two drivers, one contract
Section titled “Two drivers, one contract”| Driver | Surface | Status | Targets the contract via |
|---|---|---|---|
| v1 — operator-side | app/api/items/[id]/workspaces POST (route.ts:60-211) | Ships in v1 | The route’s existing insert + 23505-benign block (:170-181) — already idempotent on the composite key, no contract change needed (see Q2). One pair per call. |
| v1.1 — ingest-side | flow.py Path A declares one junction row per id returned by a future multi-workspace resolver | Deferred (driver + multi-workspace carrier are v1.1; contract reused unchanged) | The same insert-with-23505-benign upsert, expressed as a cocoindex collector→export keyed on the composite PK, applied once per resolved id. |
Why the contract is provably reusable unchanged (BI-4)
Section titled “Why the contract is provably reusable unchanged (BI-4)”The contract is ((content_item_id, workspace_id) composite PK, idempotent-upsert
with 23505-benign). Neither driver may bake an assumption the other would have
to unwind:
- No surrogate-key dependence. The table has
id uuid DEFAULT gen_random_uuid()(:3511), but the contract keys on the composite PK (content_item_projects_pkey,:4318), never onid. Both drivers conflict on(content_item_id, workspace_id). The existing route already keys on the composite pair, notid— a v1-onlyid-based upsert would be a BI-4 violation, explicitly avoided. - No non-idempotent blind insert. The contract makes
23505benign, so neither driver assumes “row absent”. A v1.1 batch declare landing a pair the v1 operator already created is a no-op, not a duplicate or an error. - No driver-specific cardinality in the contract. v1 links one pair per HTTP call; v1.1 links N pairs from a resolver list — but each pair is the same single-pair contract applied N times. The multi-workspace cardinality lives in the v1.1 resolver/manifest (Q1), NOT in the contract. The contract is always single-pair.
- No workspace-on-record shortcut. Neither driver writes
source_documents.workspace_idor anycontent_itemsworkspace column (BI-1 / BI-2). Association rides the junction only.
Acceptance for BI-4: a reviewer reads this section, the route POST insert block
(route.ts:170-181), and the Q1 design for the v1.1 carrier, and confirms both
drivers target the identical composite key with identical upsert semantics — and
that nothing v1 ships requires rework to add the v1.1 collector.
Resolution of the three deferred mechanism questions
Section titled “Resolution of the three deferred mechanism questions”Q1 — Manifest carrier shape for the v1.1 ingest-side path (a vs b).
Choose (a): extend the existing manifest with a workspace_ids: [...] list per
prefix (extend WorkspaceMapping to carry a list; add a plural
resolve_workspaces(manifest, rel_path) -> list[UUID] alongside the retained
single-valued resolve_workspace shim for the Path-B form-write). This is a
v1.1 change — it does NOT exist on the branch today (the resolver is currently
single-valued; see Current state). Justification against BI-4:
- Shape (a) extends the manifest the flow already loads and mandates at
<COCOINDEX_SOURCE_PATH>/.kh-workspace-map.json(load_workspace_manifest,app_main:1635+) — no second loader, no second raise/abort path, no second manifest contract to keep in lock-step. The Path-B form-write keeps consuming a single workspace via the retainedresolve_workspaceshim (e.g.= resolve_workspaces(...)[0], or the existing single longest-prefix match), so (a) is strictly additive to the existing single-tenant form path — zero Path-B regression. - Shape (b) (a separate canonical-association manifest) would duplicate the
Pydantic schema, the
load_workspace_manifestloader, and theManifestLoadError/abort semantics, creating two manifest files an operator must keep consistent — a maintenance liability with no BI-4 benefit. The multi-workspace cardinality belongs in the value of an existing mapping, not in a new file. - Critically for BI-4: under (a), the v1.1 ingest driver consumes the resolver
list and calls the unchanged single-pair contract once per id. The contract
(composite-key,
23505-benign) is untouched whether the id came from an HTTP call (v1) or a manifest list (v1.1). The v1 operator route is provably reusable because it never sees the manifest at all — it only ever applies the single-pair contract.
Schema-version note (forward-compat). When (a) lands in v1.1, bump
WorkspaceManifest.schema_version from 1 to 2 and accept workspace_id (single,
v1) OR workspace_ids (list, v2) per mapping, so existing v1 manifests keep
loading. The extra="forbid" config means a new field requires a schema bump —
this is the v1.1 spec’s concern; v1 must not pre-build it (BI-4: don’t ship v1.1
in v1).
Q2 — v1 operator-surface exactness. The existing
app/api/items/[id]/workspaces route is sufficient for BI-3 as-is; no batch or
idempotency hardening is required for v1. Verified by direct read
(route.ts:60-211): the POST handler (i) is admin/editor-gated via
getAuthorisedClient(['admin', 'editor']) + authFailureResponse(auth)
(:65-66), (ii) validates the item id as a UUID (:70-75), (iii) on action === 'assign' inserts { content_item_id, workspace_id } (:170-173), (iv) treats
23505 as a benign 409 (:176-181) — idempotent on the composite key, (v)
supports unassign via delete (:189-193) and create+assign (:80-160), and (vi)
GET lists workspaces via the get_item_workspaces RPC (:21-57). This is the
BI-3 contract exactly.
Batching is explicitly out of v1 scope — multi-id declare is the v1.1
ingest-side concern (the Q1 resolver list), not an operator-API change. Adding a
batch endpoint now would be speculative v1.1 surface (BI-4: don’t build v1.1 in
v1). One observation for the Checker: the route’s assign path uses the raw
client’s .insert() directly rather than sb()/tryQuery() from
@/lib/supabase/safe (it does check error.code, so it is not silent), and
ensureWorkspaceLink uses sb() only for the pre-check select, not the insert.
v1’s only operator-side deliverable is a test asserting the contract
(idempotency + role gate); a sb()-hardening tidy of the route’s insert is an
optional, non-load-bearing follow-up (noted under Follow-ups), not a BI-3
requirement.
Q3 — Re-ingest association preservation (BI-6). Automatic via the stable
uuid5 identity; no explicit guard required — but mandate a regression test.
Justification: content_item_id = uuid.uuid5(_KH_PIPELINE_DOC_NS, f"ci:{rel_path}")
(flow.py:1240) is deterministic in rel_path, so a changed-bytes re-ingest
re-stamps the same content_item_id (the Stage-6 comment at :1233-1238
states this explicitly: declare_row UPSERTs the existing row, op_id is not in
the PK). Junction rows FK to content_items.id ON DELETE CASCADE (:5361) and are
keyed on (content_item_id, workspace_id); a content_items UPSERT
(deterministic PK) leaves the parent row’s id unchanged, so existing junction
rows are never orphaned or cascaded away. The load-bearing precondition is that the
canonical re-ingest declares content_items as an UPSERT, not a DELETE+INSERT
— which is exactly how declare_row with a deterministic PK behaves. Because an
accidental delete-and-reinsert would silently violate BI-6 via FK cascade, TECH
mandates a Python regression test proving the parent row is UPDATEd (not
deleted) and the junction count is invariant across re-ingest. The guard is
therefore a test assertion on the upsert discipline, not runtime code.
Proposed changes (one-to-one against PRODUCT BI-1..BI-13)
Section titled “Proposed changes (one-to-one against PRODUCT BI-1..BI-13)”Each row maps to exactly one invariant. “Verify” = how the change is proven (detailed in Testing).
BI-1 — Canonical record has no intrinsic workspace; association is M2M only
Section titled “BI-1 — Canonical record has no intrinsic workspace; association is M2M only”- Change: None to schema (
content_itemsalready has noworkspace_idcolumn). Confirm the ingest declarescontent_itemsindependently of workspace resolution (it does — declare atflow.py:1266;resolve_workspaceruns later at:1433for Path B only), and that a record with zero junction rows is a complete record. - Files:
flow.py:1266(declare, no workspace dependency);supabase/migrations/…:3511(schema check, read-only). - Verify: schema assertion (no
content_items.workspace_id); a canonical record queried back with zero junction rows still hascontent_text,embedding,source_document_id.
BI-2 — Workspace association written ONLY to content_item_workspaces, never source_documents
Section titled “BI-2 — Workspace association written ONLY to content_item_workspaces, never source_documents”- Change: None (negative invariant). Confirm the
source_documentsdeclare (flow.py:1247-1257) does not setworkspace_id; ast-dataflow confirms no TS writer. - Files:
flow.py:1247-1257(source_documentsdeclare —workspace_idnot set);lib/**/app/**(no writer). - Verify: ast-dataflow
column-writes --table source_documents --column workspace_idshows no canonical-path writer; Python integration assertion thatsource_documents.workspace_id IS NULLpost-ingest while the junction holds the pair.
BI-3 — Junction-write contract: idempotent upsert on the composite key
Section titled “BI-3 — Junction-write contract: idempotent upsert on the composite key”- Change: (v1, operator-side) none — the route POST block already
implements the contract (insert +
23505-benign,route.ts:170-181; Q2). (v1.1, ingest-side — contract only, driver deferred) the contract is documented once (Central design) and the future cocoindex collector→export will reuse it. v1’s deliverable is the test proving idempotency. - Files:
app/api/items/[id]/workspaces/route.ts:170-181(v1 driver);lib/intelligence/pipeline.ts:549-569(precedent); Central design section. - Verify: call the operator association twice for the same pair → row count = 1, second call benign (409, no duplicate, no unhandled error).
BI-4 — Junction-write contract identical across v1 and v1.1 drivers
Section titled “BI-4 — Junction-write contract identical across v1 and v1.1 drivers”- Change: Document the contract once (Central design); specify that the v1.1
carrier (Q1, shape (a)) feeds the same single-pair contract N times via a
future
resolve_workspaceslist, NOT a bespoke multi-row writer or a surrogate-key upsert. - Files: Central design section; Q1 (v1.1 carrier design);
route.ts:170-181(v1 driver, present). - Verify: reviewer confirms both drivers target
(content_item_id, workspace_id)with idempotent-upsert +23505-benign; no surrogate-key / blind-insert / workspace-on-record assumption baked in (see BI-4 sub-bullets).
BI-5 — A canonical record can be associated to MANY workspaces
Section titled “BI-5 — A canonical record can be associated to MANY workspaces”- Change: The schema already supports many junction rows per
content_item_id(M2M,:3511). v1 composes many associations via repeated operator POST (one pair per call). v1.1 adds theworkspace_ids: [...]carrier + pluralresolve_workspaces(Q1) so a single ingest declares many pairs — deferred, not built in v1. - Files: schema
:3511(M2M);route.ts:170-173(repeated POST, v1); Q1 (v1.1 plural carrier, deferred). - Verify: a single
content_item_idholds ≥2 junction rows (distinctworkspace_ids) after two operator POSTs; adding a second leaves the first intact; GET (route.ts:21-57) returns all.
BI-6 — Re-ingest re-stamps identity and preserves associations idempotently
Section titled “BI-6 — Re-ingest re-stamps identity and preserves associations idempotently”- Change: Confirm
content_itemsis declared as a deterministic-PK UPSERT (not DELETE+INSERT) so the parentidis stable and FK-cascade does not orphan junction rows (Q3 — this is the existingdeclare_rowbehaviour at:1266). No new runtime code; add the regression test. - Files:
flow.py:1240(identity),flow.py:1266(content_itemsdeclare/upsert). - Verify: Python integration test — ingest, associate (operator), re-ingest
(changed bytes);
content_item_idunchanged,content_itemsrow UPDATEd not duplicated/deleted, junction row count invariant.
BI-7 — Base-path stability is part of the canonical contract
Section titled “BI-7 — Base-path stability is part of the canonical contract”- Change: Documentation only — state base-path (
COCOINDEX_SOURCE_PATH) stability as an operator obligation in the readiness contract (BI-10 doc), with the identity-fork consequence of drift. No code (the stable corpus volume is ID-66’s concern, referenced not owned). Note the manifest also lives atsource_path/.kh-workspace-map.json, so drift breaks both identity and manifest resolution. - Files: readiness contract doc (new, see BI-10).
- Verify: reviewer points to the stated obligation in the readiness doc (no code assertion).
BI-8 — Association is explicit, never inferred from folder layout or classification
Section titled “BI-8 — Association is explicit, never inferred from folder layout or classification”- Change: None (negative invariant). The manifest prefix→workspace mapping is
an explicit operator declaration; classification (
primary_domain/primary_subtopic,flow.py:1278-1279) comes from the LLM, not the folder, and never maps to workspace. Confirm no folder-path→workspace or classification→workspace code path is introduced. - Files:
flow.py:1278-1279(classification declare — no workspace coupling);workspace_resolver.py(explicit manifest mapping only). - Verify: review/test confirms no implicit folder-or-classification-to- workspace mapping; association rows exist only from explicit manifest/operator assertion.
BI-9 — Curated-corpus discipline: clean staged folder is the v1 contract (no ingest filter)
Section titled “BI-9 — Curated-corpus discipline: clean staged folder is the v1 contract (no ingest filter)”- Change: Documentation only — readiness contract states the
clean-staged-folder operator obligation and that v1 adds no ingest-side
include/exclude filter.
adapters.py:74-77raise-on-unsupported stays unchanged; the contract notes the v1.1 skip-not-raise upgrade without implementing it. - Files: readiness contract doc (BI-10);
adapters.py:74-77(unchanged — reviewer confirms no allowlist/skip added). - Verify: reviewer confirms the stated obligation + that raise-on-unsupported is unchanged for v1; no ingest-side allowlist/skip code.
BI-10 — AI-ready canonical-form readiness model is a client-readable contract (not a UI)
Section titled “BI-10 — AI-ready canonical-form readiness model is a client-readable contract (not a UI)”- Change: Author a new client-readable readiness checklist for file sources — what enters the corpus, in what shape, what a client must do to a plain-file source before staging. No UI deliverable; explicitly defers database sources and onboarding UX to v1.1.
- Files: new doc, e.g.
docs/specs/ID-69-client-corpus-onboarding/READINESS.md(client-readable, distinct from internal ingest mechanics). PLAN sets the exact path/filename. - Verify: reviewer confirms the checklist is consumable by a non-developer client admin, contains no UI/implementation scope, and defers DB sources + UX to v1.1.
BI-11 — Association is dedup-agnostic in v1
Section titled “BI-11 — Association is dedup-agnostic in v1”- Change: None (negative invariant). The junction write is purely
additive/idempotent; no v1 code path merges
content_itemsrows or removes a record on cross-workspace duplication. - Files:
route.ts:170-181(additive only);flow.py(no junction merge). - Verify: reviewer confirms no dedup/merge behaviour introduced that a v1.1 dedup feature would have to re-architect.
BI-12 — Operator-side association is admin/editor-gated (RLS-coupled)
Section titled “BI-12 — Operator-side association is admin/editor-gated (RLS-coupled)”- Change: None to the route (already gated). Confirm the operator POST uses
getAuthorisedClient(['admin', 'editor'])({ success }) +authFailureResponse(auth)(route.ts:65-66) and that the write rides theciw_insertadmin/editor RLS policy; reads viaciw_select(USING (true)). - Files:
app/api/items/[id]/workspaces/route.ts:65-66(auth gate); migration:5958(ciw_insert),:5962(ciw_select). - Verify: admin/editor can associate; reviewer/viewer rejected at auth/RLS;
test asserts the role gate. (Note:
content_item_workspaceshasGRANT ALLtoanon/authenticated/service_roleat:8301-8303, but RLS is ENABLED (:6029) so theciw_insertpredicate is the effective gate — the role check is genuine, not bypassed by the grant.)
BI-13 — ID-69 gates ID-45/T7 (sequencing invariant)
Section titled “BI-13 — ID-69 gates ID-45/T7 (sequencing invariant)”- Change: State the gating relationship in TECH (here) and surface to the
Orchestrator. No code; the
dependenciesedit on the ledger is the Orchestrator’s action. - Files: this TECH;
docs/reference/task-list.json(Orchestrator edits T7’sdependenciesto add ID-69 — not done here). - Verify: Checker confirms the ID-69 → ID-45/T7 gating is stated (in PRODUCT BI-13 and here), not that the ledger is already edited.
Migration plan
Section titled “Migration plan”No DDL migration is required. content_item_workspaces already exists with
the correct shape: composite PK (content_item_id, workspace_id) (:3511,
:4318), FK cascades to both parents (:5361, :5366), RLS ENABLED (:6029)
with ciw_insert (admin/editor, :5958) + ciw_select (USING (true),
:5962). BI-1 (no content_items.workspace_id), BI-2
(source_documents.workspace_id stays NULL) and BI-12 (RLS gate) are satisfied
by the existing schema — TECH adds no columns, no functions, no grants.
Consequently no new REVOKE EXECUTE … FROM anon and no SET search_path clause
are in scope (no new PL/pgSQL functions). The GET path’s get_item_workspaces RPC
already exists.
The v1 implementation is therefore operator-driver tests + Python upsert- discipline tests + one doc, not schema or new feature code:
- TypeScript (operator API): no code change; ships the contract test
(idempotency + role gate, BI-3/BI-12). Optional follow-up:
sb()-harden the route insert (non-load-bearing). - Python (cocoindex): no v1 runtime change to the junction path (the v1
driver is operator-side). Ships the re-ingest upsert-discipline regression
test (BI-6) and the BI-1/BI-2 ingest assertions. The v1.1 ingest-side
junction collector + the
workspace_idscarrier (Q1) are deferred. - Docs: the client-readable readiness contract (BI-7/BI-9/BI-10).
Testing and validation
Section titled “Testing and validation”Tests verify real behaviour, not implementation (per
docs/reference/test-philosophy.md). Python: python3 -m pytest scripts/tests/.
TypeScript: bun run test (not bun test).
| Invariant | Test / verification |
|---|---|
| BI-1 | Schema assertion: content_items has no workspace_id. Python integration: canonical record with zero junction rows is complete (content_text, embedding, source_document_id present). |
| BI-2 | ast-dataflow column-writes --table source_documents --column workspace_id = no canonical writer (regression). Python integration: source_documents.workspace_id IS NULL while junction holds the pair. |
| BI-3 | TS operator-route test (__tests__/): associate the same (content_item_id, workspace_id) twice → row count = 1, second call benign (409, no duplicate, no unhandled error). |
| BI-4 | Review checklist (no automated test): both drivers target the composite key with idempotent-upsert + 23505-benign; no surrogate-key/blind-insert/shortcut. Asserted against route.ts:170-181 + Q1 carrier design + Central design. |
| BI-5 | Integration: two operator POSTs link one content_item_id to ≥2 distinct workspaces; second leaves the first intact; GET returns both. (v1.1 plural-resolver tests are out of v1 scope.) |
| BI-6 | Python integration in test_cocoindex_flow_write_path.py: ingest → associate (operator) → re-ingest (changed bytes); assert content_item_id unchanged, content_items UPDATEd (not duplicated, not delete+reinsert), junction count invariant. Critical: assert the parent row was UPDATEd, catching an accidental delete-and-reinsert that would cascade the junction. |
| BI-7 | Doc review: readiness contract names base-path stability as an operator obligation + identity-fork consequence. |
| BI-8 | Review/test: no folder-path→workspace or primary_domain/primary_subtopic→workspace mapping; association rows only from explicit manifest/operator assertion. |
| BI-9 | Doc review + code review: clean-staged-folder obligation stated; adapters.py:74-77 raise-on-unsupported unchanged; no allowlist/skip added in v1. |
| BI-10 | Doc review: client-readable readiness checklist for file sources exists, no UI, defers DB sources + UX to v1.1, consumable by a non-developer admin. |
| BI-11 | Code review: no merge/dedup/remove on cross-workspace duplication during association; junction write additive/idempotent only. |
| BI-12 | TS test (operator route): admin/editor associate succeeds; reviewer/viewer rejected at auth/RLS; uses getAuthorisedClient(['admin','editor']) + authFailureResponse(auth). |
| BI-13 | Checker confirms the ID-69 → ID-45/T7 gating is stated here + in PRODUCT BI-13; Orchestrator owns the ledger dependencies edit. |
Risks and mitigations
Section titled “Risks and mitigations”- Re-ingest cascade silently drops associations (BI-6). If a future change
declares
content_itemsas DELETE+INSERT instead of deterministic-PK UPSERT, the FKON DELETE CASCADE(:5361) silently removes every junction row for that record. Mitigation: the BI-6 regression test asserts the parent row was UPDATEd and the junction count is invariant — it fails loudly on a delete-and-reinsert regression. - Manifest still mandatory and aborts the whole flow (RESEARCH §5.3). A
missing/unparseable
.kh-workspace-map.jsonraisesManifestLoadError(workspace_resolver.py:41) atapp_main’sload_workspace_manifestcall and aborts the run. v1 does not change this (out of scope per PRODUCT Non-goals); the readiness contract must state the manifest as a required operator artefact so a clean-corpus run does not abort. (Note:ingest_fileitself early-returns when no manifest is bound at:1422-1424, so Path-A content still declares — butapp_mainresolves the manifest at flow start.) - Single-valued resolver today; multi-workspace is v1.1. The branch resolver returns ONE workspace per file. v1 reaches “many workspaces per record” only via repeated operator POSTs (BI-5). The N-per-ingest path needs the Q1 carrier (shape (a)) which is deliberately deferred — PLAN must not schedule it under v1 (BI-4 boundary), and the readiness contract must not promise ingest-side multi-workspace in v1.
- Operator route insert bypasses
sb(). The assign insert (:170-173) uses the raw client with manualerror.codehandling (it is not silent, but notsb()-wrapped). Non-load-bearing for BI-3; flagged as an optional Follow-up tidy. - Proxy
publicRoutes. No new public API route is added (the operator route already exists and is auth-gated) — noproxy.tsallowlist change needed.
Follow-ups
Section titled “Follow-ups”sb()/tryQuery()-harden the operator route inserts (route.ts:170-173,:145-150) for consistency with@/lib/supabase/safediscipline. Optional, non-load-bearing for BI-3.- v1.1 ingest-side junction driver +
workspace_idscarrier (Q1, shape (a)) — extendWorkspaceMapping/WorkspaceManifest(bumpschema_versionto 2), addresolve_workspaces -> list[UUID], declare one junction row per resolved id inflow.pyPath A using the unchanged contract. Tracked under the v1.1 deferral register (docs/themes/canonical-pipeline/reference/deferral-register.md, OQ-69-1 option a); NOT an ID-69 v1 Subtask.
Verification (Q-EX2 — external-library API empirical check)
Section titled “Verification (Q-EX2 — external-library API empirical check)”This TECH cites the external library cocoindex only as current-state description of the existing flow; no new cocoindex API is introduced by ID-69 v1 (the v1 driver is operator-side TS + tests + a doc; the Python side ships tests, not new cocoindex calls).
| Item | Value |
|---|---|
| Date | 31/05/2026 |
| Pinned version | cocoindex[postgres]==1.0.3 (requirements.txt) |
| Installed version | cocoindex 1.0.3 (verified S290 RESEARCH §9; pin unchanged on this branch) |
| Symbols cited | None new. The v1.1 collector→export (deferred) is the only place a new cocoindex call would appear; v1 introduces none. |
| Result | N-A for v1. No new external symbol is called by v1. The resolver, manifest models (Pydantic v2), and route are KH-internal (Python + TS) — covered by gitnexus/ast-dataflow/grep above, outside Q-EX2 scope. No ABSENT / SIGNATURE_DRIFT. |
Pydantic (BaseModel, ConfigDict, model_validator) is used by the existing
resolver and is not modified by v1; no new Pydantic API is introduced.
Notes for {69.4} PLAN
Section titled “Notes for {69.4} PLAN”- Decomposition warranted (compound + chain-dependent). Distinct slices:
(1) operator-route contract test (TS — BI-3/BI-12, independent); (2) re-ingest
upsert-discipline regression test (Python
test_cocoindex_flow_write_path.py— BI-6, independent of the TS slice); (3) BI-1/BI-2 ingest + ast-dataflow regression assertions (independent); (4) readiness contract doc (BI-7/BI-9/BI-10, independent). Optional follow-up slice:sb()-harden the route inserts. Estimated effort 2-4h (tests + one doc; no schema, no new feature code on the v1 path — the v1 junction write already exists operator-side). - Sibling-only deps hold — all slices are within ID-69; no cross-Task Subtask dependency surfaced. (BI-13 is a Task-level dep the Orchestrator adds to T7, not a Subtask dep.)
- BI-13 ledger action for the Orchestrator: add ID-69 to ID-45/T7’s
dependencies. - v1.1 driver is explicitly NOT a v1 Subtask — the ingest-side junction
collector→export + the
workspace_idscarrier (Q1, shape (a)) are deferred; PLAN must not schedule them under ID-69 v1 (BI-4 boundary). The branch’s resolver is single-valued today; multi-workspace-per-ingest is v1.1.
Changelog
Section titled “Changelog”| Date | Session | Note |
|---|---|---|
| 31/05/2026 | S291 | {69.3} TECH authored by a fresh Planner against verified canonical-pipeline-setup HEAD. One-to-one Proposed-changes mapping to BI-1..BI-13. Resolved the 3 deferred mechanism questions: (Q1) v1.1 manifest carrier shape (a) workspace_ids: [...] extending the existing single-manifest loader — deferred, not yet built (branch resolver resolve_workspace is single-valued today, returns one UUID); (Q2) v1 operator route app/api/items/[id]/workspaces is sufficient as-is for BI-3 (insert :170-173 + 23505-benign :176-181, admin/editor-gated :65 — no batch/idempotency hardening); (Q3) re-ingest preservation is automatic via stable uuid5 identity (flow.py:1240) + declare_row UPSERT discipline, guarded by a regression test (not runtime code). Central BI-3/BI-4 driver-agnostic contract section. No DDL migration (schema already correct; PK :4318, FK cascades :5361/:5366, RLS :5958/:5962). |