PRODUCT — ID-127: Platform ingestion pipeline standup (Platform-owned cocoindex E2E rig, upstream of client deploys)

PRODUCT — ID-127: Platform ingestion pipeline standup

{127.2} PRODUCT artefact. This is the product-behaviour phase of the spec-driven lifecycle: it states what the Platform ingestion pipeline standup must do, as numbered, testable Behavior invariants a Checker can verify against acceptance criteria. It does not decide implementation mechanism (that is {127.3} TECH) and does not decompose into Subtasks (that is {127.4} PLAN).

Task: ID-127 “Platform ingestion pipeline standup” (promoted from backlog bl-301).
Predecessor: {127.1} RESEARCH.md — ported from specs/bl-301-platform-pipeline/RESEARCH.md (S384). Read it for the verified current-state gap analysis and the env/secrets/webhook matrices. Three RESEARCH premises were corrected after authoring (see “Corrected model” below) — this PRODUCT encodes the corrected model, not the stale RESEARCH premises.
Authoritative grounding: ${KH_PRIVATE_DOCS_DIR}/src/content/docs/reference/platform-direction.md (entity glossary — workspace ≠ client/tenant, verified S391) wins on any entity-definition conflict, including against older tasks (esp. ID-69) that use the legacy “workspace = tenant” equation.
Authored: 23/06/2026, agent worktree (task-planner, fresh context per Q-PLANNER-2).
Status: Draft, pending ratification. Liam ratifies the open decisions (collected at the end) before {127.3} TECH.
Language: UK English throughout. Dates DD/MM/YYYY.

Summary

Stand up the Platform’s own cocoindex ingestion pipeline — a Platform-owned deployment that ingests our own synthetic dev/test corpus into the Platform DB (zjqbrdctesqvouboziae), giving us a controlled, full end-to-end pipeline configuration that is the UPSTREAM of client deployments. The artefact being developed and tested here — the deploy shape (compose + env keyset + pinned image tag), the source-binding model, and the in-app ingestion seams — is precisely what gets promoted out to client configs. This standup is also the opportunity to rebuild the Coolify topology for extensibility (correct project / server / source / destination conventions, kh→ca / knowledge-hub→canonical renaming) that the first-client setup did not have, and to adopt the cocoindex-based URL-extraction replacement for the now-retired PullMD stack.

Problem

The Platform DB is a full dev instance (per ID-108) but has no ingestion compute pointed at it. The only cocoindex deployments that exist are the two client (Phew) Coolify apps on the IONOS host (77.68.122.71) — kh-onprem-pipeline-production (uuid ybiczck7f7e1xbdev3bk89cr, branch release/v1.0.0) and kh-onprem-pipeline-staging (uuid ljurtiab99kb6dst38k76419, branch staging). Neither writes the Platform DB. There is no Platform Coolify app, no docker-compose.platform.yaml, no Platform deploy job, and no defined Platform corpus. That absence is the ID-127 gap.

Without a Platform-owned pipeline we have no faithful upstream on which to develop and test the whole pipeline configuration before it is pushed to clients — a local-run-only path never exercises the Coolify deploy, Traefik routing, the PIPELINE_RUN_WEBHOOK_URL round-trip, or the app→worker ingestion seams that are the actual promote artefact.

Code-intelligence orientation (cited verbatim per .gitnexus/CLAUDE.md “Always Do” and .claude/agents/references/shared-discipline.md §Code-intelligence discipline; run against HEAD on 23/06/2026):

gitnexus_query({query: "cocoindex ingestion pipeline platform deploy COCOINDEX_SOURCE_PATH localfs source", repo: "canonical"}) returned the worker flow definitions verbatim: scripts/cocoindex_pipeline/flow.py (app_main L3300–3835; _ingest_content_branch L2022–2444; _build_db_ctx L926–948), scripts/cocoindex_pipeline/server.py, the operational sidecar deploy/onprem/monitor/datapath-watch.ts (loadConfig, isIdle), and scripts/cocoindex_pipeline/url_source.py (FeedUrlSource.items). The pipeline IS in-repo and complete; the gap is a Platform-targeted deployment + corpus, not new pipeline code. (gitnexus: flow proc_84_post family; corpus binding in Cocoindex_pipeline module.)
gitnexus_context({name: "app_main", repo: "canonical", file_path: "scripts/cocoindex_pipeline/flow.py"}) — app_main (L3300–3835) calls load_workspace_manifest (workspace_resolver.py:195–223) and FeedUrlSource.items (url_source.py). It has zero incoming callers in the graph (it is the flow entrypoint invoked by cocoindex’s runtime, not by in-repo code). Verdict: the Platform pipeline reuses this entrypoint unchanged — additive deployment, LOW structural risk to the worker; the design surface is config + corpus + topology.
gitnexus_context({name: "load_workspace_manifest", repo: "canonical", file_path: "scripts/cocoindex_pipeline/workspace_resolver.py"}) — the manifest loader (L195–223) is called by app_main and exercised by ~16 tests (test_workspace_resolver.py, test_cocoindex_flow_fork_routing.py, test_cocoindex_server.py). It loads a WorkspaceManifest keyed on path_prefix → workspace_id (longest-prefix-wins, optional route), raising ManifestLoadError on a missing/invalid file; UUIDs are FK-checked at INSERT time, not load time. This is the existing files→workspaces binding the Platform corpus reuses — the .kh-workspace-map.json manifest (to be renamed .ca-workspace-map.json, BI-7).

Verified live facts (RESEARCH S384 + Coolify recon S402, read-only):

COCOINDEX_SOURCE_PATH resolves to /cocoindex-state/corpus (a subdir of the cocoindex-state named volume) in BOTH live client apps; flow.py reads it via localfs.walk_dir(...), server.py /stage+/walk read it (loud-400 if unset), datapath-watch.ts reads it to detect idle boot. The corpus root is a single source-of-truth path; there is no client/platform fan-out in the deployed config today.
local-fs-client/ and local-fs-platform/ exist as top-level local source folders (under ~/Documents/development/). local-fs-client/corpus/{markdown,binary-originals} already holds the real Phew content (the ID-45 reingest set). local-fs-platform/corpus is empty — the synthetic Platform corpus does not yet exist. Neither folder carries a workspace-map manifest yet.
ID-113 release versioning has shipped: onprem-deploy.yml’s tag-release job (L546–593) aliases a known-good sha- cocoindex image to a vX.Y.Z tag without a rebuild. The promote model’s image-tag substrate is therefore already in place.

Goals / Non-goals

Goals

Give the Platform DB a cocoindex ingestion path of its own that mirrors the client deploy shape closely enough to be a promotable, faithful E2E upstream.
Use a Platform-owned synthetic corpus (local-fs-platform/corpus) that we control, mirroring the kinds of content clients upload — never client data, never a sanitised copy of client fixtures (re-introduces the ID-68 IP-separation concern).
Rebuild the Coolify topology for extensibility: correct, future-proof project/server/source/destination conventions and the kh→ca / knowledge-hub→canonical rename, applied to the Platform standup and (where the rename is safe to land in the same pass) the existing client config.
Adopt the cocoindex-based URL-extraction replacement for the retired PullMD stack — the Platform stack carries no pullmd/playwright services.
Treat backup and monitoring as non-optional standard parts of the standup, not low-priority extras.
Formalise the promote mechanism (Platform-proven config → client config) as a first-class behaviour, tied to ID-113’s release-cut model.

Non-goals (explicit)

Building the cocoindex pipeline itself. The 6-stage flow, the worker HTTP server, the manifest resolver, the URL source, and the in-app ingestion seams all exist in-repo and are unchanged by ID-127. This Task stands up a deployment + corpus + topology, not new pipeline logic. Any pipeline-logic change surfaced mid-build is an escalation, not in-scope work.
Curating a client-volume corpus. The Platform corpus is sized for fast E2E iteration, not coverage of a real client’s volume.
The vestigial client PullMD deletion. The deployed client compose still runs pullmd/playwright vestigially pending {112.13}; ID-127 does not carry that into the Platform stack and does not own the client deletion.
Changing the cocoindex write-model, RLS, or schema. Write-model prerequisites (service-account row, auth.users triggers, user_profiles mirror, role-based RLS) are verified, not authored, by ID-127 (BI-10).
Migrating the workspace-association ingest gap (ID-69 scope). ID-127 reuses the existing single-workspace_id-per-prefix manifest binding; the content_item_workspaces M2M ingest writer is ID-69’s concern, referenced not owned.
Promoting the Platform corpus or DB to production. The Platform pipeline is a dev/test rig; its corpus is synthetic and its DB is the Platform dev instance.

Behavior

Each invariant is numbered and testable. The acceptance criterion states how a Checker verifies it. “Operator” = the developer/Liam standing up and running the rig; “downstream consumer” = the MCP/UI/eval surfaces that read the resulting Platform content_items / q_a_pairs rows.

BI-1 — A Platform-owned ingestion deployment exists and writes ONLY the Platform DB

A Platform-owned cocoindex ingestion deployment exists, distinct from both client apps, and all its database writes target the Platform DB (zjqbrdctesqvouboziae). No Platform ingestion write reaches a client DB, and no client app is repurposed to write the Platform DB.

Acceptance: After standup, an ingest run against the Platform corpus produces content_items / source_documents rows in zjqbrdctesqvouboziae and zero rows in any client DB (rovrymhhffssilaftdwd / turayklvaunphgbgscat). The Platform deployment’s database DSN/keys resolve to the Platform project ref; a reviewer confirms the client app uuids are untouched by the Platform deploy path.

BI-2 — The Platform deploy shape mirrors the client deploy shape closely enough to be promotable

The Platform deployment is the same shape the client runs — a cocoindex docker-compose deployment deployed by the same GitHub Actions on-prem deploy workflow, with the in-app ingestion seams and the pipeline-run webhook wired — so the configuration developed/tested on Platform is the artefact promoted downstream. The only deliberate structural difference from the client stack is the absence of the PullMD service trio (BI-4) and Platform-suffixed names/volumes/FQDN.

Acceptance: The Platform deployment is defined by a deploy/coolify/ compose file and deployed by a job in .github/workflows/onprem-deploy.yml (not a bespoke one-off path). A reviewer can map each Platform compose service + env key to its client-stack counterpart; the only structural deltas are the removed PullMD trio and the Platform-suffixed names/router. The Platform deploy does not reuse the client branch guard (both track main); it is a parallel/parameterised target with its own deploy trigger.

BI-3 — The Platform pipeline ingests our own synthetic corpus, never client data

The Platform pipeline’s source corpus is the operator-controlled synthetic set under local-fs-platform/corpus, assembled to mirror the content shapes clients upload (Q&A, capability/methodology, sector intelligence, procurement; markdown / PDF / DOCX, and — if URL ingestion is in the E2E matrix — seeded feed rows). It is not the client corpus (local-fs-client/corpus, the ID-45 Phew set) and not a sanitised copy of client fixtures.

Acceptance: The Platform COCOINDEX_SOURCE_PATH resolves to a corpus rooted at the Platform synthetic set, not local-fs-client/corpus. A reviewer confirms no file in the Platform corpus is a client document or a derived/sanitised client fixture. The corpus exercises ≥3 content shapes and ≥2 file formats so the E2E run is representative.

BI-4 — The Platform stack carries NO PullMD/Playwright services; URL extraction uses the cocoindex-based replacement

Ratified (S403): URL/feed ingestion is in the Platform E2E matrix — the file corpus is file-only, but a feed slice exercises feed_articles through the worker’s FeedUrlSource. URL/feed extraction is served by the cocoindex-based URL-extraction replacement (the trafilatura /extract cleaner path that replaced PullMD per ID-110/111/112 — all shipped), not by pullmd/playwright services. The Platform compose declares no pullmd, playwright, or pullmd-data-* volume, and the Platform env carries no PULLMD_* variables. (The vestigial client PullMD is out of scope — Non-goals.)

Acceptance: The Platform compose contains zero pullmd/playwright services and zero pullmd-data volumes; the Platform env set contains no PULLMD_* keys. If URL ingestion is exercised, a URL ingest produces cleaned content via the /extract cleaner path. A reviewer confirms no PullMD reference (service, volume, env, or token) was carried into the Platform standup.

BI-5 — The Coolify topology is rebuilt for extensibility with correct, future-proof conventions

The Platform standup establishes correct, extensible Coolify conventions — project / server / source / destination topology and ca/canonical naming (kh→ca, knowledge-hub→canonical) — rather than copying the first-client layout that was built for one client without extensibility in mind. The convention is documented so a future second client (or second Platform environment) slots in without bespoke rework.

Acceptance: A reviewer can point to the documented Platform Coolify topology conventions (naming scheme, project/server/source/destination roles) and confirm the Platform resources follow them. The Platform app name uses the ca/canonical scheme, not kh/knowledge-hub. The conventions are written so they generalise to N environments, not hard-coded to “platform + one client”.

Deferred to TECH (S403 — Liam): the exact naming scheme and how it interacts with the GitHub Actions secret/variable scoping (per-environment secrets vs a shared set) is a TECH decision — PRODUCT requires an extensible, documented scheme, not a specific string. Starting point (RESEARCH rec): reuse the existing unprefixed env contract (SUPABASE_*/COCOINDEX_*/EXTRACT_API_TOKEN) scoped to the Platform DB rather than inventing a KH_PLATFORM_* prefix the worker does not read; revisit only if the kh→ca/CA rename forces it.

BI-6 — The local-fs source split (`local-fs-client` / `local-fs-platform`) cleanly separates the two configs

The two local source folders are named, distinct, and bound to the correct deployment: local-fs-client feeds the client pipeline’s corpus and local-fs-platform feeds the Platform pipeline’s corpus. The split is the mechanism that keeps client and Platform source content from co-mingling, and the relationship between each local folder and the on-prem COCOINDEX_SOURCE_PATH (/cocoindex-state/corpus) is explicit.

Acceptance: A reviewer can state, from the spec/config, which local-fs folder feeds which deployment and how each reaches its on-prem corpus volume. The Platform deployment’s corpus derives from local-fs-platform/corpus; the client deployment’s from local-fs-client/corpus. No config points a Platform deployment at the client folder or vice versa.

Ratified (S403): v1 corpus-sync is a simple local→on-prem bridge from local-fs-platform/corpus to the on-prem named volume /cocoindex-state/corpus; the exact mechanism (manual copy / rsync / a deploy sync step) is a TECH choice. Pluggable external sources (SharePoint / Notion / others) are explicitly OUT of ID-127 scope — WS-6-gated per platform-direction.md (“connectors come later, gated on pipeline confidence”) and ID-71 B-INV-12 (a net-new source write-back is REFUSED at the surface until WS-6). OKF (Open Knowledge Format) alignment is a forward-looking note, NOT an ID-127 dependency — see Notes for TECH.

BI-7 — Canonical identity depends on a stable base path; a root manifest is mandatory, but only the forms slice carries a workspace_id

Because cocoindex canonical identity is rel_path-derived (uuid5(namespace, "ci:" + rel_path)), the Platform corpus base path is part of the canonical contract: moving or re-rooting the corpus forks every record’s identity. The Platform corpus therefore declares a stable staged layout.

Corrected twice (S403) — manifest is mandatory at the FLOW level, but is NOT a content-partitioner. Verified against HEAD: content_items has no workspace_id and the content_item_workspaces M2M is deliberately not populated by ingest (workspace-agnostic records — ID-69 BI-1; the M2M writer is ID-69’s deferred v1.1). A root manifest FILE is, however, mandatory: the deployed flow entrypoint app_main (flow.py:3403-3427) loads <source>/.kh-workspace-map.json once at flow start and aborts the whole run (manifest_missing / manifest_invalid) if it is absent or invalid — the per-item manifest is None → route "content" default at flow.py:1947 is a defensive fork the deployed entrypoint never reaches (Q-EX2 re-verify, {127.3} TECH). The manifest is nonetheless a route-tagger + forms-workspace-assigner, NOT a content-partitioner: it tags path-prefixes with a route (content / forms / qa_sidecar) and supplies a workspace_id only for the forms route (forms — form_templates/form_responses — ARE workspace-scoped, unlike content/q_a_pairs). Content / q_a_pairs / feed files match no prefix, soft-warn (UnmappedPath), and default to route "content" → workspace-agnostic records. So the Platform corpus ships a root manifest as a non-optional artefact carrying only the minimal forms-slice mapping (BI-8): content needs no mapping, but the manifest file must exist. (The kh→ca rename of the manifest filename constant is a code change deferred to bl-368 — OQ-127-MANIFEST-RENAME; ID-127 ships .kh-workspace-map.json to match deployed code.)

Acceptance: The Platform corpus base path is fixed and documented as an operator obligation; a reviewer can point to the stated base-path-stability obligation and the identity-fork consequence. A root manifest file exists at the corpus root (app_main aborts without it) using the existing schema (schema_version, mappings: [{path_prefix, route, workspace_id}]) and loads without ManifestLoadError. The content / q_a_pairs / feed slices carry no mapping (they soft-warn UnmappedPath → route "content", workspace-agnostic); only the forms slice mapping carries a workspace_id, routing its files to the intended Platform workspace. Re-running ingest on an unchanged corpus re-stamps the same canonical identities (no duplication).

BI-8 — Platform workspaces follow the canonical entity model (`workspace` ≠ `client`/`tenant`)

Any workspace the Platform corpus maps to is a container within the single Platform database, bound N:1 to one application_type — consistent with the canonical glossary in platform-direction.md, where workspace is NOT synonymous with client/tenant (the tenant boundary is the database; the Platform DB is one tenant). The Platform corpus may map to multiple workspaces (one per application_type it exercises), all within the Platform DB. Where the legacy “workspace = tenant” reading appears (e.g. ID-69), this PRODUCT and platform-direction.md win.

Acceptance: Every workspace_id referenced by the Platform corpus manifest resolves to a row in the Platform DB’s workspaces table bound to an application_type. A reviewer confirms no Platform config treats a workspace as a tenant/client boundary, and that “cross-workspace” Platform operations are understood as INTRA-tenant (within the one Platform DB). The number of Platform workspaces is driven by the application_types the synthetic corpus is designed to exercise, not by a notion of multiple clients.

Ratified (S403): the Platform corpus exercises all 6 baseline application_types (procurement, intelligence, sales_proposal, product_guide, competitor_research, training_onboarding) — one Platform-DB workspace each — with headroom for future types. Per BI-7, content records stay workspace-agnostic (no workspace_id; M2M unpopulated by ingest); workspaces are exercised by (a) standing up the 6 workspace rows and (b) routing a minimal forms slice (≥1 workspace — procurement bid-questions) to its workspace via the manifest. Full content↔workspace M2M association is ID-69’s deferred v1.1, referenced not owned here.

BI-9 — The Platform pipeline is dev-time / intermittent, not continuously standing like a client’s

The Platform pipeline’s operational shape is dev-time and intermittent — it runs when we develop/test, not as an always-on production ingestion service the way a client’s pipeline serves a live corpus. Boot never auto-walks (the ID-83/bl-221 burn gate), so the source path may stay set permanently without burning on boot; walks happen on demand (operator-triggered) or on a light, explicitly-chosen cadence — not as a high-frequency standing schedule.

Acceptance: The Platform deployment can be brought up and torn down (or left idle) without unintended ingestion: a Platform boot with the corpus mounted does not trigger an automatic walk. A walk occurs only on an explicit operator trigger or a deliberately configured cadence. A reviewer confirms the operational shape is documented as intermittent/dev-time, distinct from a client’s continuous standing pipeline.

Open question (OQ-127-CADENCE): whether the Platform rig has any scheduled walk cadence (e.g. a light nightly) or is purely on-demand is a TECH/operational decision — the PRODUCT requires only that it is not a continuous standing schedule and that boot is burn-safe.

BI-10 — Write-model prerequisites are verified on the Platform DB before first ingest

Before the first Platform ingest writes, the Platform DB’s write-model prerequisites are verified present (not authored): the pipeline service-account identity row (a0000000-0000-4000-8000-000000000001, the created_by identity), the auth.users triggers + user_profiles mirror (the S374 defect that must be remediated by the squash baseline), and the role-based RLS / schema reconciliations (ID-64). The Platform write path uses the direct asyncpg pooler DSN (RLS-exempt postgres role), not the Supabase service-role JWT as a password.

Acceptance: A verification step (e.g. scripts/seed-e2e-users.ts / scripts/seed-platform-from-staging.ts assertions) confirms the service-account row and auth triggers exist on zjqbrdctesqvouboziae before first write. The Platform COCOINDEX_DB_DSN is a pooler DSN for the postgres role with the correct aws-N prefix (read from the dashboard, never guessed); a reviewer confirms the service-role JWT is not used as the PG password. A first ingest stamps created_by = a0000000-0000-4000-8000-000000000001.

BI-11 — Backup and monitoring are standard, non-optional parts of the standup

The Platform standup includes backup (LMDB cold-backup, reusing the existing deploy/onprem/backup/ tooling, env-parameterised for Platform) and monitoring (the idle/stall datapath watch, reusing deploy/onprem/monitor/datapath-watch.ts / app/api/cron/datapath-watch) as standard deliverables — not deferred low-priority extras. The monitoring correctly treats the Platform pipeline’s idle/intermittent shape (BI-9) as normal, not as a stall to page on.

Acceptance: The Platform standup wires a Platform LMDB backup mechanism and a Platform monitoring path; a reviewer confirms both are present in the delivered scope, not listed as “optional / later”. The monitor does not raise on expected Platform idleness (empty/unset source or no recent walk during a dev-idle period reads as idle, not as a failure).

BI-12 — The pipeline-run webhook records Platform runs against the Platform app

Platform pipeline runs report their per-run rollup to the Platform app’s pipeline-run record endpoint (/api/internal/pipeline-runs/record, which already exists on canonical-platform), authenticated by a shared Platform CRON_SECRET. The webhook is status-only and best-effort (it never blocks or fails the ingest run).

Acceptance: After a Platform ingest run, a pipeline_runs row is recorded in the Platform DB via the Platform app endpoint; the worker’s PIPELINE_RUN_WEBHOOK_URL points at the Platform app and shares the Platform CRON_SECRET. A reviewer confirms the webhook is best-effort (an unreachable endpoint or unset URL does not fail the run).

BI-13 — The Platform deployment pins a deliberate image tag (ID-113 semver), never a rolling tag or `:latest`

The Platform deployment pins its cocoindex image to a deliberate tag and never to :latest or an implicit rolling main sha that drifts unmanaged. Because ID-113 has shipped (the tag-release job aliases known-good sha- images to vX.Y.Z without rebuild), the Platform deployment adopts a vX.Y.Z semver pin where a release exists, consistent with the “Platform tracks main, deployments pin a version” model.

Acceptance: The Platform COCOINDEX_IMAGE_TAG resolves to a concrete vX.Y.Z (or a pinned sha-) tag, never :latest; the compose hard-fails if the tag is unset. A reviewer confirms the pin is deliberate and rollback works by repointing to a prior tag without a rebuild (the ID-113 mechanism).

BI-14 — A documented promote mechanism pushes a proven Platform config to a client config

There is a documented, repeatable mechanism by which a Platform-proven configuration — the compose structure, the env keyset, and the pinned image tag — is promoted to a client configuration. This is the payoff of the rig (former OQ-301-7) and ties directly to ID-113’s release-cut / client-upgrade runbook: a config proven on Platform (tracking main) becomes a client’s pinned vX.Y.Z deployment.

Acceptance: A reviewer can follow a documented promote runbook from “Platform config proven at main” to “client config updated to the corresponding pinned release” — covering the compose delta (Platform-suffixes → client-suffixes), the env re-key (Platform DB → client DB), and the image-tag pin. The runbook references the ID-113 release-cut model and states whether the promote is manual or scripted.

Ratified (S403): v1 promote = a documented manual runbook. The promote artefact is the kh-cocoindex-pipeline GHCR image (ghcr.io/<owner>/kh-cocoindex-pipeline, built off-host by onprem-deploy.yml, pinned vX.Y.Z via ID-113’s tag-release): the same immutable image moves Platform→client unchanged; only config differs per environment (DB DSN, source path, secrets, compose name-suffixes). The runbook = pin the proven image tag + apply a documented config-delta template. (Build-once / config-per-environment.)

BI-15 — The Platform config is guarded against silent drift from `main`

Because deploy-target composes live on branches that do not auto-sync from main, the Platform config (cut off main) is guarded against silent divergence: the standup replicates the never-auto-deploy sentinel guard and wires the deploy-branch parity guard (bl-346) so the Platform compose cannot drift from main unnoticed.

Acceptance: The Platform app carries the .never-auto-deploy-sentinel watch-path guard (no unintended auto-deploy) and is covered by the parity/drift guard. A reviewer confirms a Platform-compose change that diverges from main is surfaced (guard/sentinel), not silently shipped.

KH quality bars (apply to any downstream implementation)

Surfaced so Executors inherit them via TECH/PLAN (.claude/agents/references/shared-discipline.md §KH quality bars):

No silent Supabase failures. Any TS-side read/write on the Platform app surfaces uses sb() / tryQuery() from @/lib/supabase/safe; composite responses via warningsEnvelope(). (The worker’s DB writes go over asyncpg, not the Supabase client — BI-10.)
Auth. Webhook + ingest seams: bearer-gated (CRON_SECRET / EXTRACT_API_TOKEN) inside each /api/* route; any role-gated surface uses getAuthorisedClient() returning { success } and routes failures via authFailureResponse(auth). No proxy.ts publicRoutes edit is required (the ingest/webhook routes bypass the login redirect as /api/* and enforce auth internally).
No barrel re-exports — direct file imports only.
TanStack Query for any app-side data fetching.
bun run test (not bun test); Python pipeline tests via python3 -m pytest scripts/tests/.
Test philosophy — tests verify real behaviour (a real ingest lands rows in the Platform DB; the webhook records a run; the monitor reads idle correctly), not implementation. Reference ${KH_PRIVATE_DOCS_DIR}/src/content/docs/reference/test-philosophy.md for behaviour-change Subtasks.
UK English, DD/MM/YYYY dates, semantic tokens only for any UI (none expected in v1).

Empirical verification (Q-EX2 — pre-ratification)

External-library symbols this PRODUCT leans on, checked against the pinned version on 23/06/2026:

Symbol	Pinned version	Result
`cocoindex` (module)	`cocoindex[postgres]==1.0.7` (requirements.txt)	PRESENT — installed `cocoindex 1.0.7` matches pin.
`cocoindex.connectors.localfs.walk_dir`	`1.0.7`	PRESENT — `hasattr(localfs, 'walk_dir') == True`.
`cocoindex.connectors.localfs.declare_file`	`1.0.7`	PRESENT — `hasattr(localfs, 'declare_file') == True`.

No ABSENT / SIGNATURE_DRIFT findings — the source-binding the spec assumes (the localfs connector the deployed pipeline already uses) is present at the pinned version. The {127.3} TECH author should re-verify any additional cocoindex symbols (update_blocking live-mode flag, the /extract trafilatura path) it cites at implementation depth, per the same forcing function.

Ratified decisions (S403 — Liam)

All six open decisions were ratified by Liam in S403. Recorded here for the {127.3} TECH author; the inline **Ratified (S403):** markers at each BI carry the same outcomes.

Host placement (OQ-127-HOST). Deferred to TECH → SUPERSEDED in S408 (see AMENDMENT-staging-prod-two-server.md). TECH resolved this as a separate Coolify project on the shared client IONOS host (prod-only); S408 ratified it OUT to a two-server split — a dedicated Platform VPS (Server B) running staging + prod Platform environments, with one Coolify control-plane and no new GH environment (OQ-2). The amendment is authoritative; the three framing questions below are retained as the historical deferral. (Original text: TECH must explicitly answer Liam’s three framing questions: (i) would a real SMB host its pipeline on existing infrastructure rather than a dedicated VPS? (ii) with PullMD removed, is temporary Mac-hosting feasible? (iii) with PullMD removed, is co-locating Platform + client on the one IONOS VPC acceptable until launch, given the topology rebuild? RESEARCH leans “separate Coolify project on the existing IONOS host” (cheapest; materially lighter now the PullMD trio is gone).)
Corpus-sync (OQ-127-SYNC, BI-6). Ratified: v1 = a simple local→on-prem bridge (local-fs-platform/corpus → /cocoindex-state/corpus); exact mechanism (copy / rsync / deploy sync step) is TECH’s call. Pluggable external sources (SharePoint / Notion / others) are OUT of scope — WS-6-gated (platform-direction.md; ID-71 B-INV-12). OKF is a forward-looking note (Notes for TECH), not a dependency.
Workspace model (OQ-127-WS, BI-8). Ratified: exercise all 6 baseline application_types (one Platform-DB workspace each), with headroom for future types. Content records stay workspace-agnostic (BI-7/BI-8); the manifest is required only for a minimal forms slice (≥1 workspace — procurement). Full content↔workspace M2M is ID-69 v1.1, referenced not owned.
Stack breadth / URL ingestion (OQ-127-URL, BI-4). Ratified: the file corpus is file-only; URL/feed ingestion is in the matrix via feed_articles + FeedUrlSource
- the /extract cocoindex replacement (ID-110/111/112 all shipped). Feed rows are seeded into the Platform DB.
Naming ↔ GitHub secret scoping (OQ-127-NAMING, BI-5). Deferred to TECH. Starting point (RESEARCH rec): reuse the existing unprefixed env contract (SUPABASE_*/COCOINDEX_*/ EXTRACT_API_TOKEN) scoped to the Platform DB; revisit only if the kh→ca/CA rename forces it.
Promote automation (OQ-127-PROMOTE, BI-14). Ratified: v1 = a documented manual runbook. The kh-cocoindex-pipeline GHCR image is the promote artefact; promote = pin the proven vX.Y.Z tag (ID-113 substrate, shipped) + a config-delta template. Build-once / config-per-environment.

Already settled (do not re-open): corpus = our own synthetic dev/test content (not client data); the Platform deploy mirrors the client deploy shape (promotable upstream); PullMD is retired and its replacement is cocoindex-based; backup/monitor are non-optional; ID-113 has shipped so the Platform deployment pins a vX.Y.Z semver; priority/sequencing is a non-issue (Platform and client work proceed in parallel).

Notes for TECH (`{127.3}`)

Q-EX2 re-verify duty. Re-verify at implementation depth: (a) the BI-7 manifest/forms-route claim (flow.py:1947, 3100-3104, workspace_resolver.py:load_workspace_manifest, resolve_route) — confirm the content slice ingests manifest-free and only the forms route consumes workspace_id; (b) the cocoindex symbols this build leans on (localfs.walk_dir / declare_file re-confirmed S402; plus the update_blocking/live-mode flag and the /extract trafilatura path).
OKF (Open Knowledge Format) — forward-looking, non-blocking. Google Cloud’s OKF (markdown + YAML frontmatter, directory-organised, git-versionable, vendor-neutral — “format not platform”) aligns strongly with platform-direction.md’s controlled-local-fs / “one record, many views” v1 shape. The synthetic Platform corpus (BI-7 stable layout) is the cheapest place to trial an OKF-shaped layout, but adopting OKF is its own spike (the cocoindex flow expects content files + a forms manifest today, not OKF frontmatter). Do not block ID-127 on it; a backlog spike tracks the alignment evaluation.

Provenance pointers

Predecessor RESEARCH: specs/bl-301-platform-pipeline/RESEARCH.md (S384) — ported as {127.1}; corrected premises below override its stale ones.
Liam’s RESEARCH feedback: specs/bl-301-platform-pipeline/notes/bl-301-feedback.md (main checkout) — folded in full (kh→ca rename, PullMD retirement + cocoindex replacement, local-fs split + synthetic Platform corpus, Coolify extensibility/topology, secrets-ready-except-newly-minted, COCOINDEX_SOURCE_PATH↔local-fs relationship, backup/monitor non-optional, ID-113 shipped, feed_articles non-IP subset, OQ responses).
Live Coolify topology recon (S402): scratchpad bl-301-coolify-recon.md — current topology facts (1 project, 2 client apps, host specs, env-var name sets, deploy-job hardcoded uuids, local-fs-* existence, COCOINDEX_SOURCE_PATH=/cocoindex-state/corpus).
Entity glossary (authoritative): reference/platform-direction.md (S391) + reference/state-of-the-product.md §Tenancy — workspace ≠ client/tenant.
Onboarding/corpus + manifest model: specs/id-69-client-corpus-onboarding/{PRODUCT,RESEARCH}.md — inherited the path_prefix → workspace_id manifest binding; cross-checked workspace usage against platform-direction.md (ID-69’s “cross-workspace” reading is INTRA-tenant under the corrected glossary).
Release/promote substrate: specs/id-113-platform-release-versioning/TECH.md (PI-10) + onprem-deploy.yml:546–593 (tag-release job — shipped).
Code-intelligence orientation: gitnexus_query/gitnexus_context outputs cited verbatim in Problem; flow.py:app_main (L3300–3835), workspace_resolver.py:load_workspace_manifest (L195–223), url_source.py:FeedUrlSource.items, server.py, deploy/onprem/monitor/datapath-watch.ts.

Corrected model (authoritative over the RESEARCH’s stale premises)

The {127.1} RESEARCH was authored against three premises corrected at S402; this PRODUCT encodes the corrected model:

PullMD is RETIRED (ID-112/110). The RESEARCH/recon described pullmd/playwright as live in the client stack; the Platform stack carries NEITHER. The URL-extraction replacement is cocoindex-based (the trafilatura /extract cleaner). (The deployed client compose still runs PullMD vestigially pending {112.13} — not carried into Platform; Non-goals + BI-4.)
local-fs-client / local-fs-platform exist as top-level local source folders (the RESEARCH’s recon found “no such named configs in canonical” — correct for the repo, but the folders exist under ~/Documents/development/). local-fs-client/corpus holds the real client (ID-45) set; local-fs-platform/corpus is empty (the synthetic Platform corpus is to be created). They are local simulated source folders feeding the on-prem corpus volume, not in-repo configs. (BI-6.)
Execution = on-prem Coolify, not local-run. The deliverable is a Platform-owned Coolify deployment mirroring the client stack (promotable upstream); local-run is an inner dev loop only, not the deliverable. Host placement (separate host vs separate project on the existing host vs temporary Mac) is an open decision, but the shape is the Coolify deploy shape. (BI-2 + collected decision 1.)

Changelog

Date	Session	Note
23/06/2026	task-planner (fresh ctx, Q-PLANNER-2)	`{127.2}` PRODUCT authored. 15 numbered Behavior invariants. Encoded the corrected S402 model (PullMD retired + cocoindex URL replacement; local-fs-client/platform split + synthetic Platform corpus; on-prem Coolify execution mirroring the client deploy shape as promotable upstream). Folded all of Liam’s RESEARCH feedback (kh→ca rename, Coolify topology/extensibility rebuild, backup/monitor non-optional, ID-113 shipped, COCOINDEX_SOURCE_PATH↔local-fs relationship). Workspace model held to the canonical glossary (`workspace` ≠ `tenant`; Platform workspaces are intra-Platform-DB containers bound to application_types). Empirical Q-EX2 check on cocoindex 1.0.7 localfs connector PASSED. 6 open decisions collected for Liam’s ratification (host, corpus-sync, workspace set, URL/stack breadth, naming↔GH-secrets, promote automation).
23/06/2026	Orchestrator (S403, Liam ratification)	All 6 decisions ratified. SYNC: v1 local→on-prem bridge (mechanism→TECH); pluggable sources WS-6-gated/OUT; OKF = forward note + backlog spike. WS: all 6 application_types, minimal forms slice (≥1 workspace, procurement). URL: file-only corpus + feed_articles via FeedUrlSource/`/extract`. PROMOTE: manual runbook, `kh-cocoindex-pipeline` GHCR image is the promote artefact. HOST + NAMING deferred to TECH (with framing). BI-7 CORRECTED (Liam’s manifest query): verified against HEAD — content records are workspace-agnostic (no `workspace_id`; M2M unpopulated by ingest, ID-69 v1.1) and ingest is manifest-free for content; the manifest is a route-tagger + forms-workspace-assigner needed ONLY for the forms slice. BI-4/5/6/8/14 inline markers updated; Notes-for-TECH added (Q-EX2 re-verify duty + OKF). Ready for `{127.3}` TECH (fresh planner).