TECH — Fixture-staging live-verification infra — co-located /stage route + localhost verify driver (B1 on-prem) (ID-62.3)
TECH — Fixture-staging live-verification infra — co-located /stage route + localhost verify driver (B1 on-prem) (ID-62.3)
Section titled “TECH — Fixture-staging live-verification infra — co-located /stage route + localhost verify driver (B1 on-prem) (ID-62.3)”Task: ID-62 — Fixture-staging live-verification infra.
Subtask: {62.3} TECH.
Predecessors (read in full): {62.2} PRODUCT
(docs/specs/id-62-fixture-staging-infra/PRODUCT.md, 29 invariants Inv-1..29, ratified
S283 under B1) + {62.1} RESEARCH (docs/specs/id-62-fixture-staging-infra/RESEARCH.md) +
the ratified directive set docs/specs/id-62-fixture-staging-infra/P4-RECONCILIATION.md
(§2 SURVIVES / §3 CHANGES / §4 net-shape) + the B1 host model
docs/specs/ID-66-onprem-pivot/RESEARCH.md (§1-a..f).
Author date: 29/05/2026 (S283).
SUPERSEDES the S279 P2 “in-VPC Cloud Run Job + WIF + Secret Manager + empirical entry gate” TECH. This is a wholesale re-spec, not an amendment. The prior TECH operationalised an in-VPC Cloud Run Job reaching an
ingress: internalcocoindex Service over WIF, mounting secrets via--set-secrets, gated on an empirical internal→internal reachability probe. The S282 ratification of Option B1 (co-locate cocoindex + pullmd on one on-prem host — ID-66 RESEARCH) dissolved that design’s organising spine. The whole Cloud-Run-Job / WIF / Secret-Manager / entry-gate block is deleted, not relabelled. Git history preserves the old version. Cross-reference: P4-RECONCILIATION §1 (WHOLESALE verdict), §4 (B1 net shape).
This is the
{62.3}artefact authored by a fresh Planner instance (Q-PLANNER-2 — NOT the{62.1}or{62.2}author). It operationalises PRODUCT’s 29 behaviour invariants into an implementation plan — wire shapes, module boundaries, migration mechanics, a per-invariant verification map. It does NOT relitigate the ratified basis (B1 co-located host; localhost reachability;/stage-load-bearing default per OQ-62-P4-2;/healthkept per OQ-62-P4-3; entry gate deleted). Proposed changes are mapped one-to-one against the numbered PRODUCT invariants Inv-1..29 so the Checker can verify spec compliance per-invariant. The{62.4}PLAN slice outline at the end is an outline only —{62.4}(a separate fresh Planner) formalises the Subtask records; this TECH does NOT author them and does NOT edittask-list.json.
Context
Section titled “Context”ID-62 makes the cocoindex integration tier (*.integration.test.ts under
__tests__/integration/cocoindex/, authored but never run live) executable against the live
cocoindex stack on the B1 co-located on-prem host (the ID-66 host where cocoindex +
pullmd run as sibling containers and cocoindex reaches pullmd over http://localhost:3000).
Two cooperating surfaces are built — a co-resident POST /stage byte-drop route on the
cocoindex aiohttp server, and a co-located verify driver (a process on the same B1 host)
that stages fixtures by POSTing bytes to /stage over http://localhost:<port> and exits
non-zero on any failure — after which the existing Vitest assertion surface verifies produced
rows against live Supabase. The same primitive, invoked with the ID-42 {42.10} HTML fixture
- the Inv-7/8/9 assertion set (where the
/s/<share_id>round-trip runs against pullmd on localhost), clears the{42.10}blocker. PRODUCT §Behavior carries the full behavioural contract; this section grounds the implementation surfaces in current code.
Code-intelligence orientation (verbatim — per the planner-citation block)
Section titled “Code-intelligence orientation (verbatim — per the planner-citation block)”gitnexus_query({query: 'cocoindex server route health corpus watcher stage', repo: '…/knowledge-hub'})— returned"processes": [],"process_symbols": [](the aiohttp server surface is NOT modelled as an execution flow), but itsdefinitionssurfaced the exact load-bearing Python symbols this TECH modifies/extends, all in moduleCocoindex_pipeline:Function:scripts/cocoindex_pipeline/server.py:_health_handler(startLine 131, endLine 144),Function:scripts/cocoindex_pipeline/server.py:build_app(152–160),Function:scripts/cocoindex_pipeline/server.py:_target(203–217),Function:scripts/cocoindex_pipeline/flow.py:ingest_file(1039–1489),Function:scripts/cocoindex_pipeline/flow.py:app_main(1523–1867),Function:scripts/cocoindex_pipeline/flow.py:_emit_stage_error_log(363–396). The Python test home was also surfaced:Method:scripts/tests/test_cocoindex_server.py:TestWorkerLiveness.test_health_200_when_worker_healthy(361–370) +test_health_503_when_worker_crashed(372–390).gitnexus_query({query: 'fixture staging stage upload corpus', repo: '…/knowledge-hub'})— returned"processes": []and NO matching TS staging-helper symbols: it surfaced only Python stage-error helpers (flow.py:_classify_stage_exception231–335,_emit_stage_error_log363–396,_empty_stage_counts469–486), unrelated form byte-fixtures (test_form_extractors.py:csp_xlsx_bytes448–455 etc.), and an e2e upload spec. The TS staging helpers (stageFixture,pollContentItemsFor,dropFixture) are NOT in the GitNexus index —__tests__/integration/is outside the indexed TS corpus (confirmed: identical to PRODUCT §Context). They are grounded by direct file read (verbatim line cites throughout), not bygitnexus_context.- Index-freshness caveat (grounding discipline). The GitNexus index covers two
knowledge-hubrepos — the main checkout (…/knowledge-hub, reported ~77 commits behind) and a sibling worktreesubo-id-52. This worktree (subo-id-62) is NOT separately indexed. Every line number cited below was therefore re-grounded by direct file read against this worktree’s working tree this session — the gitnexusdefinitionsline numbers above (e.g.build_app152–160) confirm the symbol identity; the working-tree reads confirm the current line numbers (which agree within ±1:build_app153–161 by read). - ast-dataflow / ts-morph — not applicable to the symbols this TECH touches:
server.py/flow.py/adapters.pyare Python (ts-morph does not cover Python, per.ast-dataflow/CLAUDE.md), and the one TS surface (fixture-staging.ts) is a test-corpus helper whose consumers are*.integration.test.tsfiles enumerated by thegrepsweep in §Testing, not production call sites. Nogitnexus_impactis claimed where the symbol is not indexed; the modified symbol set is the Python server + the out-of-index TS helper, both grounded by read.
Current state — grounded surfaces (this worktree, by direct read)
Section titled “Current state — grounded surfaces (this worktree, by direct read)”-
The cocoindex server has exactly one route.
scripts/cocoindex_pipeline/server.py:build_app()(153–161) createsweb.Application()and registers ONLYapp.router.add_get("/health", _health_handler)(160)._health_handler(132–145) returns200 {"status": "ok"}whileworker_is_healthy()and503 {"status": "error", "reason": …}once the worker thread crashed (140–145).main()(234–255) installs SIGTERM handling, callsstart_cocoindex_thread()(169–226 — spawns the daemon thread runningKH_PIPELINE_APP.update_blocking(live=True)at 210), thenweb.run_app(app, host="0.0.0.0", port=port, print=None)(255). The handler and the watcher share ONE process and ONE host filesystem.aiohttpis already imported (from aiohttp import web, 44).build_app()is factored so route-table tests run without a socket (docstring 154–158). -
The watched corpus is a local-fs path, armed once at boot, manifest-gated.
scripts/cocoindex_pipeline/flow.py:app_main(1524): readsos.environ.get("COCOINDEX_SOURCE_PATH", "")(1536); if empty → logs “idle mode” and returns cleanly (1537–1543); if the path does not.exists()→ idle return (1546–1553). When set+existing it loadssource_path / ".kh-workspace-map.json"(1582) viaload_workspace_manifest; aManifestLoadError(1585) → emits a structuredmanifest_missing/manifest_invalidstage error (1586–1595) + afailedpipeline-run webhook (1596–1605) thenraises (1606) — aborting the whole flow. Only on a valid manifest does it armsource = localfs.walk_dir(source_path, live=True, recursive=True)(1683) and fan out per-item viacoco.mount_each(ingest_file, …)(≈1717). The watch arms once atapp_main; today the value is""→ idle (PRODUCT Inv-15/16; ID-66 §4-i: the “/health green ≠ ingesting” no-op finding). -
The TS client helper POSTs JSON
{ fixturePath, destPath, titlePrefix }.__tests__/integration/cocoindex/_helpers/fixture-staging.ts(814 lines):hasFixtureStagingUrl()(43–45);StageFixtureArgs(51–71:fixturePath57,destPath63,titlePrefix70);StageFixtureResult(73–85:destPath79);stageFixture(100–139) builds${baseUrl.replace(/\/$/, '')}/stage(111), throws fast ifCOCOINDEX_FIXTURE_STAGING_URLunset (105–108), POSTsContent-Type: application/json(115) with JSON body{ fixturePath, destPath, titlePrefix }(116–119), throws on non-2xx (125–128), reads back{ destPath?, requestId? }(131–136).pollContentItemsFor(179–222) pollscontent_itemsilike('title', '${titlePrefix}%')(199) via the live service-role client, timeout error (220–221).dropFixture(376+) — empty-contentIdsthrow (378), empty-titlePrefixno-op+warn (383–385),entity_mentionscleanup best-effort (411–425, “ID-49.5 deferred per S273 OQ-1”). Form analoguespollFormTemplatesFor(546+) poll onname/storage_pathILIKE (matchColumn = opts.matchStoragePath ? 'storage_path' : 'name', 559;.ilike(matchColumn, '${prefix}%'), 568). -
The cocoindex→pullmd URL is env-configurable (supports the B1 localhost repoint).
scripts/cocoindex_pipeline/adapters.py:125readspullmd_url = os.environ.get("PULLMD_SERVICE_URL")withAuthorization: Bearer(120) — there is no hardcoded host. So repointing cocoindex→pullmd from the Cloud-Runrun.appURL tohttp://localhost:3000under B1 is a one-line env change (ID-66 §1-b / OQ-66-2, here empirically confirmed by read). This is an ID-66 concern (the Compose env), not ID-62 work; cited because the{42.10}Inv-9 round-trip (Inv-23) and the AGPL sibling-container constraint depend on pullmd being a localhost sibling. -
The AGPL Inv build assertion.
cloudrun/cloudbuild-cocoindex.yaml:assert-inv9-no-pullmd-no-playwright(117–167)docker image inspects the publishedkh-cocoindex-pipelineimage, countspullmdlayer hits (134–139) + Playwright/chromium markers (144–149), andexit 1s on any hit (158–159, 164). This build-time assertion keeps pullmd OUT of the cocoindex image — the AGPL boundary. It is independent of where the image runs (PRODUCT Inv-8 prose; carried CHECKER NOTE — see §AGPL constraint below). -
The Python test home for the route table.
scripts/tests/test_cocoindex_server.py: theaiohttp_appfixture (145–155) builds the app viabuild_app();_exercise_health(158–176) resolves a registered route in-process viaaiohttp_app.router.resolve(...)withmake_mocked_request— NO real socket.TestHealthEndpoint(180+) asserts the/health200/JSON contract. This is exactly the home for new/stageroute-table tests. -
The launch-flip sequencing home.
docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.mdrecords the live-tier-not-running caveat (lines 20, 147, 156) and has a T13 “Pre-launch operational pre-decisions + observability” row (241) — the canonical home for the Inv-29 launch-flip action item. It does NOT yet carry an explicit “flip to a scheduled cadence at launch” item. -
B1 substrate is ID-66’s to build, not ID-62’s. The co-located Docker host + Compose stack (cocoindex + pullmd over localhost, ID-66 §1-a/b), the persistent local-disk volume hosting the corpus + the cocoindex LMDB (§1-c), the push-to-deploy path replacing
cloud-run-deploy.yml/WIF (§1-d), and the host secrets store + datapath monitoring (§1-f) are ID-66 concerns. ID-62 builds the/stageroute + verify driver that RUN ON that host (Task-level dependencyID-62→ID-66, PRODUCT §ID-66-precondition / OQ-62-P4-4; the Orchestrator records the edge).
Resolved TECH-level OQs
Section titled “Resolved TECH-level OQs”These resolve the carried PRODUCT TECH-level constraints (Inv-24..27) + the carried TECH OQs (OQ-62-1/6/7/11/12) + the ratified directives (OQ-62-P4-2/P4-3), re-resolved for B1. The moot Cloud-Run OQs (old OQ-62-2 volume kind, old OQ-62-3 co-resident-vs-separate-Service in its Cloud-Run framing, old OQ-62-9 entry gate) are NOT carried.
| OQ | Resolution (this TECH, B1) | Satisfies |
|---|---|---|
| OQ-62-1 (reframed) corpus path value + activation | COCOINDEX_SOURCE_PATH = /corpus, a directory on the B1 host’s persistent local-disk volume (the same disk hosting the cocoindex LMDB, ID-66 §1-c) — set in the B1 Compose env, NOT a Cloud Run manifest. server.main() (234–255) gains a startup step that mkdir -ps ${COCOINDEX_SOURCE_PATH} AND seeds a valid .kh-workspace-map.json at the corpus root BEFORE start_cocoindex_thread() (252) so the watch arms (not idle on a missing dir, not aborting on a missing manifest, 1546/1582). Activation = container (re)start (the watch arms once at app_main; today "" = idle). Corpus is persistent local disk → survives restarts, NO Filestore, NO ephemeral-tmpfs caveat. | Inv-15, Inv-16, Inv-17, Inv-24 |
OQ-62-6 titlePrefix injection per fixture format | Filename-derived title (no in-byte injection) — topology-agnostic, PORTABLE. /stage writes the bytes verbatim and names the file <destPath>, where the caller embeds TEST_PREFIX IN the dest filename (existing tests already do this: destPath: 'inv-1/${TEST_PREFIX}.xlsx'). cocoindex’s ingest_file derives content_items.title from the source-path identifier (the relative path from walk_dir().items()), so a TEST_PREFIX-bearing filename → a TEST_PREFIX-bearing title that pollContentItemsFor’s ilike('title', '${titlePrefix}%') (199) matches. The form tier polls name/storage_path (pollFormTemplatesFor, 546+) — form_templates has no title — and the {52.12} write sets storage_path = rel_path, so the same dest-path-prefix strategy carries the form tier via matchStoragePath (559). /stage performs NO format-specific metadata injection. | Inv-25 |
| OQ-62-7 wire contract: exact byte encoding | multipart/form-data, NOT base64-in-JSON. Body = one file part file (raw bytes) + text parts destPath and titlePrefix. Rationale: fixtures include binary .xlsx/.docx/.pdf — base64 inflates ~33% + forces a full-buffer JSON parse; multipart streams the part to disk. StageFixtureArgs (51–71) keeps fixturePath, destPath, titlePrefix UNCHANGED (callers compile unchanged); the helper reads the bytes runner-side from fixturePath and sends them as the multipart file part — the change is internal to the helper’s request construction. | Inv-2, Inv-19, Inv-26 |
OQ-62-P4-2 /stage HTTP vs direct disk-drop | /stage HTTP over localhost is the load-bearing default (PRODUCT Inv-12; OQ-pending OQ-62-P4-2 default). The verify driver POSTs multipart to http://localhost:<port>/stage; it does NOT raw-write the corpus disk by default. Direct disk-drop is recorded as a non-default, explicit-flag-gated fallback for the co-located case (the driver shares the host fs), implemented only if {62.4} chooses to. Keeping /stage load-bearing preserves the bytes-on-the-wire contract, centralises the path-escape guard + corpus-root resolution in ONE handler, and keeps the stageFixture client unchanged. This default also places the {42.10} Inv-9 round-trip + any Tier-3 topology assertions on the co-located driver reaching localhost (OQ-62-TECH-12). | Inv-12, Inv-23, Inv-27 |
OQ-62-P4-3 keep /health? | KEEP the cheap GET /health endpoint (server.py:_health_handler, 132–145) as an optional B1 container-liveness hook a Compose/Coolify healthcheck MAY reuse (ID-66 §1-f). The Cloud Run httpGet /health PROBE is gone with the Cloud Run manifest regardless. PRODUCT Inv-6 is re-framed as a /stage-must-not-break-/health regression note, NOT a probe requirement. B1 datapath monitoring (watch pipeline_runs arrival, ID-66 §1-f) is ID-66’s ops concern, out of ID-62 scope. | Inv-6 |
| OQ-62-11 (re-resolved for B1) verify-driver runtime/image | A thin in-repo Python module scripts/cocoindex_pipeline/verify_driver.py, run on the B1 host (python3 -m scripts.cocoindex_pipeline.verify_driver …) by the operator or a B1 push-to-deploy hook (ID-66 §1-d / OQ-66-6). It reads fixture bytes from the repo checkout on the host (docs/testing/test-data/**) and POSTs multipart to http://localhost:<port>/stage. NO dedicated container image, NO cnb-launcher, NO kh-pipeline Cloud Run image, NO Job manifest. (Recorded as OQ-62-TECH-11 in OQ-pending.md; default = in-repo module; override = Compose-stack container.) | Inv-7, Inv-8, Inv-10, Inv-11 |
| OQ-62-12 (re-resolved for B1) {42.10} Inv-9 + Tier-3 home | The co-located driver / B1 host runs them over localhost. The {42.10} Inv-9 /s/<share_id> round-trip is one GET http://localhost:3000/s/<id> within the driver invocation (pullmd is a localhost sibling — the old in-VPC-Job split DISSOLVES). The Tier-3 topology *.integration.test.ts files run on the B1 host’s Vitest go-live step where COCOINDEX_STAGING_URL resolves to localhost — they go live rather than skip. (Recorded as OQ-62-TECH-12; {62.4} decides per-file live-vs-defer.) | Inv-23, Inv-18 (Tier-3) |
| OQ-62-10 file→invariant map | Enumerated in §Testing “File → invariant tier map”. | Inv-18 |
DELETED (not carried): the OLD TECH’s empirical entry gate (old OQ-62-9 / old
Inv-34) — under B1 there is no internal-ingress Service to probe and no Cloud Run Job, so the
infra-side import-and-call check is moot (P4-RECONCILIATION §1, §4 item 7). The premise it
would have tested (internal→internal Cloud-Run-Job egress) was itself the thing the S281
inversion found false; B1 removes the surface entirely. NOT carried: old OQ-62-2
(Filestore-vs-ephemeral volume — moot, persistent local disk by construction), old
--set-secrets mechanics, old WIF gcloud run jobs execute.
AGPL image-isolation constraint (carried CHECKER NOTE from {62.2} — TECH owns container shape)
Section titled “AGPL image-isolation constraint (carried CHECKER NOTE from {62.2} — TECH owns container shape)”The
{62.2}Checker flagged that the AGPL boundary (old PRODUCT Inv-8) survives in PRODUCT prose (P4-RECONCILIATION §2 “AGPL boundary on the cocoindex image | Inv-8”) but is NOT a numbered PRODUCT invariant in the ratified{62.2}, and assigned ownership to{62.3}(TECH owns container shape). This TECH discharges that as an explicit, concretely-verifiable TECH-level constraint, mapped against PRODUCT Inv-8 (the verify driver “stages only, does not assert” invariant lives next to the AGPL boundary in the P4-RECONCILIATION crosswalk).
TECH-CONSTRAINT-AGPL. Under B1, pullmd MUST remain a sibling container on the host
(per the ID-66 §1-b Compose topology: pullmd :3000 + Playwright sidecar + Trafilatura sidecar
as their own service in the Compose network) and MUST NOT be baked into the cocoindex image.
The cocoindex container reaches pullmd over http://localhost:3000 via the env-configurable
PULLMD_SERVICE_URL (adapters.py:125 — confirmed env-driven, no hardcoded host). The
build-time cloudrun/cloudbuild-cocoindex.yaml assert-inv9-no-pullmd-no-playwright
assertion (117–167 — zero pullmd layer entries, zero Playwright/chromium markers in the
cocoindex image, exit 1 on any hit) continues to hold for the cocoindex image B1 deploys
(the same image is consumed by the Compose stack via the Dockerfile/Compose build pack, ID-66
§2.3). Adding POST /stage to server.py introduces NO pullmd/Playwright dependency — the
handler imports only os, pathlib, uuid, and the already-present aiohttp.web — so the
assertion stays green after the route lands.
Concretely verifiable: (a) cloudbuild-cocoindex.yaml:117–167 passes on the post-/stage
image (zero pullmd / Playwright); (b) the B1 Compose file (ID-66’s artefact) declares pullmd
as a separate service, NOT a layer in the cocoindex image; (c) the existing
agpl-boundary.integration.test.ts (image-inspect assertion) stays green on the GH runner.
The B1-equivalent of the build assertion: if the B1 push-to-deploy path (ID-66 §1-d) does NOT
re-run the cloudbuild assertion, {62.4}/ID-66 must wire an equivalent image-inspect check
into the B1 deploy so the AGPL boundary is not silently lost when the deploy mechanism changes
from Cloud Build to Compose/Coolify. (This wiring is an ID-66 deploy-path concern; ID-62
asserts the constraint and flags it.) [Inv-8]
Proposed changes
Section titled “Proposed changes”The work is five slices, one-to-one against P4-RECONCILIATION §4 (Slice A AS-IS, Slice B
reframed, Slice C′ re-architected, Slice D AS-IS, Slice E′ reframed) plus the {42.10}
invocation. Each change names the PRODUCT invariant(s) it satisfies. There is NO entry gate
and NO precondition probe — the only precondition is the ID-66 host existing (Task-level
dependency).
Slice A — POST /stage route — scripts/cocoindex_pipeline/server.py (PORTABLE AS-IS; topology-agnostic)
Section titled “Slice A — POST /stage route — scripts/cocoindex_pipeline/server.py (PORTABLE AS-IS; topology-agnostic)”Add _stage_handler(request) and register it in build_app():
app.router.add_post("/stage", _stage_handler) alongside the existing add_get("/health", …)
(160). The route is identical on Cloud Run or B1 — it writes to the local-fs corpus dir the
co-resident walk_dir(live=True) watcher polls (no process boundary, no network hop). The
handler:
- Resolve corpus root, loud-reject when unset/missing. Read
COCOINDEX_SOURCE_PATH. If unset/empty →return web.json_response({"error": "COCOINDEX_SOURCE_PATH is unset", …}, status=400). If set but the dir does notPath(...).exists()→status=400naming the path. (Client-correctable mis-wire → 4xx, never silent-accept; 5xx reserved for an unambiguous server-side mount failure.) Mirrors theapp_mainidle contract (1537/1546). [Inv-5] - Read the multipart body.
reader = await request.multipart(); iterate parts; capture thefilepart bytes (await part.read(decode=False)), thedestPathtext part, thetitlePrefixtext part. (aiohttpRequest.multipart/MultipartReader/BodyPartReader.readempirically PRESENT — see §Verification.) Reject (400) when thefilepart ordestPathis absent (a request carrying only a path string with no bytes is rejected). [Inv-2] - Write the bytes to the corpus-relative destination. Resolve
target = corpus_root / destPath; reject path-escape (os.path.realpath(target)must be withinrealpath(corpus_root); reject../, absolute paths) with 400 writing nothing;mkdir -pthe parent dirs under the corpus root; write the bytes totarget. The watcher (walk_dir(live=True), 1683) picks the file up on its next cycle. [Inv-3] - Respond 2xx echoing the dest path.
return web.json_response({"destPath": <relative or corpus-prefixed path written>, "requestId": <uuid4 hex>}, status=200).requestIdis informational (the helper reads{ destPath?, requestId? }131–136; absence is not an error). [Inv-4] - Co-resident, AGPL + health untouched. The route is registered on the SAME
build_app()app the daemon thread runs inside (no second process, no second Service). The handler imports no pullmd binary, no Playwright driver — the cloudbuild Inv assertion stays green (TECH-CONSTRAINT-AGPL)._health_handler/worker_is_healthy()are not touched — the worker-liveness contract is unchanged; adding/stagedoes not flip/healthto 503. [Inv-1, Inv-6, Inv-8]
Python unit coverage lands in scripts/tests/test_cocoindex_server.py (the existing
aiohttp_app fixture + aiohttp_app.router.resolve in-process pattern, 145–176): assert the
route table includes POST /stage; assert loud-reject 400 on unset/missing
COCOINDEX_SOURCE_PATH; assert a well-formed multipart write lands bytes at
corpus_root/destPath; assert path-escape rejection; assert /health still 200/503. Runs on
any runner under python3 -m pytest scripts/tests/ (no live host needed — build_app is
socket-free).
Slice B — corpus activation (REFRAMED for B1) — server.main() + B1 Compose env
Section titled “Slice B — corpus activation (REFRAMED for B1) — server.main() + B1 Compose env”COCOINDEX_SOURCE_PATH=/corpusin the B1 Compose env (ID-66’s Compose file, an ID-66 artefact — NOT a Cloud Run manifest), where/corpusis a directory on the host’s persistent local-disk volume (ID-66 §1-c). The corpus survives container restarts (NO ephemeral-tmpfs, NO Filestore, NO new volume design — the host disk).- Seed at startup in
server.main()(234–255), BEFOREstart_cocoindex_thread()(252):mkdir -p ${COCOINDEX_SOURCE_PATH}AND seed a valid${COCOINDEX_SOURCE_PATH}/.kh-workspace-map.jsonif absent. This is the load-bearing carry:app_mainaborts the WHOLE flow on a missing/unparseable/schema-invalid.kh-workspace-map.json(1582–1606,raiseat 1606), so it MUST exist at process start or every fixture produces zero rows. The seed must satisfyload_workspace_manifest’s schema (the minimal valid workspace map —{62.4}pins the exact shape from theload_workspace_manifestvalidator). This is a cocoindexapp_mainbehaviour (not a Cloud Run behaviour) and carries forward as aserver.main()step, portable as-is. - Activation is observable, never a false green. If the watch is unarmed (path unset → the
/stageloud-reject Inv-5 fires) or the dir exists but the manifest is invalid (app_main abort → the downstream Vitest poll times out diagnostically, Inv-18), the failure is loud — never a silent pass.
Update scripts/tests/test_cocoindex_service_manifests.py (or the B1-Compose-shape test
{62.4} chooses) to assert COCOINDEX_SOURCE_PATH == /corpus and the seed step’s presence.
[Inv-15, Inv-16, Inv-17, Inv-24]
Activation caveat (carry to
{62.4}+ the operator): corpus activation is NOT merely “set the env var” — the.kh-workspace-map.jsonmust be a SCHEMA-VALID manifest at the corpus root before the first fixture is staged, orapp_mainaborts and every run produces zero rows.{62.4}slice B owns the seed mechanics + the manifest shape.
Slice C′ — co-located verify driver (RE-ARCHITECTED; replaces the in-VPC Cloud Run Job)
Section titled “Slice C′ — co-located verify driver (RE-ARCHITECTED; replaces the in-VPC Cloud Run Job)”A new thin Python module scripts/cocoindex_pipeline/verify_driver.py (OQ-62-TECH-11), run
ON the B1 host:
-
Runs on the host, reaches
/stageover localhost. Invoked aspython3 -m scripts.cocoindex_pipeline.verify_driver --fixtures <set>by the operator or a B1 push-to-deploy/operator hook (ID-66 §1-d / OQ-66-6). It reads its(fixture set)— a list of(fixturePath-in-repo-checkout, destPath, titlePrefix)tuples — from the repo checkout on the host (docs/testing/test-data/**; the host HAS the repo, so the bytes-source ambiguity the old Cloud-Run-pod spec wrestled with dissolves). For each, it reads the bytes and POSTs multipart tohttp://localhost:<port>/stage(the default — Inv-12). Because it runs on the same host as the cocoindex container, the request reaches/stageover loopback / the host’s private container network — noingress: internalto defeat, no 403. NO Cloud Run Job, NO WIFgcloud run jobs execute, NO Secret-Manager--set-secrets. [Inv-7, Inv-12] -
Stages only; does not assert. It reads bytes and POSTs to
/stage, then exits. It does NOT poll Supabase, does NOT run the invariant SQL probes, does NOT embed a copy of the Vitest assertions. No Supabase query originates from the driver process. [Inv-8 (stages-only half), Inv-9] -
Exit-code semantics + idempotent re-stage. Exit 0 iff every
/stagePOST returned 2xx; exit non-zero on any 4xx/5xx/timeout/connection-refused, surfacing the failing fixture + the/stageresponse status/body so the operator can diagnose without re-running. Re-running re-stages the same fixtures (overwrite-on-name; the watcher re-processes on change); no clean-corpus precondition. Cleanup (dropFixture) is the test layer’s responsibility, not the driver’s. [Inv-10, Inv-11] -
Secrets from the B1 host store, no plaintext. The driver needs only the cocoindex port (a localhost URL) for staging; for the
{42.10}invocation it also reads the pullmd Bearer token + the Supabase service-role creds the Vitest step needs — all from the B1 host secrets store (Coolify env / host secrets manager, ID-66 §1-f / OQ-66-3; the STORE is ID-66’s to stand up). The behaviour invariant “no plaintext secret value in manifest/workflow/logs” survives; the driver reads from the host env, never echoing values. NO GCP Secret Manager, NO--set-secrets. [Inv-7 secrets half; the no-plaintext behaviour] -
Direct disk-drop fallback (non-default). Because the driver shares the host fs with the watcher, it COULD
cpbytes into${COCOINDEX_SOURCE_PATH}/<destPath>bypassing/stage. This is recorded as a permitted convenience gated behind an explicit non-default flag/mode (--disk-drop), NOT the default. The default codepath is the HTTP POST (OQ-62-P4-2). If{62.4}decides the fallback is not worth implementing, the driver is HTTP-only. [Inv-12]
Slice D — stageFixture wire-contract change — fixture-staging.ts (PORTABLE AS-IS)
Section titled “Slice D — stageFixture wire-contract change — fixture-staging.ts (PORTABLE AS-IS)”StageFixtureArgs (51–71) is unchanged (keeps fixturePath, destPath, titlePrefix —
callers compile unchanged, OQ-62-7). Inside stageFixture (100–139), replace the JSON body
(115–119) with multipart: read the bytes at args.fixturePath runner-side (Bun.file /
fs.readFile), build a FormData with a file part (a Blob/File named after
basename(destPath)), a destPath text part, and a titlePrefix text part; POST FormData
(drop the explicit Content-Type: application/json header at 115 — fetch sets the multipart
boundary). Keep the unset-env throw (105–108), the non-2xx throw (125–128), and the response
read { destPath?, requestId? } (131–136) verbatim. The behaviour invariant: bytes on the
wire, not a path the writer can’t see. pollContentItemsFor (179–222), dropFixture
(376+), and the form helpers are unchanged — they run wherever the Vitest tier runs (on
the B1 host under B1) against live Supabase (reachable from anywhere). [Inv-2, Inv-19,
Inv-22, Inv-26]
dropFixture’sentity_mentionscleanup stays best-effort (411–425) because ID-49.5 is deferred (S273 OQ-1). Carry to{62.4}: confirm theentity_mentionstable/FK shape when the live tier first produces rows so cleanup does not leak rows across runs (Inv-20).
Slice E′ — trigger + Vitest go-live (REFRAMED; replaces the WIF + gcloud run jobs execute mechanism)
Section titled “Slice E′ — trigger + Vitest go-live (REFRAMED; replaces the WIF + gcloud run jobs execute mechanism)”- The B1 trigger replaces
cocoindex-live-verify.yml’s WIF +gcloud run jobs execute. The live-tier verification is triggered on the B1 host: an operator runs the verify driver + the Vitest go-live step on the host, OR a B1 push-to-deploy/operator hook triggers it (ID-66 §1-d / OQ-66-6). NOgcloud run jobs execute, NO WIF, NO Cloud Run Job in the trigger path. KEEP the policy (topology-agnostic): on-demand, NOT inlined into the PR-blockingci.ymlintegrationjob, NOT scheduled today. The existing PR-blocking integration job is unchanged. [Inv-28] - The commented
schedule:/launch-flip marker survives. Whatever B1 trigger artefact{62.4}chooses (a host runbook step, a Coolify scheduled task stub, or a thin workflow that SSHes the host) carries a commented-out scheduled-cadence block with a# FLIP ON AT LAUNCH (ID-62 Inv-29)marker so the launch-time flip is a one-line uncomment. [Inv-29] - The Vitest go-live step +
COCOINDEX_*env block. After the driver stages, run the cocoindex Vitest tier (bun run test:integrationor a cocoindex-scoped invocation) on the B1 host with the env block:COCOINDEX_STAGING_URL(=http://localhost:<port>),COCOINDEX_FIXTURE_STAGING_URL(=http://localhost:<port>),COCOINDEX_SOURCE_PATH(=/corpus), plus the live-Supabase creds from the host secrets store. The previously-skipped tests now execute and assert against live Supabase. A non-zero driver exit fails before the Vitest step runs. [Inv-18, Inv-20] - Tier-3 topology tests reframe — REDUCED under B1 (cleaner than the Cloud-Run gymnastics).
Because the Vitest step runs ON the B1 host,
COCOINDEX_STAGING_URLresolves to localhost, so the Tier-3 files thatfetch ${COCOINDEX_STAGING_URL}/...(health-probe,sidecar-*,stage-topology,latency-budget,transient-retry,audit-log-shipping) reach localhost and run live — NO “skip-on-GH-runner + assert-in-Job exit surface” gymnastics.{62.4}decides per-file whether each goes live on the host now or is deferred (OQ-62-TECH-12). [Inv-18 (Tier-3 half)] - Inv-29 launch-flip action item.
{62.4}adds the recorded action item — “flip the live-tier verification to a scheduled/automatic cadence at launch” — to the T13 pre-launch-ops row ofdocs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md(line 241). Inv-29 is satisfied by the doc’s presence; the omission of a live schedule today is deliberate and documented. [Inv-29]
{42.10} invocation — SIMPLIFIED (same primitive + the HTML fixture)
Section titled “{42.10} invocation — SIMPLIFIED (same primitive + the HTML fixture)”The SAME verify driver + /stage route, invoked with the (HTML fixture, Inv-7/8/9 assertion set), produces the {42.10} proof — ONE primitive parameterised by (fixture set, assertion set); no HTML-special-casing branch. The driver stages one real HTML source via /stage; the
co-located worker ingests it; the assertions verify: (PRODUCT Inv-22) content_items.content_text
non-empty markdown + source_documents.extraction_method = pullmd_* matching the live
X-Source header (live-Supabase assertions, reachable from anywhere); (PRODUCT Inv-23) the
Inv-9 GET http://localhost:3000/s/<share_id> -> 200 non-empty round-trip asserting
pullmd_share_id = X-Share-Id, run from the co-located driver over localhost (pullmd is a
sibling container, ID-66 §1-b — the old “split-home-because-pullmd-is-internal” rationale
dissolves; the round-trip is one HTTP call within the driver invocation, not a separate
external curl). [Inv-21, Inv-22, Inv-23]
Testing and validation
Section titled “Testing and validation”No entry gate. Unlike the OLD TECH, there is NO operator-run reachability probe precondition. The only precondition is the ID-66 host existing (Task-level dependency, PRODUCT §ID-66-precondition). On a ready host, the first validation is simply running the verify driver + the Vitest go-live step.
Per-invariant verification map (1:1, all 29 Inv covered)
Section titled “Per-invariant verification map (1:1, all 29 Inv covered)”The Checker reads the right column as the acceptance criterion (do not restate the invariant text):
| Inv | Verification |
|---|---|
| Inv-1 co-resident route | Python unit (test_cocoindex_server.py): POST /stage registered on the SAME build_app() app the daemon thread runs in (no second Service in cloudrun/services/, no second process). |
| Inv-2 bytes in body | Python unit: well-formed multipart file part lands bytes at corpus_root/destPath; a path-only request (no file part) → 400. TS: stageFixture sends FormData, not JSON (assert no application/json header; assert a file part). |
Inv-3 write to ${SOURCE_PATH}/<destPath> + path-escape guard | Python unit: bytes readable at the resolved path under the corpus root; ..//absolute destPath → 400, nothing written outside root. |
| Inv-4 2xx echo destPath | Python unit: 200 + { destPath, requestId }; TS stageFixture reads destPath back. |
| Inv-5 loud-reject unset/missing | Python unit: COCOINDEX_SOURCE_PATH unset → 400 naming the var; set to a non-existent dir → 400 naming the path; neither returns 2xx, neither silently accepts. |
Inv-6 /health survives, /stage doesn’t break it | Python unit: existing /health 200(healthy)/503(crashed) tests still pass after /stage lands; staging a fixture does not flip /health to 503. (Regression note, not a probe.) |
| Inv-7 driver on host, localhost staging | Driver run on a host where cocoindex serves /stage on a localhost port stages a fixture; run off-host (no localhost /stage) fails fast. Driver source reads secrets from host env, no plaintext echoed in logs. |
| Inv-8 driver stages only + AGPL boundary | Driver source contains no Supabase query / SQL probe; cloudbuild-cocoindex.yaml:117–167 green on the post-/stage image; B1 Compose declares pullmd as a sibling service, not a cocoindex-image layer (TECH-CONSTRAINT-AGPL). |
| Inv-9 no duplicated assertion logic | Driver source contains no SQL probe equivalent to the Vitest invariant assertions; the Vitest tier is the single assertion surface. |
| Inv-10 exit-code semantics | A run where every /stage returns 2xx exits 0; a run where one fixture’s /stage returns 4xx exits non-zero and names the failing fixture + status/body in output. |
| Inv-11 idempotent re-stage | Running the driver twice in succession produces no error on the second run; the corpus reflects the most-recently-staged bytes; no clean-corpus precondition. |
Inv-12 /stage HTTP is the default | The driver’s default codepath issues an HTTP POST to http://localhost:<port>/stage (not a raw fs write); any disk-drop path is gated behind an explicit non-default flag. |
Inv-13 /stage not public | /stage binds to localhost / the host’s private container-network interface, reachable on-host, NOT from a public/external address (the B1 Compose port-binding — ID-66 §1-b; ID-62 introduces no public exposure). |
| Inv-14 localhost reachability, no isolation toggle | No ingress: all flip, no VPC connector, no self-hosted GitHub Actions runner registration introduced by ID-62; the only reachability mechanism is on-host localhost. |
| Inv-15 corpus on persistent local disk | A staged fixture is still present after the cocoindex container restarts (persistent-disk volume, ID-66 §1-c); no Filestore/GCS-FUSE/Cloud-Run corpus volume introduced. |
| Inv-16 activation observable, never false green | With the watch unarmed, a staged fixture produces no rows and the Vitest poll times out with the awaited title-prefix (a real failure signal), not a pass. |
| Inv-17 manifest must exist before watch arms | Staging into a corpus dir lacking a valid .kh-workspace-map.json produces a manifest_missing/manifest_invalid stage error (flow.py:1586–1606) and no content_items rows; with a valid seeded manifest, rows are produced. |
| Inv-18 Vitest tier is the assertion surface | The tier runs live (not skipIf-skipped) when the env is wired on the host; a landed row satisfies the poll; a row that never lands fails with the awaited title-prefix. File→invariant tier map below. |
Inv-19 stageFixture posts bytes | TS: helper reads fixturePath runner-side and POSTs multipart (a file part), not a bare path; env-gate (105–108) + non-2xx throw (125–128) preserved. |
| Inv-20 cleanup test-owned + scoped | dropFixture/dropFormFixture unchanged; PK-scoped + titlePrefix-guarded; empty titlePrefix refuses to run (383–385); entity_mentions best-effort carry confirmed (411–425). |
Inv-21 one primitive, {42.10} is one invocation | The {42.10} HTML proof + the ID-62 tier are driven by the same driver + /stage route with different (fixture set, assertion set) params; no HTML-specific staging branch. |
Inv-22 {42.10} content/extraction assertions live | Staging the HTML fixture yields a content_items row with non-empty markdown content_text + a source_documents row whose extraction_method is pullmd_* (live Supabase). |
Inv-23 {42.10} Inv-9 round-trip on the driver over localhost | The driver, run on the B1 host, issues GET http://localhost:3000/s/<id> for the staged HTML’s pullmd_share_id and asserts a 200 non-empty response; not attempted off-host. |
| Inv-24 OQ-62-1 corpus value+activation (B1) | COCOINDEX_SOURCE_PATH=/corpus on the persistent-disk volume in the B1 Compose env; server.main() mkdir -p + manifest seed before the thread spawns; container (re)start arms the watch. |
| Inv-25 OQ-62-6 titlePrefix per format | Filename-derived title (content tier, ilike('title', ...) 199) + dest-path-prefix matchStoragePath (form tier, 559); /stage performs no in-byte injection. |
| Inv-26 OQ-62-7 wire encoding | multipart/form-data (file part + destPath/titlePrefix text parts); StageFixtureArgs keeps fixturePath; helper reads bytes host-side. |
Inv-27 OQ-62-P4-2 /stage HTTP vs disk-drop | /stage HTTP over localhost is the default; disk-drop is an explicit non-default flag; the {42.10} round-trip + Tier-3 assertions run on the co-located driver. |
| Inv-28 on-demand, not PR-blocking | The live tier is not in the PR-blocking ci.yml integration job; it is invoked by an explicit operator/B1-host trigger; no gcloud run jobs execute/WIF in the trigger path. |
| Inv-29 launch-flip recorded | {62.4} adds the action item to canonical-pipeline-sequencing.md T13 (241); the commented scheduled-cadence/flip marker is present; no schedule wired today. |
File → invariant tier map (OQ-62-10)
Section titled “File → invariant tier map (OQ-62-10)”Grounded by grep over __tests__/integration/cocoindex/*.integration.test.ts (≈35 files
reference the gate/helpers; the authored assertion count is “39”) plus the two form files at
__tests__/integration/ root. Tiers tell {62.4} exactly what goes live and how each file
reaches its data. Under B1 the Vitest step runs ON the host, so all tiers — including
Tier-3 — can reach localhost.
Tier 1 — STAGE + poll content_items (driver stages via /stage; host-run Vitest polls
live Supabase). Call stageFixture (beforeAll) + pollContentItemsFor + dropFixture:
admin-merge-coexistence, classify-content-coexistence, context-snippet-populated,
cross-document-dedup, cross-workspace-isolation, extract-contract-honour,
idempotency-memo-hit, inv-1-content-items-row-produced, legacy-alias-preload,
nested-corpus, op-id-round-trip, op-id-scoping, op-id-stamping,
pair-resolver-determinism, per-doc-canonicalisation, stage-5-attach-point,
stage-5-op-id-memo, stage-5-row-counter, unresolved-mention-retains-canonical,
file-change-detection. Inv tier: Inv-1/Inv-3/Inv-17/Inv-18.
Tier 1b — poll-only content_items (no stage; depends on a Tier-1 fixture / memo state):
extract-memoisation, stage-5-failure-non-destructive, memo-hit-pipeline-run. {62.4}
confirms whether these need a sibling stage to have run first (sequencing within the driver
invocation).
Tier 2 — FORM stage + poll form_templates/form_template_fields (at
__tests__/integration/ root): form-extraction.integration.test.ts (the ID-52.13 substrate),
form-extraction-rls.integration.test.ts (RLS isolation, 0 stages). Need a valid seeded
/corpus/.kh-workspace-map.json (Slice B caveat) and poll on name/storage_path (Inv-25
form half). Inv tier: Inv-17/Inv-18/Inv-25.
Tier 3 — TOPOLOGY / health / version (gate on COCOINDEX_STAGING_URL; fetch the cocoindex
Service URL): health-probe, sidecar-cold-start, sidecar-mime-coverage,
sidecar-version-metadata, stage-topology, latency-budget, transient-retry,
audit-log-shipping. Under B1 these run on the host where COCOINDEX_STAGING_URL =
localhost — they go live (no skip gymnastics). {62.4} decides per-file live-vs-defer
(OQ-62-TECH-12). Inv tier: Inv-18 (Tier-3 half).
Tier 4 — AGPL boundary. agpl-boundary.integration.test.ts — asserts the cocoindex image
is pullmd/Playwright-free; overlaps the build-time cloudbuild-cocoindex.yaml assertion
(TECH-CONSTRAINT-AGPL / Inv-8). Runs anywhere (image inspect), no host needed.
Tier 5 — no-gate, always-run (no COCOINDEX_* dependency): extractor-version-cross-ref,
faiss-pin, no-partial-row-writes, non-pipeline-write, persistent-failure-dlq. Already
run; unaffected by ID-62.
Local validation commands (RELATIVE paths)
Section titled “Local validation commands (RELATIVE paths)”python3 -m pytest scripts/tests/test_cocoindex_server.py scripts/tests/test_cocoindex_service_manifests.py—/stageroute table, loud-reject 400, multipart write, path-escape rejection,/healthregression, corpus-path/seed assertion (any runner; no live host).bun run test— TS unit/guard suite (thestageFixturemultipart change must not break the helper’s typed contract;StageFixtureArgsis unchanged). Notebun run test, NOTbun test.- The live tier is proven by running the verify driver + the Vitest go-live step on the B1
host (NOT by
bun run testlocally — off-host there is no localhost/stage/pullmd, so the tier skips clean as designed). This is the on-demand operator/B1-host trigger (Inv-28), not a PR gate.
Risks and mitigations (B1-specific — replace the Cloud-Run risks)
Section titled “Risks and mitigations (B1-specific — replace the Cloud-Run risks)”- Host-not-ready precondition. ID-62 implementation cannot begin until the ID-66 B1 host
(Compose stack + persistent volume) exists (PRODUCT §ID-66-precondition). Mitigated: this is
a Task-level dependency
ID-62→ID-66(Task.dependencies[], the Orchestrator records the edge; OQ-62-P4-4); the Slice A/stageroute + Slice D helper change are host-agnostic and CAN be built + unit-tested ahead of the host (they don’t need a live host to passpytest/bun run test). Only the live-tier go-live (Slice C′/E′ on-host run) waits on ID-66. .kh-workspace-map.json-abort zero-rows.app_mainaborts the whole flow on a missing/unparseable/schema-invalid manifest (flow.py:1582–1606,raiseat 1606). If the seed is absent or schema-invalid, EVERY fixture produces zero rows. Mitigated: (a)server.main()seeds a schema-valid manifest before the thread spawns (Slice B); (b) the failure mode is observable — the Vitest poll times out diagnostically (Inv-16/Inv-18), never a false green. Carry to{62.4}as a sequencing dependency + pin the exactload_workspace_manifestschema.- Corpus-injection if
/stageis bound to a public interface./stagewrites arbitrary bytes that flow through the full extraction pipeline into the live DB. Mitigated:/stagebinds to localhost / the host’s private container-network interface only (Inv-13); ID-62 introduces no public exposure. The Bearer-token defence-in-depth on pullmd is retained on localhost (ID-66 §1-b). (Public-exposure hardening of the B1 host perimeter is an ID-66 ops concern; ID-62 asserts the not-public-binding constraint.) - AGPL boundary lost if the B1 deploy path drops the image-inspect assertion. The
build-time
assert-inv9-no-pullmd-no-playwrightruns in Cloud Build today; if the B1 push-to-deploy path (ID-66 §1-d) doesn’t re-run an equivalent, the AGPL boundary could silently regress. Mitigated: TECH-CONSTRAINT-AGPL flags this for{62.4}/ID-66 to wire an equivalent image-inspect check into the B1 deploy;agpl-boundary.integration.test.ts(Tier 4) provides a test-time backstop. - Index-staleness in grounding. This worktree is not separately indexed; all line cites
were re-grounded by direct read this session (Context §Index-freshness caveat). Mitigated:
the Checker re-reads the cited surfaces against the working tree — the symbol identities
match the gitnexus
definitionsand the line numbers match the reads within ±1. proxy.tsis NOT involved./stageis on the cocoindex aiohttp server, not a Next.js route — nopublicRoutesallowlist entry is needed (that gotcha applies only to Next.js API routes).
External-API verification (OQ-3 / Q-EX2 — empirical import-and-call)
Section titled “External-API verification (OQ-3 / Q-EX2 — empirical import-and-call)”The B1 topology change (Cloud Run in-VPC → co-located localhost) is agnostic to the
route/watcher code — the /stage route, the walk_dir(live=True) watcher, and the aiohttp
server run identically regardless of where the host sits. This re-spec cites the SAME external
symbols as the OLD TECH (aiohttp multipart + cocoindex walk_dir) and NO new ones. They were
re-verified against the installed pins this session:
- Date: 29/05/2026.
- Pinned versions:
aiohttp>=3.9.0,<4.0.0(requirements.txt:59; installed 3.13.5),cocoindex[postgres]==1.0.3(requirements.txt:44; installed 1.0.3). - Symbols checked (import-and-call, this session):
aiohttp.web.Request.multipart→ PRESENT (<function BaseRequest.multipart>).aiohttp.web.Request.post→ PRESENT (<function BaseRequest.post>).aiohttp.MultipartReader→ PRESENT (<class 'aiohttp.multipart.MultipartReader'>).aiohttp.BodyPartReader.read→ PRESENT (<function BodyPartReader.read>).aiohttp.BodyPartReader.filename→ PRESENT (cached-property descriptor).aiohttp.web.json_response→ PRESENT (<function json_response>).cocoindex.connectors.localfs.walk_dir→ PRESENT (<function walk_dir>; already used atflow.py:1683, empirical-verification docstringserver.py:191).
- Result: all PRESENT — no SIGNATURE_DRIFT, no ABSENT, no BEHAVIOUR_DRIFT. The
/stagemultipart byte-read path and thewalk_dirwatcher are safe to build against the installed pins. - TS side:
FormData/Blob/fetchare Web-standard (Bun runtime built-ins) — out of OQ-3 scope (framework/stdlib built-ins).StageFixtureArgsis internal KH (out of scope). - The OLD spec’s one infra import-and-call analogue — the internal→internal entry gate (old Inv-34 / OQ-62-9) — is DELETED, not carried: under B1 there is no internal-ingress Service to probe and no Cloud Run Job, so the gate is moot (P4-RECONCILIATION §4 item 7).
Verification table
Section titled “Verification table”| Date | Pinned version | Symbol path checked | Result |
|---|---|---|---|
| 29/05/2026 | aiohttp 3.13.5 (requirements.txt:59) | aiohttp.web.Request.multipart | PRESENT |
| 29/05/2026 | aiohttp 3.13.5 | aiohttp.web.Request.post | PRESENT |
| 29/05/2026 | aiohttp 3.13.5 | aiohttp.MultipartReader | PRESENT |
| 29/05/2026 | aiohttp 3.13.5 | aiohttp.BodyPartReader.read | PRESENT |
| 29/05/2026 | aiohttp 3.13.5 | aiohttp.BodyPartReader.filename | PRESENT (cached-property) |
| 29/05/2026 | aiohttp 3.13.5 | aiohttp.web.json_response | PRESENT |
| 29/05/2026 | cocoindex[postgres]==1.0.3 (requirements.txt:44) | cocoindex.connectors.localfs.walk_dir | PRESENT (call site flow.py:1683; docstring server.py:191) |
{62.4} PLAN slice outline (NOT authored here — for {62.4} to formalise)
Section titled “{62.4} PLAN slice outline (NOT authored here — for {62.4} to formalise)”The recommended decomposition (sibling-only Subtask deps within Task 62; NO entry-gate precondition Subtask — the only precondition is the ID-66 host, a Task-level dependency):
- Slice A —
/stageroute._stage_handler+build_appregistration inserver.py; Python unit tests intest_cocoindex_server.py(route table, loud-reject, multipart write, path-escape,/healthregression). Host-agnostic — buildable ahead of the ID-66 host. Sibling-dep: none. [Inv-1/2/3/4/5/6/8] - Slice B — corpus activation.
server.main()mkdir -p+ schema-valid.kh-workspace-map.jsonseed beforestart_cocoindex_thread();COCOINDEX_SOURCE_PATH=/corpusin the B1 Compose env (an ID-66 artefact ID-62 documents); manifest-shape test. Sibling-dep: A. [Inv-15/16/17/24] - Slice C′ — co-located verify driver. New
scripts/cocoindex_pipeline/verify_driver.py(in-repo Python module; HTTP-POST multipart to localhost/stage; stage-only; exit-code; idempotent; host-secrets-store; optional--disk-dropnon-default). Sibling-dep: A (the driver POSTs to/stage). [Inv-7/8/9/10/11/12/13/14] - Slice D — helper wire-contract change.
stageFixtureJSON→multipart infixture-staging.ts(StageFixtureArgsunchanged). Sibling-dep: A (the wire must match the route’s multipart contract). [Inv-2/19/20/22/25/26] - Slice E′ — trigger + Vitest go-live +
{42.10}invocation + sequencing-doc item. The B1 trigger artefact (operator runbook / host hook, NOTgcloud run jobs execute) with the commented launch-flip marker; theCOCOINDEX_*Vitest env block run on-host; the{42.10}HTML-fixture/Inv-7-8-9 invocation (Inv-9 round-trip over localhost); the Tier-3 per-file live-vs-defer decision; add the Inv-29 launch-flip action item tocanonical-pipeline-sequencing.mdT13. Sibling-dep: B, C′, D (needs the activated corpus, the driver, and the byte-posting helper). [Inv-18/21/22/23/27/28/29]
This is 5 code slices + 0 precondition gates — well within the 25-Subtask soft ceiling, all
dependencies sibling-only (within Task 62). No cross-Task Subtask dependency is implied; ID-42’s
{42.10} is unblocked by ID-62 at the Task level, and ID-62 depends on ID-66 at the
Task level (Task.dependencies[]) — both consistent with the §3.3 sibling-only constraint.