PLAN — Fixture-staging live-verification infra — co-located /stage route + localhost verify driver (B1 on-prem) (ID-62.4)
PLAN — Fixture-staging live-verification infra — co-located /stage route + localhost verify driver (B1 on-prem) (ID-62.4)
Section titled “PLAN — Fixture-staging live-verification infra — co-located /stage route + localhost verify driver (B1 on-prem) (ID-62.4)”Task: ID-62 — Fixture-staging live-verification infra.
Subtask: {62.4} PLAN.
Predecessors (read in full): {62.2} PRODUCT
(docs/specs/id-62-fixture-staging-infra/PRODUCT.md, 29 invariants Inv-1..29, ratified S283
under B1) + {62.3} TECH (docs/specs/id-62-fixture-staging-infra/TECH.md, Slices A/B/C′/D/E′
{42.10}, 1:1 per-invariant verification map, ratified S283) + the ratified directive setdocs/specs/id-62-fixture-staging-infra/P4-RECONCILIATION.md(§4 net shape) + the B1 host modeldocs/specs/ID-66-onprem-pivot/RESEARCH.md(§1-a..f). Author date: 29/05/2026 (S283).
SUPERSEDES the S279 P2 “in-VPC Cloud Run Job + operator entry-gate
{62.0}” PLAN. This is a wholesale re-spec, not an amendment. The prior PLAN decomposed an in-VPC Cloud Run Job verify mechanism (old{62.7}), akh-pipeline-image reuse +cnb-launcher entrypoint, acocoindex-live-verify.ymldoing WIF +gcloud run jobs execute, and an operator entry-gate{62.0}(curl internal/healthfrom a throwaway Job) carried as a blocking note in every Subtask. The S282 ratification of Option B1 (co-locate cocoindex + pullmd on one on-prem host — ID-66 RESEARCH) dissolved that design’s organising spine. The whole Cloud-Run-Job / WIF /cnb-launcher / entry-gate block is deleted, not relabelled. Git history preserves the old version. Cross-reference: P4-RECONCILIATION §1 (WHOLESALE verdict), §4 (B1 net shape).
This is the
{62.4}artefact authored by a fresh Planner instance (Q-PLANNER-2 — NOT the{62.1}/{62.2}/{62.3}author). It formalises TECH’s{62.4}slice outline into TM-shape Subtask records. It does NOT relitigate the ratified basis (B1 co-located host; localhost reachability;/stage-load-bearing default per OQ-62-P4-2;/healthkept per OQ-62-P4-3; entry gate deleted; in-repo Python verify-driver module per OQ-62-TECH-11; co-located driver / host Vitest for the{42.10}Inv-9 + Tier-3 home per OQ-62-TECH-12). It does NOT author code and does NOT edittask-list.json— the Orchestrator writes the{62.5+}records from §6 below intodocs/reference/task-list.json.
1. Decomposition shape
Section titled “1. Decomposition shape”Six implementation Subtasks {62.5}–{62.10}, sibling-only dependencies within Task 62.
There is NO entry-gate Subtask and NO entry-gate blocking note — the OLD PLAN’s operator
precondition ({62.0}, OQ-62-9: curl internal /health from a Cloud Run Job) is deleted
because it is moot under B1 (no internal-ingress Service to probe, no Cloud Run Job —
P4-RECONCILIATION §1, §4 item 7; TECH §“DELETED (not carried)”). It is replaced by the B1
dual implementation-gating precondition (§4 below), carried as a blocking note in every
{62.5+} details field.
This honours the §3.3 sibling-only constraint: no {62.M} depends on any other Task’s
Subtask. Two Task-level edges (NOT Subtask cross-deps) are recorded by the Orchestrator in
task-list.json: (a) ID-62 depends on ID-66 (the B1 host must exist — PRODUCT
§ID-66-precondition / OQ-62-P4-4); (b) ID-42 depends on ID-62 (ID-62’s primitive unblocks
{42.10} at the Task level — TECH closing §).
Subtask → TECH slice → PRODUCT-invariant map
Section titled “Subtask → TECH slice → PRODUCT-invariant map”| Subtask | TECH slice | PRODUCT invariants | Sibling deps |
|---|---|---|---|
{62.5} POST /stage route | Slice A | Inv-1, Inv-2, Inv-3, Inv-4, Inv-5, Inv-6, Inv-8 | none |
{62.6} corpus activation + manifest seed | Slice B | Inv-15, Inv-16, Inv-17, Inv-24 | {62.5} |
{62.7} co-located verify driver | Slice C′ | Inv-7, Inv-8, Inv-9, Inv-10, Inv-11, Inv-12, Inv-13, Inv-14 | {62.5} |
{62.8} stageFixture JSON→multipart | Slice D | Inv-2, Inv-19, Inv-20, Inv-22, Inv-25, Inv-26 | {62.5} |
{62.9} B1 trigger + Vitest go-live | Slice E′ | Inv-18, Inv-20, Inv-28, Inv-29 | {62.6}, {62.7}, {62.8} |
{62.10} {42.10} parameterised invocation | {42.10} | Inv-21, Inv-22, Inv-23, Inv-27 | {62.9} |
Count: 6 of 25 Subtasks — well within the §3.4 soft ceiling. No Task-boundary split is
signalled. TECH outlined Slice E′ + the {42.10} invocation as one combined slice; this PLAN
splits them into {62.9} (the ID-62 trigger + host Vitest go-live) and {62.10} (the
{42.10} HTML invocation) because they have distinct invariant sets (Inv-18/20/28/29 vs
Inv-21/22/23/27), {62.10} adds the Inv-23 localhost pullmd round-trip on top of the driver,
and {62.10} is the ID-42 Task-level payoff the Orchestrator may want to dispatch / verify
independently. The split keeps each Subtask’s details load-bearing without one oversized
brief.
AGPL constraint home: the AGPL image-isolation constraint (TECH-CONSTRAINT-AGPL, mapped to
PRODUCT Inv-8) lands in two Subtasks: {62.5} (the /stage handler adds NO
pullmd/Playwright dependency, so the cocoindex image stays pullmd/Playwright-free) and
{62.7} (the verify driver asserts the AGPL boundary held + flags the B1-deploy-path
image-inspect equivalent for ID-66 to wire). See §3 carry 4 and the two details fields.
2. PLAN-level decisions (carried from TECH by reference — NOT re-opened)
Section titled “2. PLAN-level decisions (carried from TECH by reference — NOT re-opened)”All PLAN-level OQs were RESOLVED in {62.3} TECH §“Resolved TECH-level OQs”. This PLAN
carries them by reference and bakes the consequences into the relevant Subtask details; it
does not re-open them.
- OQ-62-11 (verify-driver runtime) — RESOLVED in TECH §C′. A thin in-repo Python module
scripts/cocoindex_pipeline/verify_driver.py, run on the B1 host (python3 -m scripts.cocoindex_pipeline.verify_driver). NOT a Cloud Run image, NOcnb-launcher, NO Job manifest. Reads fixtures from the host’s repo checkout (docs/testing/test-data/**). Carried into{62.7}details. (Recorded asOQ-62-TECH-11inOQ-pending.md; default = in-repo module.) - OQ-62-12 (
{42.10}Inv-9 + Tier-3 home) — RESOLVED in TECH §C′/§E′/§{42.10}. The co-located driver / B1 host runs them over localhost: the Inv-9/s/<id>round-trip is oneGET http://localhost:3000/s/<id>within the driver invocation; the Tier-3 topology files go live on the host’s Vitest step whereCOCOINDEX_STAGING_URLresolves to localhost. Carried into{62.9}/{62.10}details. (Recorded asOQ-62-TECH-12; per-file live-vs-defer is{62.9}’s call.) - OQ-62-P4-2 (
/stageHTTP vs direct disk-drop) — RESOLVED in TECH./stageHTTP over localhost is the load-bearing default; direct disk-drop is a non-default, explicit---disk-drop-flag-gated fallback the driver MAY omit. Carried into{62.7}details. (Recorded asOQ-62-P4-2inOQ-pending.md.) - OQ-62-P4-3 (keep
/health?) — RESOLVED in TECH. KEEP the cheapGET /healthendpoint as an optional B1 container-liveness hook; PRODUCT Inv-6 is a/stage-must-not-break-/healthregression note, not a probe requirement. Carried into{62.5}details. - OQ-62-6 (
titlePrefixinjection) — RESOLVED in TECH. Filename-derived title, NO in-byte injection; the caller embedsTEST_PREFIXin the dest filename; the form tier pollsname/storage_pathviamatchStoragePath. Carried into{62.5}/{62.8}details. - OQ-62-7 (wire encoding) — RESOLVED in TECH. multipart/form-data (one
filepart + text partsdestPath,titlePrefix), NOT base64-in-JSON;StageFixtureArgskeepsfixturePathunchanged (helper reads bytes runner-side). Carried into{62.5}/{62.8}details.
3. Load-bearing sequencing carries (from TECH + the brief, grounded at source)
Section titled “3. Load-bearing sequencing carries (from TECH + the brief, grounded at source)”-
/corpus/.kh-workspace-map.jsonMUST exist at process start BEFOREapp_mainarms the watch.flow.py:app_main(1523–1867) readsCOCOINDEX_SOURCE_PATH(1536), loadssource_path/".kh-workspace-map.json"(1582), and on a missing/unparseable/schema-invalid manifest emits a structuredmanifest_missing/manifest_invalidstage error (1586–1595) + afailedpipeline-run webhook (1596–1605) thenraises (1606) — aborting the WHOLE flow. The watch arms once atapp_main; today the value is""→ idle return (1537–1543). Consequence: corpus activation ({62.6}) is NOT merely “setCOCOINDEX_SOURCE_PATH=/corpus” —server.main()must guarantee a schema-valid manifest exists at the corpus root beforestart_cocoindex_thread()spawns the worker, or EVERY fixture produces zero rows. Baked into{62.6}detailsas the FIRST step within the slice;{62.6}pins the exact minimal-valid shape from theload_workspace_manifestvalidator. -
The
mkdir -p+ manifest seed belongs inserver.main()(Python), at the pre-thread-spawn step. TECH §B placesmkdir -p ${COCOINDEX_SOURCE_PATH}+ the manifest seed insideserver.main()(234–255), BEFOREstart_cocoindex_thread()(252). On B1 there is no Cloud Run manifest and no buildpackcnb-launcher PATH trap (those were Cloud-Run-era; the OLD PLAN’s launcher reasoning is DELETED), but the placement reasoning still holds: the seed lives in the Python the container already runs, not a shell entrypoint wrapper.{62.6}detailsmandates this placement. -
Tier-3 topology tests reach localhost on the B1 host — NO skip-on-GH-runner gymnastics. Under B1 the Vitest go-live step runs ON the host, so
COCOINDEX_STAGING_URLresolves to localhost and the 8 Tier-3 files (health-probe,sidecar-cold-start,sidecar-mime-coverage,sidecar-version-metadata,stage-topology,latency-budget,transient-retry,audit-log-shipping) go live rather than skipping (TECH §E′ Tier-3 reframe — REDUCED under B1).{62.9}detailsmandates a per-file live-vs-defer decision (OQ-62-TECH-12), never a silently-green skip while claiming the topology tier is verified. The TECH §Testing “File → invariant tier map” is the per-file ledger. -
The AGPL boundary must survive the B1 deploy-mechanism change. The build-time
cloudrun/cloudbuild-cocoindex.yaml:assert-inv9-no-pullmd-no-playwright(117–167) keeps pullmd/Playwright OUT of the cocoindex image.{62.5}keeps the image clean (the/stagehandler imports onlyos/pathlib/uuid+ already-presentaiohttp.web). On B1, pullmd is a sibling container (ID-66 §1-b), NOT a cocoindex-image layer — the same container-isolation logic.{62.7}flags that the B1 push-to-deploy path (ID-66 §1-d) MUST wire an equivalent image-inspect check if it does not re-run the cloudbuild assertion, so the AGPL boundary is not silently lost when the deploy mechanism changes from Cloud Build to Compose/Coolify (this wiring is an ID-66 deploy-path concern; ID-62 asserts the constraint and flags it). Theagpl-boundary.integration.test.ts(Tier 4, runs anywhere via image inspect) is the test-time backstop. Carried into{62.5}and{62.7}details.
4. IMPLEMENTATION-GATED precondition (replaces the deleted entry gate)
Section titled “4. IMPLEMENTATION-GATED precondition (replaces the deleted entry gate)”The OLD PLAN’s operator entry-gate
{62.0}(curl internal/healthfrom a Cloud Run Job) is DELETED — moot under B1 (no internal-ingress Service to probe, no Cloud Run Job). It is REPLACED by the B1 dual implementation-gating precondition below, carried verbatim as a blocking note in every{62.5+}detailsfield.
IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH:
- (a) this B1 re-spec is ratified by Liam (the
{62.2}PRODUCT +{62.3}TECH + this{62.4}PLAN chain), AND - (b) ID-66 host stand-up is complete: the B1 host + Compose stack co-locating cocoindex + pullmd over localhost + a persistent local-disk volume hosting the corpus directory and the cocoindex LMDB all exist (ID-66 RESEARCH §1-a / §1-b / §1-c).
The verify driver and the corpus live ON the ID-66 host; without it there is nothing to run
against. This is a Task-level dependency (ID-62 depends on ID-66 via
Task.dependencies[], OQ-62-P4-4), consistent with the §3.3 sibling-only Subtask-dep
constraint — it is NOT a Subtask cross-dependency. The Orchestrator records the edge in
task-list.json; the Planner flags it.
What CAN proceed before (b): the Slice A /stage route ({62.5}) and the Slice D helper
change ({62.8}) are host-agnostic — they pass python3 -m pytest scripts/tests/ and
bun run test respectively without a live host. Only the live-tier go-live ({62.9}/{62.10}
on-host run) and the corpus-activation verification ({62.6}’s live arming) wait on the ID-66
host. The Orchestrator MAY dispatch {62.5}/{62.8} ahead of the host once (a) holds; their
details say so.
5. External-API verification (OQ-3 / Q-EX2)
Section titled “5. External-API verification (OQ-3 / Q-EX2)”This PLAN introduces NO NEW third-party library symbols beyond those {62.3} TECH already
empirically verified this session (29/05/2026, recorded in TECH §Verification):
- aiohttp multipart symbols —
aiohttp.web.Request.multipart,aiohttp.web.Request.post,aiohttp.MultipartReader,aiohttp.BodyPartReader.read,aiohttp.BodyPartReader.filename,aiohttp.web.json_response— all PRESENT against installedaiohttp 3.13.5(pinaiohttp>=3.9.0,<4.0.0,requirements.txt:59). cocoindex.connectors.localfs.walk_dir— PRESENT againstcocoindex[postgres]==1.0.3(requirements.txt:44; call siteflow.py:1683).
TS-side FormData/Blob/fetch (helper change {62.8}) are Web-standard Bun runtime
built-ins — out of OQ-3 scope (framework/stdlib built-ins). StageFixtureArgs is internal KH
(out of scope). The verify driver ({62.7}) uses stdlib + an already-pinned in-repo HTTP
client (aiohttp/urllib/requests). No re-verification required at PLAN time; the
{62.7}/{62.10} details carry the TECH verification block by reference so the Executor
builds against the verified symbol set.
The OLD spec’s one infra import-and-call analogue — the internal→internal entry gate (old Inv-34 / OQ-62-9) — is DELETED, not carried (TECH §“DELETED”).
Code-intelligence orientation (PLAN-level, verbatim — per the planner-citation block)
Section titled “Code-intelligence orientation (PLAN-level, verbatim — per the planner-citation block)”gitnexus_query({query: 'cocoindex server stage route corpus watcher verify driver fixture staging', repo: '…/knowledge-hub'})— returned"processes": [],"process_symbols": [](the aiohttp server surface is NOT modelled as an execution flow), but itsdefinitionssurfaced the load-bearing Python symbols this decomposition touches/extends, all confirming the PRODUCT/TECH grounding:Function:scripts/cocoindex_pipeline/flow.py:app_main(1523–1867),Function:scripts/cocoindex_pipeline/flow.py:ingest_file(1039–1489),Function:scripts/cocoindex_pipeline/flow.py:_emit_stage_error_log(363–396), and the Python test homeFunction:scripts/tests/test_cocoindex_server.py:aiohttp_app(144–154) +TestWorkerLiveness.test_health_503_when_worker_crashed(372–390). Out-of-index caveat: the TS staging helpers (stageFixture,pollContentItemsFor,dropFixture) are NOT in the GitNexus index (__tests__/integration/is outside the indexed TS corpus) and did NOT appear — grounded by direct file read in PRODUCT/TECH, carried by reference here.…/knowledge-hubis a placeholder for the repo path (do not record the real machine path).- ast-dataflow / ts-morph — not applicable:
server.py/flow.py/verify_driver.pyare Python (ts-morph does not cover Python per.ast-dataflow/CLAUDE.md), and the one TS surface (fixture-staging.ts) is a test-corpus helper. The code-touching Subtasks’detailscarry the discipline accordingly (gitnexus_impact/detect_changes for the indexed Python server symbols; direct read + grep for the Python verify-driver + the out-of-index TS helper).
6. Subtask records ({62.5}–{62.10}) — TM-shape, for the Orchestrator to write
Section titled “6. Subtask records ({62.5}–{62.10}) — TM-shape, for the Orchestrator to write”
idis local to Task 62 (integer).dependenciesare sibling-only (integers, other Subtasks of Task 62).description≤250 chars andtestStrategy≤300 chars are HARD-GATED by the ledger CLI — every record below is authored within budget; all load-bearing narrative lives in the unbudgeteddetailsfield. The Orchestrator appends these todocs/reference/task-list.jsonunder Task 62; this PLAN does NOT write the ledger.
{62.5} — POST /stage route in server.py
Section titled “{62.5} — POST /stage route in server.py”- title: Add co-resident
POST /stagemultipart byte-drop route to cocoindex server - description: Add
_stage_handler+build_app()registration inserver.py: read multipart body, loud-reject 400 on unset/missingCOCOINDEX_SOURCE_PATH, write bytes under corpus root, reject path-escape, echo{destPath, requestId}. - details: IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH (a)
this B1 re-spec is ratified by Liam, AND (b) ID-66 host stand-up is complete (B1 host +
Compose co-locating cocoindex+pullmd + persistent local-disk volume — ID-66 §1-a/b/c).
NOTE: this Subtask is host-agnostic — it passes
python3 -m pytestwithout a live host, so the Orchestrator MAY dispatch it once (a) holds, ahead of (b). The verify driver + corpus live on the ID-66 host. Editscripts/cocoindex_pipeline/server.py: addasync def _stage_handler(request)and registerapp.router.add_post("/stage", _stage_handler)inbuild_app()(153–161) alongside the existingadd_get("/health", _health_handler)(160). Handler steps (TECH §Slice A): (1) reados.environ.get("COCOINDEX_SOURCE_PATH"); if unset/empty →web.json_response({"error": "COCOINDEX_SOURCE_PATH is unset"}, status=400); if set butnot Path(p).exists()→ 400 naming the path; 5xx reserved for an unambiguous server-side mount failure [Inv-5]; (2)reader = await request.multipart(); iterate parts; capture thefilepart bytes viaawait part.read(decode=False), thedestPath+titlePrefixtext parts; 400 iffileordestPathabsent (a path-only request with no bytes is rejected) [Inv-2]; (3) resolvetarget = corpus_root / destPath; reject path-escape (os.path.realpath(target)must be withinrealpath(corpus_root); reject..//absolute) with 400 writing nothing;os.makedirs(target.parent, exist_ok=True); write bytes [Inv-3]; (4)web.json_response({"destPath": <written path>, "requestId": uuid.uuid4().hex}, status=200)[Inv-4]; (5) import onlyos/pathlib/uuid+ already-presentaiohttp.web— NO pullmd, NO Playwright, so the cloudbuildassert-inv9-no-pullmd-no-playwright(cloudrun/cloudbuild-cocoindex.yaml:117–167) stays green (TECH-CONSTRAINT-AGPL) [Inv-8]; do NOT touch_health_handler/worker_is_healthy—/stagemust NOT flip/healthto 503 [Inv-6]. The route is registered on the SAMEbuild_app()app the daemon thread runs inside (no second process/Service) [Inv-1]. OQ-62-6 carry:/stageperforms NO in-byte title injection — the caller embedsTEST_PREFIXin the dest filename andingest_filederives the title from the source path. OQ-62-7 carry: multipart/form-data, NOT base64-in-JSON. Tests inscripts/tests/test_cocoindex_server.py(existingaiohttp_appfixture 144–154 +aiohttp_app.router.resolve/make_mocked_requestin-process pattern 158–176, no socket): assert the route table includesPOST /stage; assert 400 on unset + on missing-dirCOCOINDEX_SOURCE_PATH(monkeypatch env); assert a well-formed multipart write lands bytes atcorpus_root/destPath(tmp_path corpus); assert path-escape 400; assert existing/health200/503 tests still pass. Runpython3 -m pytest scripts/tests/test_cocoindex_server.py. aiohttp multipart symbols verified PRESENT in TECH §Verification (29/05/2026, aiohttp 3.13.5) — build against that set. Code-intel: this is a Python symbol (ast-dataflow/ts-morph N/A — Python is not covered; use direct read + grep);build_appis indexed in GitNexus moduleCocoindex_pipeline— rungitnexus_impact({target: "build_app", direction: "upstream"})before editing andgitnexus_detect_changes()before commit. - testStrategy:
python3 -m pytest scripts/tests/test_cocoindex_server.pygreen: route table includesPOST /stage; 400 (named, not 5xx, not silent) on unset/missingCOCOINDEX_SOURCE_PATH; multipart write lands bytes under the corpus root; path-escape rejected;/health200/503 unchanged. - status: pending
- dependencies: []
{62.6} — Corpus activation (COCOINDEX_SOURCE_PATH=/corpus + boot mkdir + manifest seed)
Section titled “{62.6} — Corpus activation (COCOINDEX_SOURCE_PATH=/corpus + boot mkdir + manifest seed)”- title: Activate cocoindex corpus watch on B1 persistent disk (
/corpus+ boot seed) - description: Set
COCOINDEX_SOURCE_PATH=/corpusin the B1 Compose env; seed/corpus+ a schema-valid.kh-workspace-map.jsoninserver.main()before the worker spawns soapp_mainarms the watch instead of aborting; add a manifest-shape test. - details: IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH (a)
this B1 re-spec is ratified by Liam, AND (b) ID-66 host stand-up is complete (B1 host +
Compose co-locating cocoindex+pullmd + persistent local-disk volume — ID-66 §1-a/b/c). The
corpus lives on the ID-66 host’s persistent disk; live arming verification waits on (b).
SEQUENCING (LOAD-BEARING, PLAN §3.1):
app_main(flow.py:1523–1867) readsCOCOINDEX_SOURCE_PATH(1536) then loadssource_path/".kh-workspace-map.json"(1582) and emits amanifest_missing/manifest_invalidstage error (1586–1595) + afailedwebhook (1596–1605) thenraises (1606) on a missing/unparseable/schema-invalid manifest. The watch arms ONCE atapp_main; today""= idle return (1537–1543). So a schema-valid manifest MUST be present at the corpus root at process start, not merely before the first fixture, or EVERY fixture produces zero rows. Changes: (1) documentCOCOINDEX_SOURCE_PATH=/corpusin the B1 Compose env (ID-66’s Compose file — NOT a Cloud Run manifest), where/corpusis a directory on the host’s persistent local-disk volume hosting the cocoindex LMDB (ID-66 §1-c); the corpus survives container restarts — NO Filestore, NO ephemeral-tmpfs caveat, NO new volume design [Inv-15, Inv-24]. (2)scripts/cocoindex_pipeline/server.pymain()(234–255) — BEFOREstart_cocoindex_thread()(252):source = os.environ.get("COCOINDEX_SOURCE_PATH"); if set,os.makedirs(source, exist_ok=True)then, ifPath(source)/".kh-workspace-map.json"is absent, write a minimal schema-valid manifest. Pin the EXACT minimal shape from theload_workspace_manifestvalidator (read the validator to confirm the minimal-valid fields; the degenerate-but-legal shape, e.g. a schema_version + empty mappings list, must parse without raising). The seed lives in the Python the container runs, NOT a shell entrypoint wrapper (PLAN §3.2) [Inv-17]. (3)scripts/tests/test_cocoindex_service_manifests.py(or the B1-Compose-shape test the slice chooses): assertCOCOINDEX_SOURCE_PATH == "/corpus"; add aserver.main()test (or extendtest_cocoindex_server.py) that withCOCOINDEX_SOURCE_PATH=<tmp>set, the pre-spawn block creates the dir + seeds a manifest thatload_workspace_manifestparses without raising. Activation = container (re)start (the watch arms once atapp_main); the failure mode is observable, never a false green — unset path →/stageloud-reject (Inv-5), invalid manifest → app_main abort → downstream Vitest poll times out diagnostically (Inv-16/Inv-18) [Inv-16]. Code-intel:main/app_mainare Python symbols in GitNexus moduleCocoindex_pipeline(ast-dataflow/ts-morph N/A — Python; use direct read + grep); rungitnexus_impact({target: "main", direction: "upstream"})before editing +gitnexus_detect_changes()before commit. - testStrategy:
python3 -m pytest scripts/tests/test_cocoindex_service_manifests.py scripts/tests/test_cocoindex_server.pygreen:COCOINDEX_SOURCE_PATH == /corpus, no corpus volume added;main()seeds/corpus+ a.kh-workspace-map.jsonthatload_workspace_manifestparses without raising. - status: pending
- dependencies: [5]
{62.7} — Co-located verify driver (in-repo Python module, localhost /stage, stage-only)
Section titled “{62.7} — Co-located verify driver (in-repo Python module, localhost /stage, stage-only)”- title: Build co-located verify driver (
verify_driver.py, localhost/stage, stage-only) - description: Add
scripts/cocoindex_pipeline/verify_driver.py— a thin in-repo Python module, run on the B1 host, that POSTs fixture bytes multipart tolocalhost:<port>/stage, exiting 0 iff all 2xx. Stage-only; no Supabase/SQL/Vitest; no Job/WIF/secrets. - details: IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH (a)
this B1 re-spec is ratified by Liam, AND (b) ID-66 host stand-up is complete (B1 host +
Compose co-locating cocoindex+pullmd + persistent local-disk volume — ID-66 §1-a/b/c). The
driver runs ON the ID-66 host by construction. OQ-62-TECH-11 RESOLVED (TECH §C′): a thin
in-repo Python module
scripts/cocoindex_pipeline/verify_driver.py, invoked aspython3 -m scripts.cocoindex_pipeline.verify_driver --fixtures <set>by the operator or a B1 push-to-deploy/operator hook (ID-66 §1-d / OQ-66-6). NO dedicated container image, NOcnb-launcher, NO Cloud Run Job manifest, NO WIFgcloud run jobs execute, NO Secret-Manager--set-secrets. Behaviour: (1) read the(fixture set)— a list of(fixturePath-in-repo-checkout, destPath, titlePrefix)tuples — from the host repo checkout (docs/testing/test-data/**; the host HAS the repo). For each, read the bytes and POST multipart (filepart +destPath+titlePrefixtext parts) tohttp://localhost:<port>/stage— the LOAD-BEARING DEFAULT per OQ-62-P4-2 (do NOT raw-write the corpus disk by default) [Inv-7, Inv-12]. Because it runs on the same host as the cocoindex container, the request reaches/stageover loopback / the host’s private container network — noingress: internalto defeat, no 403 [Inv-7]. (2) Stages only — does NOT poll Supabase, does NOT run the invariant SQL probes, does NOT embed a copy of the Vitest assertions; no Supabase query originates from the driver process [Inv-8 stages-only half, Inv-9]. (3) Exit 0 iff every/stagePOST returned 2xx; exit non-zero on any 4xx/5xx/timeout/connection-refused, surfacing the failing fixture + the/stageresponse status/body so the operator can diagnose without re-running [Inv-10]. Re-running re-stages (overwrite-on-name; the watcher re-processes on change); no clean-corpus precondition; cleanup (dropFixture) is the test layer’s responsibility, not the driver’s [Inv-11]. Secrets (the cocoindex localhost port; for{42.10}, the pullmd Bearer token + Supabase service-role creds the Vitest step needs) read from the B1 host secrets store (Coolify env / host secrets manager, ID-66 §1-f) — never echoed in logs; NO GCP Secret Manager [Inv-7 secrets half]. AGPL (TECH-CONSTRAINT-AGPL / PLAN §3.4): the driver does not bake pullmd into the cocoindex image; FLAG that the B1 push-to-deploy path (ID-66 §1-d) MUST wire an image-inspect equivalent ofcloudbuild-cocoindex.yaml:117–167(zero pullmd/Playwright in the cocoindex image) if it does not re-run the cloudbuild assertion, so the AGPL boundary is not silently lost when the deploy mechanism changes — this wiring is an ID-66 deploy-path concern; ID-62 asserts + flags it;agpl-boundary.integration.test.ts(Tier 4) is the test-time backstop [Inv-8 AGPL half]. Direct disk-drop (--disk-drop) is a non-default explicit-flag fallback the driver MAY omit (OQ-62-P4-2) [Inv-12]. Uses stdlib + an already-pinned in-repo HTTP client (aiohttp/urllib/requests) — no new external symbol (PLAN §5). Local check:python3 -m pytest scripts/tests/— the stage-loop is unit-testable with a mocked HTTP client (assert exit-code semantics; assert NO Supabase/SQL symbol is imported). Code-intel:verify_driver.pyis a NEW Python module (no upstream callers yet; ast-dataflow/ts-morph N/A — Python; use direct read + grep); rungitnexus_detect_changes()before commit to confirm only the expected new symbols land. - testStrategy:
python3 -m pytest scripts/tests/green for the verify-driver stage-loop: exit 0 iff all/stage2xx; non-zero + failing-fixture log (status/body) on any failure; no Supabase/SQL import in the driver source; default codepath issues an HTTP POST to localhost/stage(not a raw fs write). - status: pending
- dependencies: [5]
{62.8} — stageFixture JSON→multipart wire-contract change
Section titled “{62.8} — stageFixture JSON→multipart wire-contract change”- title: Switch
stageFixturefrom JSON body to multipart bytes (StageFixtureArgs unchanged) - description: In
fixture-staging.ts, changestageFixtureto read fixture bytes runner-side and POST a multipartfilepart (+destPath/titlePrefix), dropping the JSON body.StageFixtureArgs, the throws, and the response read stay unchanged. - details: IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH (a)
this B1 re-spec is ratified by Liam, AND (b) ID-66 host stand-up is complete (ID-66
§1-a/b/c). NOTE: this Subtask is host-agnostic — it passes
bun run testwithout a live host, so the Orchestrator MAY dispatch it once (a) holds, ahead of (b). OQ-62-7 RESOLVED in TECH: multipart/form-data (NOT base64-in-JSON) — binary.xlsx/.docx/.pdffixtures. Editfixture-staging.tsstageFixture(100–139): KEEPStageFixtureArgs(51–71) exactly as-is — callers compile unchanged (OQ-62-7) [Inv-26]. Replace the JSON body construction (115–119): read bytes atargs.fixturePathrunner-side (fs.readFile/Bun.file); build aFormDatawith afilepart (Blob/Fileof the bytes, namedbasename(args.destPath)), adestPathtext part, and atitlePrefixtext part;fetch(endpoint, { method: 'POST', body: formData })— DROP the explicitContent-Type: application/jsonheader at 115 (fetch sets the multipart boundary). The behaviour invariant: bytes on the wire, not a path the writer can’t see [Inv-2, Inv-19]. KEEP verbatim: the unset-COCOINDEX_FIXTURE_STAGING_URLthrow (105–108), the non-2xx throw (125–128), the{ destPath?, requestId? }response read (131–136). Do NOT touchpollContentItemsFor(179–222),dropFixture(376+), or the form helpers (546+) — they run wherever the Vitest tier runs (on the B1 host under B1) against live Supabase (reachable from anywhere) [Inv-22]. OQ-62-6 carry (Inv-25): no/stagein-byte injection — the content tier matchesilike('title', '${titlePrefix}%')(199) on a filename-derived title; the form tier pollsname/storage_pathviamatchStoragePath(559). CARRY (Inv-20):dropFixture’sentity_mentionscleanup stays best-effort (411–425, ID-49.5 deferred per S273 OQ-1) — confirm theentity_mentionstable/FK shape when the live tier first produces rows so cleanup does not leak across runs (a note for the first live run, not a code change here).FormData/Blob/fetchare Bun built-ins (out of OQ-3 scope). Local check:bun run test(NOTbun test) — the typed contract must not break (StageFixtureArgsunchanged); add a helper-level unit assertingstageFixturesendsFormData(noapplication/jsonheader; afilepart present) against a mockedfetch. Code-intel: TS test helper, NOT in the GitNexus index (__tests__/integration/outside the corpus — TECH §Context); consumers are*.integration.test.ts(test corpus), so nogitnexus_impact/ast-dataflow callersrun is required — the change is internal to the helper’s request construction; ground by direct read. - testStrategy:
bun run testgreen:StageFixtureArgsunchanged (callers compile);stageFixturesendsFormDatawith afilepart and NOapplication/jsonheader (asserted against a mockedfetch); env-gate + non-2xx throws +{destPath, requestId}response read preserved. - status: pending
- dependencies: [5]
{62.9} — B1 trigger + host Vitest go-live + Inv-29 launch-flip doc item
Section titled “{62.9} — B1 trigger + host Vitest go-live + Inv-29 launch-flip doc item”- title: Add B1 trigger (operator/host hook) + host Vitest go-live + Inv-29 launch-flip doc
- description: Add the B1 trigger artefact (operator/host hook replacing
gcloud run jobs execute, with a commented launch-flip marker), run the cocoindex Vitest tier on the host with theCOCOINDEX_*env block, and add the Inv-29 item to the sequencing doc. - details: IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH (a)
this B1 re-spec is ratified by Liam, AND (b) ID-66 host stand-up is complete (B1 host +
Compose + persistent volume — ID-66 §1-a/b/c). The trigger + Vitest go-live run ON the ID-66
host; this Subtask’s live verification waits on (b). The B1 trigger REPLACES the OLD
cocoindex-live-verify.yml’s WIF +gcloud run jobs execute— NOgcloud run jobs execute, NO WIF, NO Cloud Run Job in the trigger path [Inv-28]. KEEP the policy (topology-agnostic): on-demand, NOT inlined into the PR-blockingci.ymlintegrationjob, NOT scheduled today; the existing PR-blocking integration job is unchanged [Inv-28]. Changes: (1) the B1 trigger artefact — a host runbook step (or a Coolify scheduled-task stub / thin workflow that SSHes the host, the slice chooses) that runs the verify driver (python3 -m scripts.cocoindex_pipeline.verify_driver) then the Vitest go-live step on the host; a non-zero driver exit fails BEFORE the Vitest step. Include a commented-out scheduled-cadence block with a# FLIP ON AT LAUNCH (ID-62 Inv-29)marker so the launch flip is a one-line uncomment [Inv-29]. (2) the Vitest go-live step — run the cocoindex Vitest tier (bun run test:integrationor a cocoindex-scoped invocation, NOTbun test) ON the B1 host with the env block:COCOINDEX_STAGING_URL(=http://localhost:<port>),COCOINDEX_FIXTURE_STAGING_URL(=http://localhost:<port>),COCOINDEX_SOURCE_PATH(=/corpus), plus live-Supabase creds from the host secrets store — so the previously-skipped Tier-1/Tier-1b/Tier-2/Tier-4/Tier-5 tests EXECUTE and assert against live Supabase [Inv-18, Inv-20]. TIER-3 REFRAME (LOAD-BEARING, PLAN §3.3 / OQ-62-TECH-12): under B1 the Vitest step runs ON the host soCOCOINDEX_STAGING_URL= localhost — the 8 Tier-3 files (health-probe,sidecar-cold-start,sidecar-mime-coverage,sidecar-version-metadata,stage-topology,latency-budget,transient-retry,audit-log-shipping) reach localhost and CAN go live (no skip-on-GH-runner gymnastics). Decide per file: EITHER go live on the host now, OR explicitly defer with a tracked note. Do NOT set the gate vars in a way that turns these green-by-skipping while claiming the topology tier is verified (TECH §Testing “File → invariant tier map” is the per-file ledger) [Inv-18 Tier-3 half]. Tier-1b (extract-memoisation,stage-5-failure-non-destructive,memo-hit-pipeline-run): confirm whether each needs a sibling Tier-1 stage to have run first (sequencing within the driver invocation). (3) Inv-29 action item: ADD “flip the live-tier verification to a scheduled/automatic cadence at launch” to the T13 “Pre-launch operational pre-decisions + observability” row ofdocs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md(line 241) — Inv-29 is satisfied by the doc’s presence + the commentedschedule:marker, NOT by wiring a live schedule now [Inv-29]. The staging happens via the driver ({62.7}); the host Vitest asserts produced rows only. Local check: any workflow/runbook YAML lints;bun run teststill green (off-host the tier skips clean as designed — no localhost/stage/Supabase). Code-intel: no indexed-symbol edit (artefacts are YAML/runbook + a doc append + an env block); ground the test-tier file list bygrepover__tests__/integration/cocoindex/*.integration.test.ts. - testStrategy: B1 trigger on-demand (operator/host hook, NO
gcloud run jobs execute/WIF) with a commented launch-flip marker;ci.ymlintegrationjob unchanged; a host run runs the driver then the Vitest tier (Tier-1/1b/2/4/5 live; all 8 Tier-3 files decided, none silently-green); Inv-29 doc item. - status: pending
- dependencies: [6, 7, 8]
{62.10} — Parameterised {42.10} HTML invocation (Inv-9 round-trip over localhost)
Section titled “{62.10} — Parameterised {42.10} HTML invocation (Inv-9 round-trip over localhost)”- title: Wire the
{42.10}HTML/Inv-7-8-9 invocation through the same verify driver - description: Parameterise the verify driver with the
{42.10}HTML fixture + Inv-7/8/9 set: it stages one HTML source via/stage; Inv-7/8 assert via host Vitest against live Supabase; the Inv-9/s/<id>round-trip runs from the driver over localhost. - details: IMPLEMENTATION GATED — ID-62 implementation cannot begin until BOTH (a)
this B1 re-spec is ratified by Liam, AND (b) ID-66 host stand-up is complete (B1 host +
Compose co-locating cocoindex+pullmd over localhost + persistent volume — ID-66 §1-a/b/c);
the Inv-9 round-trip needs pullmd as a localhost sibling (ID-66 §1-b). OQ-62-TECH-12 RESOLVED
(TECH §{42.10}): the co-located driver runs the Inv-9 round-trip over localhost; the old
in-VPC-Job split-home DISSOLVES (pullmd is a sibling container). The SAME verify driver +
/stageroute ({62.7}), invoked with the(HTML fixture, Inv-7/8/9 assertion set)parameterisation, produces the{42.10}proof — ONE primitive parameterised by(fixture set, assertion set); no HTML-specific staging branch [Inv-21]. Changes: (1) extendscripts/cocoindex_pipeline/verify_driver.py(or a sibling invocation mode) to accept the HTML fixture set and, for that mode only, after staging the HTML and letting the worker ingest, read the producedpullmd_share_idfrom thesource_documentsrow (via the host-secrets-store Supabase creds) andGET http://localhost:3000/s/<share_id>(pullmd on localhost, ID-66 §1-b), asserting a 200 non-empty response andpullmd_share_id = X-Share-Id; surface pass/fail through the driver exit code [Inv-23]. The round-trip is one HTTP call WITHIN the driver invocation, not a separate external curl; not attempted off-host [Inv-23]. (2) the host Vitest assertions (the existing{42.10}test home, or a NEW__tests__/integration/cocoindex/inv-7-8-9-pullmd-e2e.integration.test.ts): pollcontent_items.content_textnon-empty markdown (Inv-7) +source_documents.extraction_method = pullmd_*matching the liveX-Sourceheader (Inv-8) via the live service-role client (Supabase reachable from anywhere) — the HTML fixture is staged by the driver, not bybeforeAll[Inv-22]. These are live-Supabase assertions, reachable from anywhere (Inv-18 half) [Inv-27]. The single assertion surface is the Vitest tier; the driver does NOT re-implement the row assertions (Inv-9 no-duplication carried from{62.7}). ID-42{42.10}is unblocked at the TASK level (ID-62 in ID-42’sTask.dependencies[]), NOT via a Subtask cross-dep — this Subtask depends ONLY on its sibling{62.9}. Local check:python3 -m pytest scripts/tests/for the extended driver’s HTML-mode probe (mocked DB read- mocked pullmd
GET, exit-code semantics);bun run testfor the new/extended Vitest file’s typed shape (skips clean off-host). Code-intel:verify_driver.pyextension (Python; ast-dataflow/ts-morph N/A — use direct read + grep); rungitnexus_detect_changes()before commit.
- mocked pullmd
- testStrategy: A host run with the
{42.10}HTML parameterisation: the driver stages the HTML + asserts the Inv-9/s/<id> -> 200round-trip via exit code; host Vitest asserts Inv-7 (content_textnon-empty markdown) + Inv-8 (extraction_method = pullmd_*) against live Supabase;{42.10}proof green. - status: pending
- dependencies: [9]
7. Sibling-only dependency verification (the DAG)
Section titled “7. Sibling-only dependency verification (the DAG)”All {62.5}–{62.10} dependencies reference Subtasks of Task 62 ONLY (5, 6, 7, 8, 9). No
edge points outside Task 62. The dependency DAG:
{62.5} ──┬──> {62.6} ──┐ ├──> {62.7} ──┼──> {62.9} ──> {62.10} └──> {62.8} ──┘{62.5}(Slice A/stageroute) — no deps; host-agnostic, buildable ahead of the host.{62.6},{62.7},{62.8}— each depends on{62.5}(corpus activation, the driver, and the helper all consume the/stageroute’s multipart contract).{62.9}— depends on{62.6}(activated corpus),{62.7}(the driver),{62.8}(the byte-posting helper).{62.10}— depends on{62.9}(the activated corpus + driver + go-live step the{42.10}invocation reuses).
Sibling-only forcing function (§3.3) — HONOURED. No Subtask of Task 62 depends on a
Subtask of any other Task; no escalation (Task split/merge) is required — the boundary is
correct. The two cross-Task edges are at the Task level (Task.dependencies[]), recorded
by the Orchestrator: ID-62 → ID-66 (the B1 host precondition); ID-42 → ID-62
({42.10} unblocked by ID-62’s primitive). Neither is a Subtask cross-dep.
6 of 25 Subtasks — well within the §3.4 soft ceiling. No Task-boundary problem signalled.
8. Summary for the Orchestrator
Section titled “8. Summary for the Orchestrator”- 6 Subtasks
{62.5}–{62.10}, sibling-only deps, DAG in §7. The OLD entry gate ({62.0}, OQ-62-9) is DELETED (moot under B1); it is REPLACED by the B1 IMPLEMENTATION-GATED dual precondition (§4: re-spec ratified AND ID-66 host complete), carried as a blocking note in EVERY{62.5+}details. - PLAN-level OQs carried from TECH by reference, NOT re-opened (§2): OQ-62-11 (in-repo
Python verify-driver module, no Job/image/
cnb-launcher); OQ-62-12 (co-located driver / host Vitest for the Inv-9 round-trip + Tier-3, over localhost); OQ-62-P4-2 (/stageHTTP default, disk-drop a non-default fallback); OQ-62-P4-3 (/healthkept); OQ-62-6 (filename-derived title); OQ-62-7 (multipart/form-data). - Load-bearing carries baked into
details(§3): (1) the schema-valid/corpus/.kh-workspace-map.jsonMUST be seeded at process start inserver.main()before the watch arms (elseapp_mainraises → zero rows) —{62.6}; (2) the seed/mkdirlives in Python, not a shell entrypoint —{62.6}; (3) the 8 Tier-3 files get a per-file live-vs-defer decision, never silently-green —{62.9}; (4) the AGPL image-isolation constraint (pullmd a sibling container, NOT baked into the cocoindex image; the cloudbuild / B1-equivalent image-inspect check) lands in{62.5}(image stays clean) +{62.7}(flag the B1-deploy-path equivalent for ID-66). - Two Task-level edges for the Orchestrator to record in
task-list.json:ID-62→ID-66(B1 host precondition);ID-42→ID-62({42.10}unblock). Neither is a Subtask cross-dep. - Dispatch hint:
{62.5}and{62.8}are host-agnostic — the Orchestrator MAY dispatch them once the re-spec is ratified (precondition (a)), ahead of the ID-66 host (precondition (b)).{62.6}/{62.7}/{62.9}/{62.10}need the live host for their on-host verification. - No new external-library symbols beyond TECH’s verified set (aiohttp multipart PRESENT,
walk_dirPRESENT against the pins; TSFormData/Blob/fetchare Bun built-ins, out of OQ-3 scope). NoABSENT/SIGNATURE_DRIFT/BEHAVIOUR_DRIFT. - Ledger title carry (OQ-62-LEDGER-TITLE):
Task.titleintask-list.jsonstill reads the superseded “in-VPC harness” phrasing — the Orchestrator updates it to the B1 phrasing (seeOQ-pending.md). The Planner does not edit the ledger.