RESEARCH — ID-443 Move the cocoindex pipeline out of scripts/
RESEARCH — ID-443 Move the cocoindex pipeline out of scripts/
Section titled “RESEARCH — ID-443 Move the cocoindex pipeline out of scripts/”Artefact: {443.1}. Session: S550. Status: findings only — no file was moved,
no code edited, no task file touched.
Every number below is reproducible from scratch/inv.sh (the census script, reproduced in
§1.1). Where a number differs from the task’s ## Notes, the delta is stated explicitly.
0. Executive summary
Section titled “0. Executive summary”| Question | Verdict |
|---|---|
Does a top-level services/ tier serve a nameable requirement? | No — and no source exists. The tier fails the requirement-first test (§2.1). More decisively, it is insufficient: it moves one directory when the deployed unit is six (§2.2). |
| Then where? | A singular, root-level, Python-identifier-named directory holding the whole runtime bundle plus its tests. Naming is the residual owner call; pipeline/ is the recommendation (§2.5). |
Do the 77 scripts/tests/ importers move? | Yes — 84 of the 93 files, not 77. The governing citation is DR-117, not the test philosophy (§3). |
Does deploy/ move with it? | No — it stays at repo root, as a sibling. It has a second, explicitly carved-out tenant (§4). |
Is 550 / 114 the real size? | No — it is a floor. Measured total across all seven surfaces: ~1,000 in-repo edit sites across ~200 files, plus a docs-site partition of 42 repoint / 281 leave (§1). |
| Is the area sound to build on? | One resolve-first blocker (the dual-spelling sys.path hazard, §5.1) and three fold-ins. |
1. What exists today — the surface inventory
Section titled “1. What exists today — the surface inventory”1.1 The census is a command sequence, not a hand list
Section titled “1.1 The census is a command sequence, not a hand list”The full script is scripts/inv.sh shape (written to scratch, not committed — {443.2}
should check it in beside the migration, because it is also the post-move verification
gate: every S-row must go to zero for the old spelling and to the same number for the new).
Its structure:
EX='--exclude-dir=node_modules --exclude-dir=.next --exclude-dir=.git --exclude-dir=.gitnexus --exclude-dir=.memdb --exclude-dir=.memtrace --exclude-dir=__pycache__'# S1 import statements grep -rEn --include=*.py $EX '^\s*(from|import)\s+scripts\.cocoindex_pipeline' .# S2 alternate import forms import_module( / __import__( / 'from scripts import' / sys.modules[# S3 dotted-string paths "…scripts.cocoindex_pipeline…" in .py and in yml/sh/json/ts# S4 path strings 'scripts/cocoindex_pipeline' repo-wide# S5 segment-wise paths '"cocoindex_pipeline"' as a Path() segment; parents[N] traversals# S6 package coupling in/out edges across the scripts/ boundary# S7 CI workflows per-workflow hit counts# S8 deploy config tracked-file split, cocoindex vs not# S9 build/test config pyproject, the seven ignore-files# S10 docs-site partitioned by top-level content dirResults, run against main at 63600dcc8:
| # | Surface | Measured | vs task ## Notes |
|---|---|---|---|
| S1a | from|import scripts.cocoindex_pipeline statements | 551 | 550 — +1, drift since S549 |
| S1b | files carrying them | 114 | 114 ✓ |
| S1c | same pattern anywhere on the line (incl. inside strings) | 563 | new — the 12-statement delta is the prelude hazard, §1.3 |
| S2a | importlib.import_module("scripts.cocoindex_pipeline…") | 6 | new — the S549 timeout gap, now closed |
| S2b | import_module(f"{mod.__package__}.…") — package-relative, invisible to any literal grep | 1 | new |
| S2c | __import__( | 0 | new (clean) |
| S2d | from scripts import cocoindex_pipeline | 0 | new (clean) |
| S2e | sys.modules[…cocoindex…] keying | 2 | new — the namespace guard, §5.2 |
| S3a/b | dotted-string module paths in .py | 174 across 32 files | 156 / 25 — S549 undercounted |
| S3c | dotted paths in .yml/.sh/.json/.ts | 9 | 8 / 4 files |
| S4a/b | scripts/cocoindex_pipeline path strings (in-repo) | 210 across 100 files | — |
| S5a | "cocoindex_pipeline" used as a Path() segment | 15 across 7 files | new — a third invisible hazard, §1.4 |
| S6a | importers inside the package | 34 | 34 ✓ |
| S6b | importers under scripts/tests/ | 77 | 77 ✓ |
| S6c | importers at scripts/ root | 3 | — |
| S6d | relative imports inside the package | 0 | 0 ✓ (it cannot relocate itself) |
| S6e | outbound edges: package → scripts/ root modules | 3 | new — §2.2, the finding that changes the destination |
| S7 | CI workflow hits | cocoindex-nightly.yml 14, onprem-deploy.yml 6 | ✓, plus 4 more workflows on scripts/tests/, §1.5 |
| S8 | deploy/ tracked files: 30 total, 12 name cocoindex | — | new split, §4 |
| S9 | pyproject.toml testpaths+pythonpath; 7 ignore/config files | — | ✓ + 5 new files |
| S10 | docs-site occurrences | 1118 across 323 files | 1394 / 363 — S549 counted a wider glob |
Cross-check against gitnexus. cypher over IMPORTS edges into scripts/cocoindex_pipeline
returns {importer_files: 116, edges: 263} — reproducing the S549 measurement exactly, so
the one tool surface that works is stable. It over-reports files by 2 against the hand count
and its 263 edges are less than half the 551 statements, because it collapses repeated
imports per file-pair. Treat it as a file list, never as an edit list.
1.2 What I executed, and what it cannot see
Section titled “1.2 What I executed, and what it cannot see”Executed: the ten grep/git ls-files families above; mcp__gitnexus__cypher (the IMPORTS
projection); mcp__memtrace__get_cochange_context on flow.py at a 365-day window; four
MemPalace FTS reads against the lock-free mode=ro&immutable=1 snapshot.
Not covered — say so rather than imply a clean sweep:
- Runtime-assembled module paths. S2b found one already (
import_module(f"{flow.__package__}.flow_context")). A projection over literals cannot find a path assembled from a variable at call time. The S547getattr(...)shape is the precedent; I closed the three forms S549 timed out on, but the general form remains open. conftest.pymonkeypatching andmake_cocoindex_stubs.scripts/tests/conftest.pyinstalls stub modules intosys.modules. I did not enumerate the keys it writes.- The
.claude/worktrees/agent-*checkout was excluded from every count. It carries a stale duplicate ofdeploy/onprem/backup/Dockerfile.tools; if it is live at migration time it needs its own pass. - Anything the guards assert but do not name.
__tests__/guards/corpus-manifest.test.tsandverify_driver.py’sFIXTURE_SETSagree with each other by construction; I checked the manifest’s 47scripts/paths but did not execute the guard. - No test was run. No
pytest, nobun run test, no build. Every claim here is static.
1.3 Hazard 1 — imports inside subprocess preludes (confirmed and widened)
Section titled “1.3 Hazard 1 — imports inside subprocess preludes (confirmed and widened)”The task names two files. Measured: 19 files under scripts/tests/ both import the
package and write source to a subprocess, and 12 files call subprocess.run/Popen.
The confirmed prelude sites carry real imports inside triple-quoted strings that are
executed in a child interpreter alongside sys.path.insert(0, repo_root):
test_url_source_engine_consumption.py:53(_ENGINE_PROBE_SRC),:95(_PROBE_PRELUDE), composed into four probe bodies at:159,:173,:201,:228test_cocoindex_server.py:100(_ENGINE_PROBE_SRC)test_cocoindex_memo_self_heal.py:559(_FINGERPRINT_SCOPE_PROBE_SRC)test_file_branch_memo_fingerprint.py:147-149
These depend on the scripts. layout at runtime in a child process, and they fail only
when that test runs. The S1c−S1a delta (563 − 551 = 12) is the machine-visible part of this
surface; the rest is inside strings that do not start with from/import at line start.
1.4 Hazard 3 — segment-wise Path() construction (new; no grep for the old path finds it)
Section titled “1.4 Hazard 3 — segment-wise Path() construction (new; no grep for the old path finds it)”15 sites across 7 files build the package path from segments rather than a string. The
id-412 consumer census already missed two files this way, and the code says so in a comment
at scripts/tests/test_form_extractors.py:48-56: “a grep for the old path string cannot
see either. If this tree moves again, grep for the SEGMENTS.”
| Site | Shape |
|---|---|
scripts/tests/test_cocoindex_flow_embedding.py:113 | Path(__file__).resolve().parents[1] / "cocoindex_pipeline" / "flow.py" |
scripts/tests/test_qa_promotion_boundary.py:136 | .parent.parent / "cocoindex_pipeline" / "flow.py" |
scripts/tests/test_cocoindex_ingest_once.py:70 | .parent.parent / "cocoindex_pipeline" / "flow.py" |
scripts/tests/test_producer_enrich.py:133, :1225 | .parents[1] / "cocoindex_pipeline" |
scripts/tests/test_producer_agent_loop.py:578, 691, 789, 1045 | _REPO_ROOT / "scripts" / "cocoindex_pipeline" / "extraction.py" |
scripts/tests/test_form_extractors.py:51 | .parent.parent / "cocoindex_pipeline" / "fixtures" / "form-templates" |
This is load-bearing for the destination. The parents[1] / "cocoindex_pipeline" form
resolves relative to the test file, so it survives a move unchanged if and only if the
tests and the package keep their current sibling relationship (X/tests/ beside
X/cocoindex_pipeline/). The _REPO_ROOT / "scripts" / … form breaks either way. A further
33 sites in scripts/ use parents[N] to reach the repo root; every one of those is
depth-sensitive and breaks if the nesting depth changes.
1.5 CI — six workflows, not two
Section titled “1.5 CI — six workflows, not two”| Workflow | Binding | Failure mode on a naive move |
|---|---|---|
cocoindex-nightly.yml | 14 cocoindex_pipeline hits; runtime module paths at :529, :688, :893 | Nightly fails loudly |
onprem-deploy.yml | 6 hits; build context cp -R scripts "${CTX}/scripts" at :274; GOOGLE_ENTRYPOINT=python3 -m scripts.cocoindex_pipeline.server at :304; detect-changes filter ^(scripts/.*\.py|requirements\.txt|deploy/coolify/.*)$ at :229; Inv-9 guard greps scripts/cocoindex_pipeline at :509 | Deploy stops building the pipeline at all — detect-changes silently matches nothing |
ci.yml | paths-filter python: = scripts/**/*.py, requirements.txt, scripts/tests/** (:65-68); python3 -m pytest scripts/tests/ (:215) | Silent green. The filter stops matching, the pytest step is skipped, CI passes with the Python suite never run |
pytest-nightly.yml | python3 -m pytest scripts/tests/ (:109) — the unconditional lane that exists precisely to cover ci.yml’s filter blind spot | Fails loudly (good) — but only nightly |
identity-guard.yml | 8 :(exclude,literal) pathspecs naming scripts/tests/*.py files (:304-311) | Exclusions silently stop applying; guard starts flagging moved files |
eval-nightly.yml / others | no cocoindex_pipeline hits | — |
ci.yml is the dangerous one: a paths-filter that stops matching produces a green PR with
no Python test run at all. pytest-nightly.yml’s own header comment (:3-10) documents
this exact blind spot as the reason it exists.
Also stale today, found in passing: 4 of identity-guard.yml’s 8 excluded literals point
at files that no longer exist (scripts/kb_pipeline/eval_holder_rule.py,
scripts/split_example-client_site_content.py, scripts/tests/test_classify_store_entities.py,
scripts/tests/test_workspace_resolver.py). A literal pathspec for a missing file is inert,
so this has been decaying unnoticed. Fold the cleanup in.
1.6 Build and tool configuration — 9 files
Section titled “1.6 Build and tool configuration — 9 files”| File:line | Binding | Breaks |
|---|---|---|
pyproject.toml:2 | testpaths = ["scripts/tests"] | Yes — pytest collects nothing |
pyproject.toml:6 | pythonpath = [".", "scripts", "scripts/tests"] | Yes — and see §5.1 |
.prettierignore:79-81 | scripts/cocoindex_pipeline/fixtures/, scripts/tests/fixtures/ | Formatter starts rewriting fixture bytes |
.gitnexusignore:15, 54, 55 | __tests__/integration/cocoindex/_helpers/, pyproject.toml, scripts/tests/fixtures/ | Self-inflicted blind spots — the two cheap wins |
.memtraceignore:15, 55 | same two | Same |
.understandignore:16 | scripts/tests/ | Stale silently |
.worktreeinclude:50 | scripts/tests/fixtures/entity_aliases_snapshot.json | Worktree provisioning drops the file |
stryker.config.mjs:52 | '/scripts/tests' | Mutation runs widen silently |
docs/reference/testing/corpus-manifest.json | 47 "path": "scripts/…" entries, guard-enforced by __tests__/guards/corpus-manifest.test.ts | Fails loudly (correct behaviour — this is the one guard that will catch a partial move) |
1.7 TypeScript — 22 files, of which 4 are runtime
Section titled “1.7 TypeScript — 22 files, of which 4 are runtime”Of the ~30 TS/TSX files naming the path, all but four reference it in a comment or
docstring describing the Python contract (lib/pipeline/error-classes.ts:27,
app/api/internal/pipeline-runs/record/route.ts:40/71/134/141/162/183,
lib/ontology/concept-schema.ts:18/54, lib/okf/parse-canonical-uri.ts:4,
lib/mcp/tools/entities.ts:56, lib/extraction/clean-via-worker.ts:11). Those are
accuracy repoints, not breakage. The four that are load-bearing:
__tests__/integration/cocoindex/_helpers/fixture-staging.ts— stages fixtures by path (and is.gitnexusignored, so no tool sees it)scripts/cocoindex-census-gate.ts— reads the corpus manifest and the driver dest pathsdeploy/onprem/monitor/datapath-watch.tsdeploy/onprem/sync-platform-corpus.sh(shell, same class) —:8is the anchor DR-117 cites as “RATIFY-1: the single source of truth”
1.8 Directory-scoped agent context moves with the directory
Section titled “1.8 Directory-scoped agent context moves with the directory”Two .claude/ trees are scoped by their parent directory and will stop auto-loading if the
code they describe moves out from under them:
scripts/.claude/skills/cocoindex/SKILL.md— named byreference/platform-context.md:177as one of the four current authorities for pipeline work, and byreference/cocoindex-pipeline.md(“loads automatically when editing underscripts/”).scripts/CLAUDE.md— directory-scoped context.
Both must move with the pipeline, and platform-context.md:177 + the project CLAUDE.md
repoint in the same commit (DR-104: the doc is the authority, so moving the code without the
doc makes the doc wrong).
2. The destination
Section titled “2. The destination”2.1 The carried UNDECIDABLE, answered
Section titled “2.1 The carried UNDECIDABLE, answered”“What requirement does a top-level
services/tier serve, and what is that requirement’s current source?”
I could not name either. What I searched, so the negative is auditable:
- The in-force decision register and
reference/decisions/— the 8 DR files naming the path are DR-055, DR-057, DR-058, DR-081a, DR-087, DR-117, DR-118, DR-121. None defines a tier, a membership rule, or a layer-vs-domain policy. DR-116 rejects layer-first rooms and says nothing about directories. reference/platform-context.md,reference/deployment-architecture.md,reference/cocoindex-pipeline.md— none namesservices/.- MemPalace FTS, four passes.
services/appears exactly twice in a structural sense, and both occurrences are carriers of the suggestion, not sources of a requirement:- DR-121, Alternatives considered: “Wait for the planned
scripts/→services/move. Rejected: the files travel inside the package either way.” — names it as a plan, and rejects waiting for it. - The S525 derivation drawer: “Move the two runtime assets now, into
scripts/cocoindex_pipeline/— don’t wait forservices/.”
- DR-121, Alternatives considered: “Wait for the planned
Per DR-123 rule 1, a destination that cannot name its requirement and that requirement’s
current source is not ratifiable. services/ is not ratifiable. That is a measured
negative over named sources, not a grep-absence argument: the sources exist and were read;
they are silent.
2.2 The finding that settles it — the deployed unit is six directories’ worth, not one
Section titled “2.2 The finding that settles it — the deployed unit is six directories’ worth, not one”DR-117’s premise is that “anything under scripts/cocoindex_pipeline/ moves as one unit.”
Measured, that premise is false at the directory boundary. The package has coupling in
both directions across the scripts/ root:
Outbound — the package imports three loose scripts/ root modules (S6e):
scripts/cocoindex_pipeline/form_extractors/docx.py:84 from scripts.analyse_template import (…)scripts/cocoindex_pipeline/form_extractors/docx.py:89 from scripts.docx_utils import open_document_safescripts/cocoindex_pipeline/form_extractors/docx.py:90 from scripts.extract_tender_questions import _classify_headerInbound — three scripts/ root modules import the package (S6c):
bid_worker.py:53-57, fill_template.py:448, _okf_prototype_draft.py.
And the deployment agrees. onprem-deploy.yml:274 builds from cp -R scripts — the
whole directory — and the platform composes run three processes out of that single
kh-cocoindex-pipeline image, all keyed on the scripts. module prefix:
| Service | Command | Compose |
|---|---|---|
| cocoindex sidecar | GOOGLE_ENTRYPOINT=python3 -m scripts.cocoindex_pipeline.server | all four |
| bid-worker | python3 -m scripts.bid_worker | docker-compose.platform.yaml:258, platform-staging.yaml:239 |
| mockllm | python3 -m scripts.cocoindex_pipeline.mock_llm | platform-staging.yaml:297 |
reference/deployment-architecture.md §2.1 states this explicitly and names the reason:
“It reuses the same kh-cocoindex-pipeline image as the cocoindex sidecar — scripts/**
is packaged whole by onprem-deploy.yml, so no second image build/CI pipeline is needed.”
Therefore services/cocoindex_pipeline/ is not merely unratifiable, it is insufficient.
It would leave bid_worker.py, fill_template.py, analyse_template.py, docx_utils.py,
extract_tender_questions.py and docx_cell_to_markdown.py behind in scripts/, splitting
the deployed image’s source across two roots and forcing the build context to become
cp -R scripts && cp -R services — strictly worse than today.
The unit that actually deploys is: the cocoindex_pipeline/ package + those six sibling
.py modules. The remaining six root .py files (analyze_claude_session_tokens.py,
audit_orphan_tests.py, compute-bertscore.py, extract-agent-usage.py,
extract_pdf_text.py, and the draft _okf_prototype_draft.py) plus all ~65 .ts files are
genuine dev scripts and stay.
2.3 The requirements the destination must satisfy, each with its current source
Section titled “2.3 The requirements the destination must satisfy, each with its current source”| # | Requirement | Current source |
|---|---|---|
| R1 | The destination directory name is the deployed Python package prefix, and is the image build-context root. It must be a valid Python identifier. | onprem-deploy.yml:274 + :304; four compose files’ command: lines |
| R2 | The destination holds the whole runtime bundle (package + 6 sibling modules), not one directory. | S6c/S6e measurement; deployment-architecture.md §2.1 |
| R3 | deploy/ and the pipeline are one concern. | Owner ruling, S521, recorded at mempalace.yaml:118-120: “deploy/ belongs here, not with CI (review change 3, and the owner’s own instinct which S521 wrongly argued him out of): deployment-architecture.md lists the compose files beside scripts/cocoindex_pipeline/server.py.” Carried in deployment-architecture.md’s kh_code_sources frontmatter. |
| R4 | The palace room must survive on directory name (DR-116). | mempalace.yaml:143 keys ingest-pipeline on the byte-identical compound cocoindex_pipeline; by M1 that keyword also reaches single-token dirs cocoindex/ and pipeline/; by M3 compound-vs-compound needs byte identity. |
| R5 | The scripts/tests/ room defect must resolve, which requires the new test path to contain a cocoindex-bearing directory token. | mempalace.yaml:236-240, verbatim: “It resolves for free when cocoindex leaves scripts/ (agreed S521, deferred to the id-46 pre-cocoindex retirement): the new path will say cocoindex, and P0 declaration order puts ingest-pipeline ahead of tests.” |
| R6 | Segment-relative test paths (§1.4) survive unchanged only if tests/ stays a sibling of cocoindex_pipeline/. | 15 measured parents[1] / "cocoindex_pipeline" sites |
On R3 and DR-123: the S521 ruling is an owner directive, so it is evidence of intent at
that time, never of correctness. I tested it and it holds on its own terms — the
deployment-architecture.md kh_code_sources block does list the four compose files beside
server.py, and the retro note records the self-correction (“deploy/ belongs with the
pipeline; the owner’s instinct was right and I argued him out of it”). But note precisely
what it settles: a concern grouping, discharged in mempalace.yaml as a room. It makes
no claim about directories, and §4 measures why the directory conclusion does not follow.
2.4 Options measured against the requirements
Section titled “2.4 Options measured against the requirements”| Option | R1 | R2 | R3 | R4 | R5 | R6 | Module path |
|---|---|---|---|---|---|---|---|
services/cocoindex_pipeline/ (the suggestion) | ✓ | ✗ | n/a | ✓ | ✓ if tests follow | ✓ | services.cocoindex_pipeline.server |
services/pipeline/{cocoindex_pipeline,tests}/ | ✓ | ✓ | n/a | ✓ | ✓ | ✓ | services.pipeline.cocoindex_pipeline.server — three tiers, stutters |
pipeline/{cocoindex_pipeline, *.py, tests}/ | ✓ | ✓ | ✓ | ✓ (both pipeline and cocoindex_pipeline match) | ✓ | ✓ | pipeline.cocoindex_pipeline.server |
ingest/{…} | ✓ | ✓ | ✓ | ✓ (via the inner cocoindex_pipeline/) | ✓ | ✓ | ingest.cocoindex_pipeline.server |
Flatten: pipeline/ is the package (pipeline/server.py) | ✓ | ✗ — no home for the 6 siblings | ✓ | ⚠ single-token pipeline matches, but the DR-116 M3 byte-identical compound match is lost | ✗ | ✗ | pipeline.server |
deploy/ nested inside the new root | — | — | ⚠ | — | — | — | see §4 |
services/ fails R2 outright. The flattened form fails R2, R5 and R6 and weakens R4. The
three-tier form satisfies everything but produces services.pipeline.cocoindex_pipeline.server
as the live entrypoint string in four compose files and two workflows.
2.5 Recommendation
Section titled “2.5 Recommendation”A singular root-level directory, not a plural tier. If cocoindex is the only member a
services/ tier would ever have, the plural is a promise the repo has no source for — and
per §2.2 the tier would not even hold the whole member. Recommended shape:
pipeline/ # build-context root; module prefix `pipeline.` cocoindex_pipeline/ # package name UNCHANGED — DR-116 R4, and it keeps … # the 551 import statements a prefix-only rewrite fixtures/ # (DR-117's home) — travels inside, no manifest churn beyond the prefix data/ # (DR-121's home) — same bid_worker.py # second entrypoint on the same image fill_template.py analyse_template.py docx_utils.py extract_tender_questions.py docx_cell_to_markdown.py tests/ # sibling of cocoindex_pipeline/ — R6 keeps 15 sites byte-identical .claude/skills/cocoindex/ # follows its scope CLAUDE.mddeploy/ # UNCHANGED, root sibling (§4)scripts/ # ~65 .ts + 6 dev-only .py remainThree properties this buys that the alternatives do not:
- The rewrite is a single prefix substitution.
scripts.cocoindex_pipeline→pipeline.cocoindex_pipelineandscripts/cocoindex_pipeline→pipeline/cocoindex_pipeline. Keeping the package name means the 47 corpus-manifest paths, the fixtures tree, thedata/tree and the palace keyword all change by prefix only. - R6 holds for free. The 15
parents[1] / "cocoindex_pipeline"sites need zero edits — the sibling relationship is preserved and the depth is identical (X/tests/…→parents[1]=X/). - The name is honest about what it is.
reference/cocoindex-pipeline.mdandplatform-context.md:239both call it “the Python ingestion pipeline”;mempalace.yamlrooms itingest-pipeline; three of the four composes are namedca-pipeline-*/kh-onprem*. The repo already calls this thing “the pipeline” everywhere.
The residual owner call is the name only. pipeline/ and ingest/ both satisfy every
requirement; pipeline/ matches the existing vocabulary and DR-116’s palace keyword directly,
ingest/ matches the room name. services/ should be declined on the record above.
What I did not resolve about the recommendation: whether _okf_prototype_draft.py (an
inbound importer, name-marked as a draft) is live or dead. It is an inbound edge on paper
and I did not determine whether it serves any current requirement — if it is dead it should
be deleted rather than carried, but that is its own call and I have not made it.
3. Do the scripts/tests/ importers move? — DR-117’s own question
Section titled “3. Do the scripts/tests/ importers move? — DR-117’s own question”Verdict: yes, and the mover count is 84 of 93 files, not 77.
3.1 The owner’s steer, tested against the actual doctrine
Section titled “3.1 The owner’s steer, tested against the actual doctrine”The steer was: “the cocoindex related tests should relocate with the pipeline if we follow our production mirroring test philosophy.” The direction is right and the citation is wrong, so it is worth correcting rather than confirming.
docs/reference/testing/test-philosophy.md §3 does ratify a mirror rule, verbatim:
One rule, no exceptions: a test’s path under
__tests__/equals its production path from the repo root.
But its scope is __tests__/ — the Vitest tree. §3.1 derives the rule from measuring
“the 18 top-level directories under __tests__/”; §3.2 enumerates every directory that
mirrors nothing; §3.5’s codemod is --scoped to __tests__/app/. scripts/tests/ is not
mentioned anywhere in the document, and testing-patterns.md opens with “Runner is
Vitest” and is TypeScript-only throughout. The Python tree is a flat directory of 93
test_*.py files with no mirror structure at all — so it is not a violation of the mirror
rule, it is outside it.
The doctrine that does reach it is DR-117, whose rule is stated generally and whose Context paragraph names this exact defect:
Fixture trees are filed by the domain that owns them, never by the kind of artefact they are. […] Anything filed under
scripts/tests/is therefore roomed by kind, not by domain — the opposite of the ratified intent.
And DR-117’s Consequences hand the question here explicitly:
A partial answer to the S521
scripts/tests/room finding. It does not settle where the 99 Python pipeline tests should live — that stays open and belongs with the deferred cocoindex relocation.
DR-121 already extended DR-117’s rule once — from fixture trees to runtime data
(taxonomy_snapshot.json), on the reasoning that runtime data “has a stronger claim still.”
Extending it to the tests is the third application of one rule, not a new ruling. Tests
of a domain module are as much “filed by kind” under scripts/tests/ as the fixtures were.
So the citation is DR-117 + DR-121’s precedent, and test-philosophy.md §3 supports it by
analogy but does not currently govern it. Consequence for {443.2}: if the tests move,
§3 should gain an explicit Python clause in the same commit, or the doctrine stays silent on
the largest test tree outside __tests__/.
3.2 Independent evidence, not doctrine
Section titled “3.2 Independent evidence, not doctrine”Doctrine aside, three measurements point the same way:
- Behavioural coupling.
memtrace get_cochange_contextonscripts/cocoindex_pipeline/flow.py, 365-day window, top 20 partners: 9 of the 20 arescripts/tests/test_cocoindex_*.pyfiles (test_cocoindex_flow_failure_mode.py×7,test_cocoindex_flow_pipeline_run_webhook.py×7,test_cocoindex_flow_write_path.py×7,test_cocoindex_server.py×7, and five more). The other 11 are all insidescripts/cocoindex_pipeline/. Every one of the 20 is insidescripts/. The tests change in the same commits as the code; they are part of the unit, measured on git history rather than on filing convention. - Mechanical. The 15 segment-wise sites (§1.4) need zero edits if the tests move as a sibling, and hand-editing if they do not.
- The palace config already assumes it.
mempalace.yaml:236-240(R5) states the room defect “resolves for free when cocoindex leaves scripts/ […] the new path will say cocoindex.” If the tests stay atscripts/tests/, that resolution never happens and the comment becomes a stale claim in a live config file.
3.3 The mover partition — 84, not 77
Section titled “3.3 The mover partition — 84, not 77”The 77 figure counts files that import the package. Five more are cocoindex tests that
read the source as text (AST/source-scanning guards, which by construction do not import
their subject), plus two testing the runtime siblings’ behaviour and two more on
docx_utils/docx_cell_to_markdown:
| Group | Count | Disposition |
|---|---|---|
Import scripts.cocoindex_pipeline | 77 | Move |
Source-scanning cocoindex guards (test_cocoindex_build_dsn.py, test_cocoindex_flow_entity_mentions_target.py, test_cocoindex_flow_terminal_status.py, test_cocoindex_stage_5_crossrun_guard.py) | 4 | Move — the import graph cannot see them |
Test the runtime siblings (test_analyse_template.py, test_fill_template.py, test_docx_utils_pandoc.py, test_docx_cell_to_markdown.py) | 4 | Move — their subjects move (§2.2) |
conftest.py (shared fixtures + the ID-67 namespace guard) | 1 | Move, and see §5.2 |
_fixture_manifest.py, test_canonical_namespace_guard.py | 2 | Move — infrastructure for the above |
Genuinely non-pipeline: test_audit_orphan_tests.py, test_census_pg_evidence.py, test_pipeline_schema_uses_visibility.py | 3 | Stay — but scripts/tests/ would then hold 3 files and pyproject.toml needs two testpaths |
test_bid_worker.py, test_docx_tracked_changes_regression.py (import siblings) | 2 | Move |
Total movers ≈ 84–90 of 93. {443.2} must classify the residue file by file rather than
by import graph — the import graph undercounts the movers by at least 5, which is the
same failure mode that made gitnexus rename report files_affected: 0.
The three stayers are a real question, not a rounding error: three files left behind
means pyproject.toml grows a second testpaths entry and CI grows a second pytest
invocation, or those three move somewhere else entirely. Cheapest defensible answer: move all
93 and re-home the three genuinely-non-pipeline ones as a separate, small follow-up — but
that is a {443.2} call, and it should be made deliberately rather than absorbed.
4. Does the deploy config move? — no, and the measurement says why
Section titled “4. Does the deploy config move? — no, and the measurement says why”Owner steer tested: “I imagine the actual deploy config [would go there too].”
deploy/ has 30 tracked files. 12 name cocoindex. The 18 that do not split cleanly
into two groups, and both are reasons to leave the directory where it is:
| Group | Files | Why it is not pipeline |
|---|---|---|
deploy/.claude/skills/coolify-compose/ + coolify-deploy/ | 15 | Generic Coolify authoring skills, not cocoindex-specific. Nesting them under a pipeline root re-scopes their auto-load to the pipeline only. |
deploy/coolify/goose-pilot/goose/ | 3 | goose is explicitly carved out of the pipeline concern. mempalace.yaml:94 declares agent-tooling before ingest-pipeline* specifically *"so deploy/coolify/goose-pilot/**reaches this room at P0 (id-71 {71.26}, DR-044) without needing a carve-out there."*deployment-architecture.md: *"goose is an MCP client on a timer — it never touches the pipeline or databases directly."* Its compose says so at :5`: “does NOT share the cocoindex image/compose.” |
So deploy/ is a two-tenant directory, and the second tenant is one the palace config
goes out of its way to route elsewhere.
Three further arguments against nesting it:
- The owner’s own criterion points the other way. “the destination should be whichever
makes the most sense for the pipeline and in turn the deployment setup, which is
currently at the route level.”
deploy/is at repo root.pipeline/+deploy/as root siblings preserves that; nesting demotes it. - R1 conflict. The new root becomes a Python package root (
cp -R pipeline, module prefixpipeline.).deploy/holds.yaml,.sh,Dockerfile.toolsand a.tsfile (datapath-watch.ts) — putting them inside a Python package root reintroduces the exact mixed-language junk-drawer property this task exists to fix, and would sweep them into the image build context. - R3 is already discharged. The S521 ruling made deploy and pipeline one concern, and
mempalace.yaml’singest-pipelineroom implements that:keywords: [cocoindex_pipeline, cocoindex-chunking, pipeline-runs, deploy-onprem, coolify-deploy].deploy-onpremandcoolify-deployreachdeploy/,onprem/andcoolify/as single-token dirs under DR-116 M1 — from anywhere in the tree. The concern is grouped whether or not the directories are nested. Nesting buys nothing the room does not already give.
Verdict: deploy/ stays at repo root. What does change inside it is the 12 cocoindex
files’ path strings and module paths — docker-compose.platform-staging.yaml:297,
docker-compose.platform.yaml:272, sync-platform-corpus.sh:8, datapath-watch.ts, and the
onprem/backup/*.sh LMDB scripts.
What this check did not cover: I did not read the four backup/restore shell scripts line
by line, only confirmed they name cocoindex. Their LMDB volume paths are container-side
(/cocoindex-state/lmdb), so they are probably unaffected by a repo-side move — probably, not
verified.
5. Issues found — resolve-first vs build-on
Section titled “5. Issues found — resolve-first vs build-on”5.1 RESOLVE FIRST — the dual-spelling sys.path hazard
Section titled “5.1 RESOLVE FIRST — the dual-spelling sys.path hazard”The same physical modules are imported under two different spellings, simultaneously, in live code.
# BARE spelling — resolves only because pythonpath = [".", "scripts", "scripts/tests"]scripts/analyse_template.py:27 from docx_utils import open_document_safescripts/analyse_template.py:28 from extract_tender_questions import _classify_headerscripts/extract_tender_questions.py:28 from docx_utils import open_document_safescripts/fill_template.py:19 from docx_utils import open_document_safescripts/bid_worker.py:82 from fill_template import (…)+ 2 more prod-side sites (7 total), and 48 sites across 5 test files (test_bid_worker.py alone has 43)
# QUALIFIED spelling — the same modules, via the package pathscripts/cocoindex_pipeline/form_extractors/docx.py:84 from scripts.analyse_template import (…)scripts/cocoindex_pipeline/form_extractors/docx.py:89 from scripts.docx_utils import open_document_safescripts/cocoindex_pipeline/form_extractors/docx.py:90 from scripts.extract_tender_questions import _classify_headerscripts/tests/test_docx_tracked_changes_regression.py:53 from scripts.docx_utils import (…)This is the ID-44.5 / ID-177 defect shape, unguarded. scripts/tests/conftest.py:245-300
exists precisely because a bare-vs-qualified import of the same physical files creates two
sys.modules identities, and for cocoindex_pipeline that produced double
coco.ContextKey registration and independent ContextVar storage — the ContextKey-leak root
cause. ID-67 canonicalised the pipeline onto scripts.cocoindex_pipeline.* and installed a
pytest_collection_finish guard that fails collection loudly if a bare alias ever
reappears. That guard covers cocoindex_pipeline only. docx_utils, analyse_template,
fill_template, extract_tender_questions and bid_worker have exactly the same shape and
no guard.
They do not leak today because those modules are stateless. The relocation is when it
detonates: the move changes pythonpath, and from docx_utils import … either stops
resolving (loud, best case) or resolves to a different copy than
from scripts.docx_utils import … (silent, worst case — two identities of the DOCX helpers
inside the extraction path).
Resolve first, in its own commit before the move, so the failure is attributable:
canonicalise every bare-spelling site onto the package-qualified form, delete
scripts/analyse_template.py:24-25’s and bid_worker.py:78-80’s sys.path.insert shims,
and widen conftest.py’s guard from cocoindex_pipeline to the whole runtime bundle. That
last step also gives {443.3} a live regression gate for the move itself. Owner-flagged:
this is 55 edit sites (7 prod + 48 test) of scope that the task does not currently carry.
5.2 BUILD ON, with care — the ID-67 namespace guard becomes the migration’s best test
Section titled “5.2 BUILD ON, with care — the ID-67 namespace guard becomes the migration’s best test”conftest.py’s _bare_cocoindex_alias_keys() flags any sys.modules key equal to or
starting with cocoindex_pipeline. After a move to pipeline/cocoindex_pipeline/, the
canonical key becomes pipeline.cocoindex_pipeline — which is still not a “bare alias”, so
the guard keeps working with a one-line message update. But note the new risk it now guards:
if pythonpath gains the new root’s parent, import cocoindex_pipeline starts resolving
bare for the first time, and the guard is the only thing that catches it. Do not weaken it
during the move. test_canonical_namespace_guard.py asserts on the literal string
"cocoindex_pipeline" at :46, :54, :59-60 — it moves with, and its assertions need
re-reading against the new prefix rather than a blind rewrite.
5.3 BUILD ON — DR-117’s “moves as one unit” needs an amendment, not a contradiction
Section titled “5.3 BUILD ON — DR-117’s “moves as one unit” needs an amendment, not a contradiction”§2.2 shows DR-117’s premise is false at the directory boundary. DR-117 is not wrong about
fixtures (its actual subject) and should not be reopened; but a decision that says “moves
as one unit” while the unit has six cross-boundary edges will mislead the next reader.
DR-intent, returned to the Orchestrator rather than written here: a decision recording
that (a) the deployed unit is the runtime bundle, not the package; (b) the destination is a
singular root-level directory, name TBD by the owner; (c) the Python tests file by owning
domain, extending DR-117 a third time; (d) deploy/ stays at root as a sibling. It should
amend DR-117 rather than supersede it.
5.4 FOLD IN — four cheap wins on the same files
Section titled “5.4 FOLD IN — four cheap wins on the same files”.gitnexusignore:54(pyproject.toml) and:15(__tests__/integration/cocoindex/_helpers/) — self-inflicted blind spots on two of the files that break. Same two lines in.memtraceignore.identity-guard.yml:301-311— 4 of 8:(exclude,literal)pathspecs point at deleted files..understandignore:16and.worktreeinclude:50— both namescripts/testspaths and are invisible to every tool.mempalace.yaml:236-240’s “resolves for free” comment becomes either true or stale on the day of the move; it must be updated in the same commit either way.
6. Best practice — constraints from the domain skills and the deployment doc
Section titled “6. Best practice — constraints from the domain skills and the deployment doc”scripts/.claude/skills/cocoindex/SKILL.mdis one of four named current authorities for pipeline work (platform-context.md:177) and is directory-scoped — it moves with the code or it stops loading (§1.8).- The pipeline is a cocoindex flow, not a script.
reference/cocoindex-pipeline.mddescribes a 6-stageflow.py:app_mainoncocoindex[postgres]==1.0.18, a@coco.lifespanasyncpg pool, an LMDB engine store on a mounted volume, and an HTTP surface with/healthand/walk. The owner’s standing frame (platform-context.md:179) is “the correct approach is using cocoindex correctly”. The filing should say “service”, not “script” — which is the whole argument for the move and is why a singular service-shaped name beats a speculative tier. cocoindexContextKeyregistration is process-global and single-shot. This is what makes §5.1 a correctness issue rather than a tidiness one, and it is whyflow_context.py:55andflow.py:1163both carry comments aboutsys.path.insertcausing a secondContextKey(...)call to trip the registry.- DR-116 M1/M3 are the palace’s actual matching semantics (
mempalace.yaml:29-33), read fromminer.pysource rather than modelled. A room projection must re-implementdetect_roomfrom source — so {443.3}‘s “MemPalace room routing still resolves” AC must be verified by importing the realdetect_room, not by reasoning about keywords. - No external research ran. This is a repo-structure task squarely inside understood territory; input (iv) was skipped deliberately.
7. Prior decisions and context
Section titled “7. Prior decisions and context”| Source | What it binds here |
|---|---|
| DR-117 (accepted, S522) | Files by owning domain, never by kind. Ratifies the move in principle, defers it, and routes the scripts/tests/ question to this task. Its “moves as one unit” premise is measured false at the directory boundary (§2.2). |
| DR-118 (accepted, S523) | The corpus manifest is a cross-tree register with no owning domain, so it stays at docs/reference/testing/corpus-manifest.json and does not move. Its 47 scripts/… path values repoint. |
| DR-121 (superseded by DR-130, but its rule stands, S525) | Extended DR-117 from fixture trees to runtime data. Its Alternatives explicitly reject waiting for services/. Precedent for a third extension to tests. |
| DR-116 (accepted, S521) | Rooms key on directory name, never path position. Supplies R4/R5 and the M1/M3 matching semantics. Its Consequences already promise “Rooms survive […] the cocoindex relocation and the deploy/ move with no re-mine.” |
DR-119 / id-416 (todo) | Replaces buildpacks with a checked-in Dockerfile whose COPY set is explicit, and generates the lock in CI. Direct sequencing conflict — §8. |
| DR-104 / DR-106 | Ratified docs outrank code; intended-architecture/ (except 01-vision.md), phase-0-investigation/ and specs below ~id-130 are presumed stale. Supplies the docs partition rule (§9). |
| DR-044 / id-71 {71.26} | goose is agent tooling, not pipeline — the carve-out that keeps deploy/ two-tenant (§4). |
reference/deployment-architecture.md §2.1, §2.2 | The three-process single-image topology; the CNB launcher entrypoint fix; the LibreOffice post-pack layer. The load-bearing source for R1/R2. |
| MemPalace recall | See §7.1. |
7.1 What recall turned up
Section titled “7.1 What recall turned up”The palace’s vector index is disabled (mempalace_search returned
vector_disabled_reason: "HNSW index holds 379,294 elements but sqlite has 381,762 embeddings — 2,468 drawers (1%) missing […] Vector reads are disabled until mempalace repair rebuilds it"), so mempalace_search degrades to BM25 and returned noise
for this query. All useful recall came from the lock-free FTS fallthrough
(mode=ro&immutable=1), four passes. Worth flagging to the owner independently: the palace
needs mempalace repair.
Beyond the known junk-drawer drawer, three finds:
- The S521 owner ruling on
deploy/— the load-bearing one. Verbatim from the retro: “deploy/ belongs with the pipeline; the owner’s instinct was right and I argued him out of it.” Carried intoid-411.md:245-249(“Owner rulings recorded S521 […]deploy/belongs with the pipeline perreference/deployment-architecture.md, NOT with CI as first proposed”) and implemented atmempalace.yaml:118-120. This is R3, and it is the only sourced ruling in the whole corpus that speaks to where pipeline-adjacent things belong. It settles a concern, not a directory — §4. - The S525 DR-121 derivation — “Move the two runtime assets now, into
scripts/cocoindex_pipeline/— don’t wait forservices/.” Confirmsservices/has been the working shorthand since S525 and has never been ratified; DR-121’s own Alternatives section rejects waiting for it. - The id-412 fixture-relocation lesson, quoted in the code it produced
(
test_form_extractors.py:48-56): “the id-412 consumer census missed this file […] because a grep for the old path string cannot see either. If this tree moves again, grep for the SEGMENTS.” This is §1.4, and it is the single most directly applicable piece of institutional memory for this task — the last relocation of these files was undercounted by exactly this mechanism.
8. What are we not thinking about, but should be?
Section titled “8. What are we not thinking about, but should be?”- id-416 / DR-119 is a hard sequencing conflict, and it is
todoright now. id-416 replacescp -R scripts+ buildpacks with a checked-in Dockerfile whose COPY set is explicit. Those two tasks rewrite the same lines ofonprem-deploy.ymland the same module-path strings in four composes. Doing them in either order is fine; doing them concurrently is not. DR-121 already recorded one ordering constraint of this family (“the move lands BEFORE id-416 narrows the build context”). Whoever goes second gets a trivial job; whoever goes first must not assume the other’s shape. {443.2} must state the order and record it in both task files. There is a real argument for id-416 first: an explicit Dockerfile COPY list makes the deployed unit’s boundary a checked-in fact rather than something §2.2 had to derive by measurement. ci.yml’s paths-filter fails silently and green. Of every failure mode in this task this is the only one that produces a passing PR with the Python suite never run. It should be the first thing verified post-move, by pushing a Python-only change and confirming the pytest step actually fires — not by reading the filter.- The parallel pipeline lane is live on the same files. id-412/id-415 and the OKF
producer work (id-426/427/429/439) actively own
scripts/cocoindex_pipeline/.flow.py’s co-change partners were last touched 2026-08-09, one day before this task was minted. A 551-site prefix rewrite cannot be cherry-picked around concurrent edits — the task note is right that this needs a quiet window, and {443.2} should name the specific tasks that must be at rest. - The three “stayers” force a decision about
scripts/tests/itself. If three files remain,pyproject.tomlneeds twotestpathsand CI needs two pytest invocations — for three files. That is a worse shape than either “move all 93” or “re-home the three”. _okf_prototype_draft.py. An inbound importer whose name marks it a draft. I did not determine whether it serves a live requirement; carrying it into the new home by default would file a draft as production.- The
.claude/worktrees/agent-aa3b4551acc815315/checkout contains a stale copy of the deploy tree. Per the project CLAUDE.md, check itsgit statusbefore doing anything. __tests__/integration/cocoindex/_helpers/is.gitnexusignored and.memtraceignored — the TS half of the fixture-staging path lives in a directory no code-intelligence tool indexes. Closing that ignore is one of the cheap wins and it directly reduces this task’s own blind spot.
9. Docs — the partition rule and the counts
Section titled “9. Docs — the partition rule and the counts”1118 occurrences across 323 files in the docs-site (src/content/docs/ + tasks/).
The rule: repoint a file if and only if it makes a claim about the CURRENT state that would become false. Everything else is a point-in-time record and keeps the old path. Machine-rewriting the historical surface would rewrite history to describe a layout that did not exist when the work happened.
Operationally, three tests, in order:
- Frontmatter is the discriminator. A file carrying
kh_code_sources:/kh_last_verified:/kh_docubot_owned: truedeclares itself a live description of current code. Those repoint, and theirkh_last_verifiedbumps in the same commit (tracked-doc freshness guard). - Directory is the fallback.
reference/andrunbooks/describe now → repoint.specs/,tasks/,reports/,continuation-prompts/,ledgers/,workflow-evaluation/are point-in-time → leave. reference/decisions/is the exception that proves it. A DR is both a historical record and in force. Do not rewrite DR bodies — the paths in DR-117/118/121 are part of the reasoning as it stood. A DR that becomes wrong is amended by a new DR (§5.3), never edited.
| Partition | Files | Disposition |
|---|---|---|
| REPOINT — current | 42 | |
reference/ (excl. decisions/) | 7 | cocoindex-pipeline.md, deployment-architecture.md, platform-context.md, classification-architecture.md, classification-prompt.md, decision-register.md, change-log/2026-08.md |
runbooks/ (excl. _archive/) | 5 | onprem-b1-deploy.md, corpus-writer-fence.md, per-client-release-pins.md, private-ingress-cutover.md, secret-rotation.md |
ontology/ | 6 | DR-121 already established these are live authority (its Consequences: “9 live doc files […] Under DR-104 the doc is the authority”) |
product-functionality/ | 4 | Same family as DR-121’s nine |
initiatives/ — live only | ~14 of 21 | Needs a file-level pass — initiatives/ mixes live doctrine (okf-platform/bundle-doctrine.md) with closed initiative records |
docs/reference/testing/corpus-manifest.json (in the code repo, not docs-site) | 1 | 47 machine-read paths; guard-enforced |
| LEAVE — historical | 281 | |
specs/ | 186 | Point-in-time. DR-106 additionally presumes any spec below ~id-130 stale. |
tasks/ | 46 | Progress notes are dated records |
reports/ | 14 | Census/review artefacts, dated |
continuation-prompts/ | 13 | Session records |
ledgers/ | 10 | Frozen |
workflow-evaluation/ | 2 | Dated |
runbooks/_archive/ | 1 | Archived |
reference/decisions/ | 8 | Never rewrite a DR body — amend with a new DR |
| JUDGEMENT — owner call | ~7 | |
initiatives/ closed-initiative records | ~7 | Are these “current” or “historical”? |
Two docs-site files are special and must not be treated as prose:
reference/decision-register.md is a generated index — regenerate, do not hand-edit.
docs/reference/testing/corpus-manifest.json is machine-read by two consumers.
Delta vs the task’s 1394/363: S549 used a wider glob (likely including archive/,
dist/ or node_modules). 1118/323 is over src/content/docs/ + tasks/ only. Both are
the same order and the partition rule is unaffected; use 1118/323 as the working figure and
re-measure at execution time.
10. If this breaks in three months, why?
Section titled “10. If this breaks in three months, why?”Named concretely, weakest first.
-
ci.yml’s paths-filter is repointed wrong, and nobody notices for weeks. This is the only failure mode in the whole task that is silent and green. Thepython:filter (ci.yml:65-68) namesscripts/**/*.pyandscripts/tests/**. If the move repoints the Python entrypoint but the filter keeps ascripts/prefix, the pytest step’sif: steps.changes.outputs.python == 'true'never fires and every PR passes with zero Python tests run.pytest-nightly.ymlcatches it — the next night, in a lane nobody watches on a green day. Robustness: after the move, push a one-line Python-only change and assert the pytest step ran, and add an assertion that the filter’s glob matches at least one tracked file. -
The dual-spelling hazard (§5.1) resolves to two module identities instead of failing.
from docx_utils import open_document_safeandfrom scripts.docx_utils import open_document_safeare live simultaneously today. The move changespythonpath. If the new configuration leaves both spellings resolvable, the DOCX extraction path holds two copies of the same helpers — the exact ID-44.5/ID-177 shape, in the one code path wherecocoindex.ContextKeyregistration is process-global and single-shot. It would surface as an intermittent extraction failure under load, months later, with no diff to blame. Robustness: canonicalise the spellings before the move, in their own commit, and widenconftest.py’s collection-time guard to the whole runtime bundle. -
A subprocess prelude (§1.3) or a segment-wise
Path()(§1.4) is missed. Nineteen test files write Python source into a child process; fifteen sites build the package path fromPath()segments. Neither is visible to grep-for-the-old-path, togitnexus rename, or to an attribute-node AST pass. The last relocation of these files was undercounted by exactly this mechanism, and the code says so. A missed prelude fails only when that one test runs, which on a 93-file suite can be a long time after the merge. Robustness: the census script (§1.1) is checked in and re-run post-move as a gate — S1c must equal S1a for the new spelling, and S5a must be zero for the old segment. -
The move and id-416/DR-119 land in overlapping windows. Both rewrite
onprem-deploy.yml’s build-context block and the same module-path strings in four composes. A merge that takes one side’sCOPY/cp -Rand the other’sGOOGLE_ENTRYPOINTproduces an image that builds green and fails at container start on the live deploy path that Platform staging, Platform prod, the client boxes andbid-worker-platformall pull — DR-121’s exact warning, one task later. Robustness: sequence them explicitly in both task files; prefer id-416 first so the deployed unit’s boundary is a checked-in COPY list rather than something the next reader has to derive. -
The tests do not move, and DR-117’s cost compounds a third time. DR-117 has been filing files against the “cocoindex moves out of
scripts/” assumption since 1 August. If the package moves and the tests do not: the 15 segment-wise sites all need hand-editing; the palace room defect atmempalace.yaml:236-240never resolves and its comment becomes a false claim in a live config; and the next fixture or asset question re-litigates ground DR-117 and DR-121 already settled — “guaranteeing a third move of the same files”, which is the sentence DR-117 used to reject exactly this outcome. -
The docs are rewritten wholesale. 281 of the 323 files are point-in-time records. A
sed -iacross the docs-site would make every historical spec describe a layout that did not exist when it was written, and would silently edit eight in-force DR bodies. There is no test for this and it is unrecoverable without a revert. Robustness: the partition in §9 is executed as an explicit allowlist of 42 files, never as a repo-wide substitution.
11. Open questions
Section titled “11. Open questions”O1 — carried verbatim, per the dispatch brief:
“What requirement does a top-level
services/tier serve, and what is that requirement’s current source?”
Status: answered negatively, and the answer is auditable. Named sources searched — the
in-force register and all 8 relevant DR files, platform-context.md,
deployment-architecture.md, cocoindex-pipeline.md, and four MemPalace FTS passes. No
requirement and no source exist. The only two structural occurrences of services/
(DR-121’s Alternatives; the S525 drawer) are carriers of the suggestion and both reject
waiting for it. Independently, §2.2 shows the tier would be insufficient even if it were
ratified. services/ should be declined.
O2 — the residual, and it is an owner call. The requirement-satisfying shape is settled
(§2.3–2.5): a singular, root-level, Python-identifier-named directory holding the runtime
bundle and its tests, with deploy/ left at root. The name is not settled. pipeline/
and ingest/ both satisfy R1–R6; pipeline/ matches the repo’s existing vocabulary and
DR-116’s palace keyword directly. This is a naming decision with a mechanical consequence —
it becomes the live entrypoint string in four compose files and two workflows — so it is the
owner’s, not {443.2}‘s.
O3 — does the runtime bundle scope change the task? §2.2 shows six sibling .py modules
must move with the package. That is ~+6 files and ~+55 import sites beyond the task’s current
framing, plus §5.1’s canonicalisation. Owner-flagged as a scope call, not absorbed.
O4 — the three non-pipeline test files. test_audit_orphan_tests.py,
test_census_pg_evidence.py, test_pipeline_schema_uses_visibility.py. Move all 93 and
re-home these three later, or split testpaths for three files? Recommend the former;
{443.2}‘s call.
O5 — is _okf_prototype_draft.py live? An inbound importer marked a draft by its own
name. UNDECIDABLE as measured — I did not identify a requirement it serves or a current
source for one. Do not carry it into the new home on the assumption it is live.
O6 — sequencing against id-416/DR-119 (todo, high priority). Which goes first? Both
rewrite the same lines. Recommend id-416 first. Needs an owner or Coordinator decision, and
recording in both task files.
O7 — does test-philosophy.md §3 gain a Python clause? §3.1’s mirror rule is scoped to
__tests__/. If the Python tests move by DR-117’s rule, the doctrine is silent on the largest
test tree outside __tests__/. Amend in the same commit, or accept the gap knowingly.
O8 — the palace needs mempalace repair. Vector reads are disabled (1% HNSW divergence,
above the 2,000-drawer threshold). Unrelated to this task; recall for every session is
degraded to BM25 until it is run.