DR-121: Pipeline runtime assets leave the test tree — scripts/cocoindex_pipeline/data/
DR-121 — Pipeline runtime assets leave the test tree
Section titled “DR-121 — Pipeline runtime assets leave the test tree”Context
Section titled “Context”scripts/cocoindex_pipeline/extraction.py:129 reads
scripts/tests/fixtures/taxonomy_snapshot.json at module import, in
production, and raises when it is absent or malformed — it is the runtime
content_types / form_types validator source, named as the HARD-reject gate
in ontology/04-content-type.md. canonicalisation.py:286 reads
entity_aliases_snapshot.json from the same directory and fails soft to
baseline-only.
Neither is a test fixture. Both are production runtime assets filed under a test
tree, and both paths are assembled segment-wise
(Path(__file__).parent.parent / "tests" / "fixtures" / …), so no path grep
finds them — the same class of invisibility that nearly broke two consumers
during the S524 relocation.
This is load-bearing now rather than tidy-later. DR-119 replaces the buildpacks
build with a checked-in Dockerfile whose COPY set is explicit. Today the image
carries these files only because the build context is cp -R scripts wholesale.
A Dockerfile that copies scripts/cocoindex_pipeline/ alone kills extraction.py
at import — in the live deploy path that Platform staging, Platform prod, the
client boxes and bid-worker-platform all pull.
Decision
Section titled “Decision”taxonomy_snapshot.json moves to scripts/cocoindex_pipeline/data/, and the
move lands BEFORE id-416 narrows the build context. DR-117 files fixture trees
by owning domain and explicitly rejects scripts/tests/; this extends the same
rule to runtime data, which has a stronger claim still. data/ rather than
fixtures/ because these are not fixtures and must not join the manifest’s
declared fixture roots.
entity_aliases_snapshot.json is NOT moved with it. Its generator,
scripts/generate-entity-aliases-snapshot.ts, has zero callers — absent from
package.json scripts (sync:taxonomy chains four scripts and this is not one),
from every workflow, and from deploy/onprem/*. The “generated at deploy
(PI-15)” story in canonicalisation.py:122-140 is self-justification; the same
comment concedes the wiring is a follow-up. Do not repoint a path nothing writes:
either wire the generator into deploy or delete generator + loader branch, as its
own decision.
The two remaining JSONs under scripts/tests/fixtures/ stay put —
url_normalisation_parity.json and canonicalisation_parity.json are genuine
cross-language parity case tables read only by tests. (url_normalise.py names
the file in a docstring at :19 and never opens it.)
Alternatives considered
Section titled “Alternatives considered”- Wait for the planned
scripts/→services/move. Rejected: the files travel inside the package either way, so waiting buys nothing and leaves id-416 blocked on a live deadline. scripts/cocoindex_pipeline/fixtures/. Rejected: that root is anchored by DR-117 and the corpus manifest for fixture trees. A runtime validator source filed there invites exactly the input/output category slip that put a blank extraction form in sixteen content tests.- Leave them and special-case the Dockerfile COPY. Rejected: encodes the wrong shape into the new build recipe on day one.
Consequences
Section titled “Consequences”- ~12 code consumers repoint, plus 9 live doc files —
ontology/README.md(the ID-133 Decision-A authority chain),ontology/01,04,05,runbooks/taxonomy-change-runbook.md, and four underproduct-functionality/. Under DR-104 the doc is the authority, so moving the file without them makes the docs wrong; they land in the same commit. Specs for task ids below ~130 (id-36, id-52, id-63, id-68) are presumed stale per DR-106 and are deliberately NOT churned. - The “Taxonomy dual-source” rationale in
CLAUDE.mdandextraction.py:123-126is wrong in both directions and is corrected in the same commit:content-type-registry.tsre-exports a generated tuple, and the ratified human source is the docs-site register markdown. TS and the snapshot are peer DB-derived mirrors, not one derived from the other. - Ordering constraint on id-416: this move is a precondition of the build-context
narrowing, recorded in
tasks/id-416.md.