{412.1} Cocoindex nightly reliability — grounding + workstream plan

Status: GROUNDING ARTEFACT — the shared evidence base for the W1–W9 workstream set. Authored across the id-128 sessions by the owner; relocated here at S522 from specs/cocoindex-nightly-fix-plan.md when the workstreams were split into tasks. It is the {412.1} RESEARCH artefact for id-412 and the cited grounding for id-413, id-414, id-415, id-404 (W4) and id-128 (W9).

Read the ## S522 verification and corrections section at the foot before acting on any workstream — every W was re-verified against main @ 63602df9, and three of the plan’s premises have moved since it was written.

Cocoindex nightly reliability — implementation plan

Goal: make the cocoindex-nightly lane trustworthy — no silently-dropped ingest items, the correct (Platform) corpus walked, PDF ingest working in the sidecar image, and the Stage-5/entity_mentions integration tests passing rather than timing out.

Problem statement

The nightly lane reports completed while silently dropping 159 per-item ingests in a single run (69 PDF, 55 .doc, 16 .xls, 19 .xlsx?fullreprocess=1), the wrong corpus is being ingested, and a class of Stage-5 tests fail with 60s hook timeouts. Evidence is from the run-30676215562 sidecar artifact and live reads of the Platform-staging Supabase branch (rbwqewalexrzgxtvcqrh).

Confirmed findings (evidence)

• PDF ingest fails at convert_binary_to_markdown with ImportError: libxcb.so.1: cannot open shared object file (×69). Docling’s PDF layout model imports OpenCV (cv2); opencv-python==5.0.0.93 (pulled transitively via commonforms→rfdetr→ultralytics) needs libxcb, absent from gcr.io/buildpacks/builder:google-22. do_ocr=False is necessary but insufficient. DB confirms: 0 of 73 source_documents are .pdf.
• Three more silent-drop classes are contained per-item and never flip the run status: .doc (×55) and .xls (×16) are legacy binary Office formats Docling does not support (confirmed against Docling’s InputFormat enum), and .xlsx?fullreprocess=1 (×19) is a query suffix leaking into a written filename.
• /stage (server.py:687) writes destPath verbatim with no query-suffix stripping, so a caller passing name.xlsx?fullreprocess=1 (or the injectStage5Failure ?failStage5= convention) produces a file literally named with the suffix, which then fails extension routing. injectStage5Failure’s docstring claim that the service strips the suffix is false.
• entity_mentions is not globally empty (9 rows) but is missing for the two largest XLSX docs ([53.14-INV06].xlsx cc=43 em=0; [52.13-INV19]-mode1.xlsx cc=38 em=0) while small docs carry 1–3. stage_counts.llm_extraction=4 proves all four extractor passes (incl. entity_mentions) ran; the log shows 0 UniqueViolation/ForeignKey, so this is NOT a write collision.
• Stage-5 hook timeouts: several beforeAlls (e.g. cross-document-dedup.integration.test.ts:79) budget only 60_000ms while doing W2 stage→/walk→await work (walk helper default 300_000ms + fence retries); the trivial afterAll gets 600_000ms. Budgets are inverted.
• The verify_driver.py “duplication” is intentional, not accidental: scripts/cocoindex_pipeline/verify_driver.py is the in-container, stage-only, no-Supabase FILE-fixture driver; deploy/onprem/verify/verify_driver.py is the host-side, service-role URL-mode driver (Inv-21: one primitive, two parameterisations). Keep both; no consolidation.

Deploy layout & image-build approach (owner question)

The sidecar image is built via Cloud Native Buildpacks (gcr.io/buildpacks/builder:google-22) from a pruned context (scripts/ + requirements.lock). onprem-deploy.yml builds and publishes to ghcr; the nightly replicates the SAME recipe locally without --publish; the Coolify composes (deploy/coolify/) and on-prem tooling (deploy/onprem/) consume it. The only Dockerfile in the repo is the unrelated deploy/onprem/backup/Dockerfile.tools, so dependency changes live in requirements.txt → recompiled requirements.lock. Is buildpacks the right approach? It is a reasonable, consistently-used default, but it is precisely what makes OS-level deps painful — the libxcb failure is a direct symptom. opencv-python-headless (W1) sidesteps it because it needs no OS packages, so buildpacks stays viable now. However, if id-404 (W4) adopts LibreOffice soffice pre-conversion, that DOES need an apt-level package buildpacks cannot add cleanly — at which point a purpose-built Dockerfile (full control of base image + apt layers) becomes the better architecture. Recommendation: keep buildpacks for now; treat “move the sidecar to a Dockerfile” as a decision gated on W4’s outcome, not an immediate change.

Proposed workstreams

Each maps to an ordna task; specs (RESEARCH/PRODUCT/TECH) export to the docs-site repo knowledge-hub-docs-site under specs/id-N-<slug>/ per the ledger protocol. New task IDs are placeholders pending Coordinator assignment.

W1 — PDF ingest in the image (opencv-python-headless)

Ensure only opencv-python-headless provides cv2 so no libxcb dependency remains. Add opencv-python-headless to requirements.txt and override/exclude opencv-python (both currently appear in requirements.lock), recompile the lock, and verify python -c "import cv2" in a buildpack image without X11 libs. Rebuild path is the nightly pack build step; the on-prem/Coolify deploy inherits it. Reject the alternative (apt-install libxcb1) per owner decision to go headless.

W2 — Fail loud for every file type (contained-fault visibility)

Purpose framing (owner): the lane exists to DOGFOOD the Platform E2E and catch regressions before promoting functionality to clients, and to prove the pipeline processes documents correctly. So the gate’s pass criterion is “every in-scope Platform-corpus document processed correctly”; a contained per-item fault on that corpus is a real regression and MUST be visible. Flip the terminal run status to completed_with_errors when the existing item_failures tally is non-zero (the tally already threads to the webhook; flow.py currently resolves terminal status without consulting it), and surface a per-run item-failure count. The nightly job stays GREEN for expected-unsupported inputs that are NOT part of the Platform corpus (reported as log + artifact + warning), but REDS when an in-scope Platform-corpus document fails — that is the regression signal the job exists for. TS coupling to fix: test-helpers.ts::pollPipelineRunCompleted treats only completed as success and will hang on completed_with_errors; update it (and awaitWalk consumers) to accept completed_with_errors. Maps to a new fail-loud task.

W3 — Walk the correct (Platform) corpus (id-406)

Are we configured correctly, or was something missed? Something was missed — this is a genuine misconfiguration, not a non-technical misread. The best-practice mechanism ALREADY exists: deploy/onprem/sync-platform-corpus.sh (ID-134) treats the vendored scripts/cocoindex_pipeline/fixtures/platform-corpus/ as the single source of truth and stages it onto the walked volume (/cocoindex-state/corpus) via rsync → docker cp, after which COCOINDEX_SOURCE_PATH points at it and a supervised /walk ingests it. The nightly does NOT follow this pattern: it docker cps scripts/tests/fixtures/form-extraction/ into the app dir (as source bytes for verify_driver/stageFixture) and never seeds the vendored Platform corpus as the walked baseline — so the lane isn’t dogfooding the Platform E2E path at all; it only processes per-test-staged form fixtures. “Matching production” is the wrong yardstick (the platform isn’t live yet) — align to this established best-practice mechanism instead. Fix: make the nightly seed + walk the vendored Platform corpus using the sync-platform-corpus.sh Hop-B pattern (or invoke the script) so the walked/watched baseline IS platform-corpus/content/, with per-test fixtures staged on demand (not wholesale-copied). Relocate scripts/fixtures/ (.html/.atom platform-feed seeds — processed but NOT walked/watched) and reconcile with the pre-existing scripts/tests/fixtures/extraction/ (.html/.txt) to remove the duplication that predates the internal-KH → cocoindex pivot. Consult scripts/.claude/skills/cocoindex/ (SKILL.md + references/patterns.md) for LiveMapView walk/watch best practice. id-406 proposed a relocation; a short unify-the-fixtures investigation is warranted first because the layout is genuinely scattered — validate id-406’s approach against the sync-platform-corpus.sh source-of-truth before acting.

W4 — Form-vs-supplementary ingestion + legacy binary Office (id-404, SPIKE / spec-chain)

This is NOT a simple “reject or convert” fix — it needs its own investigation/spike and likely a spec-chain, because it defines product behaviour spanning id-145 / id-146 / id-147 (and possibly id-345) and must respect id-159. The core question: when a user brings a form to complete (e.g. an RFP) PLUS related files, how does the system decide what is a FORM to extract (producing inputs matched against q&a pairs) versus a SUPPLEMENTARY document that only provides context for the human/agent completing the form? The files under scripts/tests/fixtures/form-extraction/templates/rfp-british-council/ are exactly this real scenario (a British Council RFP), so they are legitimate product inputs — not junk to drop. Working assumptions to validate in the spike: (1) legacy .doc/.xls carry valuable context and should be CONVERTED — Docling supports neither (confirmed against its InputFormat enum), and LibreOffice soffice --headless --convert-to is the standard bridge — and be viewable as binaries in the platform; (2) per id-159, .xlsx/.docx that ARE forms are converted to PDF for commonforms field extraction, while SUPPLEMENTARY files are NOT PDF-converted; (3) Docling remains the text-extraction path for supplementary context. Deliverable: a RESEARCH.md grounding + PRODUCT/TECH specs in the docs-site under the owning task before any adapter code. The buildpacks→Dockerfile decision above is gated on whether soffice must land in the image. Decoupling: W3 removes these files from the WALKED corpus, so id-404 does NOT block the nightly going green — it is a parallel product workstream, not a nightly-reliability fix.

W5 — /stage query-suffix bug + injectStage5Failure

Make /stage (server.py) strip or reject a ?... query suffix on destPath before writing, so name.xlsx?fullreprocess=1 cannot land as a filename. Repair injectStage5Failure (test-helpers.ts) so its ?failStage5= directive is actually honoured (server strips + acts on it) rather than silently corrupting the filename. Add a unit test at the /stage boundary.

W6 — Repoint the MIME-coverage tests to the Platform corpus (+ hook budgets)

Repoint ALL of sidecar-mime-coverage.integration.test.ts’s MIME fixtures — not just the PDF — to the vendored Platform corpus, so the coverage test dogfoods the same corpus the lane walks. The Platform corpus currently has .pdf (synthetic-capability-statement.pdf), .docx (synthetic-sector-intel.docx) and .md, but NO .xlsx — so add a synthetic .xlsx (a plain CONTENT workbook, e.g. a synthetic pricing/data sheet, NOT a blank extraction form) to platform-corpus/content/ to cover the XLSX MIME with a real content doc. The MIME test asserts only that each type extracts to non-empty content_chunks via canonical Docling ingest — NOT form-field extraction (the separate commonforms Plane-2 path in form_extractors/pdf.py) — so content docs are the correct instrument. Also swap the inverted Stage-5 hook budgets (give the stage→walk beforeAll a budget ≥ the walk helper’s 300s default; shrink the trivial afterAll).

W7 — entity_mentions: wrong fixture + write-drop tracing

Two things converge. (a) The Stage-5/entity tests use the CSP .xlsx as their “content” document, but it is a blank extraction FORM, not prose — the wrong instrument for a test that needs reliable entity_mentions. Repoint these tests to a Platform-corpus CONTENT document (an .md/.docx with real prose the mock reliably tags). (b) Independently, the large-XLSX em=0 case is still a real pipeline signal to explain, so complete the trace REGARDLESS (repointing (a) would otherwise just mask it): run ONE walk of the affected fixture in isolation on the mock tier and query entity_mentions by that op_id + source_document_id IMMEDIATELY after the walk, before teardown. If 0 → a per-item declare/flush drop in flow.py’s Stage-6 entity_mentions block (add a temporary declared-vs-landed structured log there; grep the sidecar log for that op_id). If >0 → teardown: dropFixture deletes entity_mentions then source_documents, and a blocked source_documents delete (FK from q_a_pairs/reference_items/record_lifecycle) would orphan doc+chunks with em already gone; check for those FK children.

W8 — Consolidate the integration-test fixture references

~22 cocoindex integration test files each re-declare const FIXTURE_PATH = '…/csp-cloud-security-principles/Cloud Security Principles Checklist V5_3.xlsx' (e.g. op-id-scoping.integration.test.ts:54) — a duplicated hardcoded path to the WRONG (form) file, with no shared source of truth. Introduce a single shared fixtures module (e.g. __tests__/integration/cocoindex/_helpers/fixtures.ts) exporting named canonical paths — Platform-corpus CONTENT docs for the walk/entity/Stage-5 tests, and the form-extraction fixtures ONLY for tests that genuinely exercise form extraction — and repoint all consumers. Result: one edit-site for any future fixture move, and the content-vs-form intent becomes explicit per test.

W9 — Documentation accuracy + retire superseded functionality (id-128)

Agents read reference/ and runbooks/ session-to-session, so stale guidance actively MISDIRECTS them — this is the review’s primary interest. Focus: corpus seeding. Every doc that describes how the walked corpus is populated must reflect the sync-platform-corpus.sh source-of-truth (vendored scripts/cocoindex_pipeline/fixtures/platform-corpus/ → docker cp onto /cocoindex-state/corpus → supervised /walk), NOT the form-extraction docker cp the nightly currently performs nor any pre-pivot internal-KH mechanism. Audit these for corpus-seeding claims and correct any that misdirect: reference/platform-context.md, reference/cocoindex-pipeline.md, reference/deployment-architecture.md, runbooks/onprem-b1-deploy.md, runbooks/corpus-writer-fence.md, runbooks/ci.md, runbooks/platform-to-client-promote.md (all in the knowledge-hub-docs-site repo). Also document the (iii)/W7 clarification so agents don’t conflate the two platform inputs: the OKF producer reads DB L-records (downstream of the ingest walk) via LRecordsSource, or OKF_SOURCE_REPO_PATH via RepoDocsSource, and NEVER the vendored fixture directly. The other architecture points (source_documents not content_items, platform-staging = a Supabase branch, buildpacks image, webhook-only pipeline_runs, writer-fence) are believed already accurate — verify in passing, but corpus seeding is the load-bearing fix. Retire-on-supersede discipline (extends id-128’s review-and-retire-redundant-functionality remit): when this work supersedes a doc section, decision record, or code path, mark it superseded / remove it in the SAME change rather than leaving a stale parallel that re-misleads the next agent. Concrete instance: reference/decision-register.md DRs from the retired content_items era (e.g. dr-034) need an explicit superseded marker pointing at source_documents. Deliverable: a contradiction list (file:line, observed vs expected) approved before any doc edit; changes land in the docs-site repo, and retirements are tracked under the id-128 review-and-retire task.

Open questions / decisions

• W3: seed the nightly walked corpus by invoking sync-platform-corpus.sh’s Hop-B pattern (docker cp of the vendored tree) vs a bespoke nightly step — and the final home for the relocated scripts/fixtures/ html/atom seeds (and whether they merge with scripts/tests/fixtures/extraction/).
• W4: full spike/spec-chain scope and owning task across id-145/146/147/345/159 — and, dependent on it, whether the sidecar moves from buildpacks to a Dockerfile to host soffice.
• W6/W8: the exact Platform-corpus content doc each repointed test should use, and the shape of the new synthetic .xlsx.
• W2: the dogfooding purpose now sets the boundary (Platform corpus = in-scope, reds the job; incidental unsupported inputs = warning) — confirm no other inputs should be treated as in-scope.

Validation

• Local: bun run test (Vitest) for the touched integration tests where they can run; python -m pytest scripts/tests/test_cocoindex_verify_driver.py and the /stage unit test; python -c "import cv2" inside a rebuilt buildpack image; targeted SQL reads against rbwqewalexrzgxtvcqrh to confirm a PDF source_documents row now lands and item_failures is surfaced.
• Nightly: a manual workflow_dispatch (mock tier) after W1–W3; confirm the run reaches completed/completed_with_errors as designed, a .pdf row lands, and the job is green.

Sequencing

W1 (unblocks PDF) and W3 (correct corpus) are highest-value and land first; W2 follows W3 so the gate is tuned against the correct corpus; W5, W6, W8 are independent and small; W7’s trace runs against a real walk; W4 is a separate spike/spec-chain that does NOT block the nightly; W9 (docs + retire-superseded) follows W3 so the docs describe the corrected corpus-seeding mechanism, and its retire-on-supersede discipline applies continuously as the other workstreams supersede things. Each workstream lands as its own reviewed change with its ordna task file updated as work progresses, and specs export to the knowledge-hub-docs-site repo.

S522 verification and corrections

Added 2026-08-01 (S522) by the Coordinator when the workstreams were split into tasks. The body above is preserved verbatim as the owner authored it; this section records what was re-verified against main @ 63602df9 and what has since moved. Where the two disagree, this section wins.

Every workstream re-checked — all nine are OPEN, but ONE was not verified

Read the W7 row before trusting this table’s header. Eight workstreams carry a file:line or a task pointer as evidence; W7 carries an assertion. The distinction was blurred in the first version of this section and is corrected here — an “all nine verified” claim over an eight-of-nine table is exactly the shape of error this document exists to stop.

None of W1–W9 has been silently completed by the intervening sessions. Evidence, per workstream:

W	State	Evidence on main @ 63602df9
W1 PDF / opencv	OPEN, half-done	requirements.txt:93 already declares opencv-python-headless==4.13.0.92 with an explanatory comment — that half predates this plan (last touched 369edcd9, S511). But requirements.lock:353 still ships opencv-python==5.0.0.93 alongside headless at :357. The lock was never recompiled, so the libxcb closure is unchanged and import cv2 remains ambiguous.
W2 fail loud	OPEN	flow.py:4203 hardcodes flow_status: PipelineRunStatus = "completed"; "completed_with_errors" appears only in the Literal at :730 and is never assigned. The item_failures tally threads to the webhook at :4700 as the plan describes.
W3 correct corpus	OPEN	.github/workflows/cocoindex-nightly.yml:657 still docker cps form-extraction fixtures only. No step seeds scripts/cocoindex_pipeline/fixtures/platform-corpus/ onto the walked volume.
W4 form-vs-supplementary	OPEN	= id-404 (backlog, initiative: core-product).
W5 /stage query suffix	OPEN	_stage_handler (server.py:~644-702) rejects absolute paths and corpus-root escapes but performs no ? query-suffix strip.
W6 MIME + hook budgets	OPEN	Budgets still inverted (cross-document-dedup.integration.test.ts:79 = 60_000 beforeAll / :84 = 600_000 afterAll). sidecar-mime-coverage.integration.test.ts:96-108 points its PDF/DOCX/XLSX fixtures at form-extraction templates, not the Platform corpus.
W7 entity_mentions	OPEN — ASSERTED, NOT VERIFIED	No entity_mentions query, grep or DB read was run this session. The em=0 evidence is the owner’s plan restating a prior run, carried forward unchecked. id-415 must establish the current state before trusting the plan’s cc=43 em=0 / cc=38 em=0 figures — they may have moved.
W8 shared fixtures module	OPEN	17 files under __tests__/integration/cocoindex/ still hardcode the CSP .xlsx path. __tests__/integration/cocoindex/_helpers/ already exists (fixture-staging.ts, walk.ts) and is the natural home for the shared module.
W9 docs + retire-on-supersede	OPEN, scope grew	= id-128. See correction (2) below — the fixture relocation invalidated a further set of doc claims beyond the corpus-seeding set the plan enumerates.

Three premises have moved since the plan was written

(1) The docs/testing/ fixture tree no longer exists. Commits b9c55abf, 90cc3b03 and eb9ccac9 moved docs/testing/test-data/ to scripts/tests/fixtures/form-extraction/templates/, and repointed cocoindex-nightly.yml, verify_driver.py and the Coolify composes with it. c64be60b then aligned the census gate and verify_driver dest paths to a flat verify/<basename> scheme. Any path in the body above rooted at docs/testing/ is dead.

(2) That relocation is NOT W3, and it is not the target shape either — but it is what surfaced W3. Owner ruling, S522: pointing at docs/testing/ was incorrect, and while repointing to form-extraction isn’t the correct approach either, it’s what led to the finding that neither location was correct — and that pointing at the actual synthetic platform corpus had been missed entirely.

This is the load-bearing correction in the whole plan. The nightly has never walked the Platform corpus. It walks per-test-staged form-extraction fixtures, which is why the lane has not been dogfooding the Platform E2E path at all. W3 remains exactly as the body describes it, and it is now understood as the primary fix rather than one of six — the corpus was not merely in the wrong directory, it was the wrong corpus.

Consequence for id-406 (corpus manifest + conformance guard): that task’s ACs, its “four fixture trees” list and both D3 orphan paths were all written against docs/testing/, and are stale. It must be re-grounded so the manifest registers the vendored Platform corpus as the walked baseline and the form-extraction templates as per-test staged fixtures — the staging_mode field TECH §1 already specifies is exactly the axis that distinguishes them. id-406 is not dispatch-safe until that re-grounding lands.

(3) A large retirement sweep landed between the plan and the split. Commits 12de303b, 082ea92f, 4ea1979b, ed03b329, db43aca1 deleted roughly 19,000 lines: the change-reports subsystem, the MCP dashboard / quality / change-report / content tool families, lib/content/content-suggestions.ts and their tests. This is W9-adjacent — it is retire-on-supersede executed ahead of the plan — and it discharges part of id-392’s AC-1 (lib/mcp/tools/content.ts was one of the named readers to repoint; it no longer exists). It did not touch W1–W8 ground: all of id-408’s surfaces, the 17 CSP-path consumers and the Platform corpus tree are intact.

Workstream → task map (settled S522, owner-selected)

Task	Workstreams	Depends on
id-412	W3 + W8 — nightly walks the Platform corpus; one fixture source of truth replacing the 17 hardcoded paths	id-406 (re-grounded)
id-413	W1 — purge opencv-python from the image closure	none — dispatchable now
id-414	W2 + W5 — contained item-faults reach the run status; /stage query-suffix	id-412
id-415	W6 + W7 — repoint MIME + Stage-5 tests to Platform-corpus content docs; fix inverted hook budgets; trace the entity_mentions drop	id-412
id-404	W4 — form-vs-supplementary + legacy OLE spike/spec-chain	none; explicitly does NOT block the nightly
id-128	W9 — documentation accuracy + retire-on-supersede	id-412

The plan’s own sequencing paragraph still governs within that map: W1 and W3 are highest-value and land first; W2 follows W3 so the gate is tuned against the correct corpus; W5/W6/W8 are independent and small; W7’s trace runs against a real walk; W4 is decoupled; W9 follows W3.

S523 corrections — the ## Deploy layout & image-build approach section

Added 2026-08-02 (S523) while executing W1 (id-413). Same precedence as the S522 section above: where these disagree with the body, these win. Three of that section’s claims are wrong, and one of them retires a W4 open question.

1. The nightly does NOT replicate the same recipe — the two images differ

The body states “onprem-deploy.yml builds and publishes to ghcr; the nightly replicates the SAME recipe locally without --publish”. The pack build call is near-identical, but onprem-deploy.yml does not stop there: it pulls the published image back and re-publishes it through a post-pack layer (onprem-deploy.yml:~403-411) that apt-get installs git, openssh-client, libreoffice-writer and libreoffice-calc as root, then restores the app user. cocoindex-nightly.yml:445 runs bare pack build and stops.

So the nightly has been exercising an image missing four baked packages the deployed image carries. That is a straight contradiction of the lane’s stated purpose — dogfooding the Platform E2E path — and it belongs to id-412 as a finding in its own right, alongside W3’s corpus problem. Same class of defect: the lane is not testing the thing it is supposed to be testing.

id-413 mirrored only its own opencv step into both recipes rather than unifying them; full parity (or extracting a shared composite action) is id-412’s call.

2. There IS a Dockerfile in the build path

The body states “The only Dockerfile in the repo is the unrelated deploy/onprem/backup/Dockerfile.tools, so dependency changes live in requirements.txt -> recompiled requirements.lock.” The first clause is true only of files named Dockerfile* on disk. onprem-deploy.yml writes one via heredoc at build time (cat > "${WORKDIR}/Dockerfile" <<DOCKERFILE -> FROM <image> / USER root / RUN apt-get … / USER <orig>) and docker builds it.

The conclusion that follows from the false premise is the load-bearing part: dependency changes do not all live in requirements.txt. OS-level packages already have an established, in-production home.

3. W4’s Dockerfile question is already answered — soffice is in the image

The body says: “if id-404 (W4) adopts LibreOffice soffice pre-conversion, that DOES need an apt-level package buildpacks cannot add cleanly — at which point a purpose-built Dockerfile becomes the better architecture”, and the Open Questions list carries “whether the sidecar moves from buildpacks to a Dockerfile to host soffice”.

LibreOffice has been baked into this image since ID-145 {145.31} (DR-059). The post-pack layer installs libreoffice-writer + libreoffice-calc --no-install-recommends (deliberately excluding libreoffice-java-common / default-jre to bound the size delta) and appends soffice --headless --version as a build-time smoke test. bid-worker-platform runs the SAME published image via a command: override, so this is the single place both consumers get soffice.

Consequences:

• Buildpacks CAN add apt-level packages cleanly here — via a thin post-pack layer, which is already load-bearing for three packages plus LibreOffice.
• The buildpacks -> Dockerfile decision is NOT gated on soffice. That gate is discharged. If the move is still wanted it needs a different justification; it should not be carried in id-404 as an open question on these grounds.
• id-404/W4’s .doc/.xls conversion assumption (1) is better positioned than the plan assumes: the binary is already present and smoke-tested at build time.

4. W1’s own recipe was necessary but not sufficient

W1 says “override/exclude opencv-python … recompile the lock, and verify python -c "import cv2" in a buildpack image without X11 libs.” The lock half landed (dc03256c). It does not fix the image, because the buildpack installs the lock with pip install --requirement and no --no-deps (GoogleCloudPlatform/buildpacks, pkg/python/python.go basePipInstallArgs), so pip re-resolves each pinned package’s own metadata. ultralytics==8.4.92 declares opencv-python!=4.13.0.90,>=4.6.0 and supervision==0.29.1 declares opencv-python>=4.5.5.64; both arrive via commonforms and neither can be dropped. Verified by dry-run resolve of the lock’s own pins, which plans + opencv-python==5.0.0.93.

Fixed at e4316cb1 in the post-pack layer of both recipes, with the import cv2 check promoted from a one-off manual verification to a build-time guard.

5. requirements.lock does not describe the image (new finding, needs a task)

The lock carries no nvidia-* / CUDA packages and effectively no environment markers (3 ; in 704 lines): it is compiled on macOS/arm64 and installed on linux-x86_64, with pip re-resolving on top. So torch’s Linux dependency set is backfilled by pip at build time, not described by the lock.

This is the mechanism behind more than one defect — it is why deleting a line from the lock changed nothing, and how the {145.13} commonforms/pypdf gap could exist unnoticed. It also rules out the otherwise-obvious PIP_NO_DEPS=1 fix, which would ship torch incomplete. Making it a genuine closure means uv pip compile --python-platform x86_64-unknown-linux-gnu, which adds the CUDA package set and interacts with the workflow’s image-size-budget step — out of scope for id-413 and tracked separately.