CocoIndex Ingestion Pipeline — reference

CocoIndex Ingestion Pipeline

The Python ingestion pipeline (scripts/cocoindex_pipeline/) is built on cocoindex — a declarative incremental-processing engine — pinned at cocoindex[postgres]==1.0.18 (requirements.txt). It runs as a long-lived sidecar container on-prem (IONOS VPS + Coolify) and turns staged corpus files + passed feed URLs into extracted, embedded rows in the Canonical platform Postgres schema.

Pin history (id-128 S507). The pin was bumped 1.0.7 → 1.0.18 after run 30377806744 forensics showed 1.0.7’s engine exhausting the pre_commit ownership-transfer retry loop under the nightly’s stage-while-walking load — 243 pre_commit gave up after 8 retries waiting for concurrent ownership transfer events across 30 staged fixtures, with affected files staying dead on every subsequent walk pass. The 1.0.8–1.0.18 window carries the LMDB / state-store hardening that resolves the race (#2299 cross-process LMDB handle validity, #2231 auto map resize, #2109 page-size rounding, #2251 cooperative timeouts). All twelve _coco_api._SYMBOL_SOURCES symbols were re-verified against 1.0.18 via scripts/tests/test_coco_api_facade.py before the pin flipped (bl-252 procedure). Deployed-surface caveat: 1.0.9 moved the UserState LMDB key (#2102); CI’s LMDB is ephemeral per run, but the staging/prod VPS /cocoindex-state/lmdb volumes need engine-migration verification or a state reset on first deploy at this pin — coordinate with the id-127 deploy topology before rolling to Coolify.

Working on the pipeline? The cocoindex library how-to lives in the in-repo skill scripts/.claude/skills/cocoindex/ (loads automatically when editing under scripts/).

The 6-stage flow (flow.py:app_main)

Stage	What	Where
1. Source walk	localfs.walk_dir(live=True, recursive=True) over COCOINDEX_SOURCE_PATH, plus a second source: FeedUrlSource enumerating the passed-URL ledger (feed_articles WHERE passed), every fetch gated by the validate_url SSRF check	flow.py, url_source.py, url_validation.py
2. Binary conversion	Per-MIME adapters: docling for PDF/DOCX/XLSX, the in-process Trafilatura cleaner for URL HTML, passthrough for markdown	adapters.py, extract.py
3. LLM extraction (“Path A”)	Four @coco.fn(memo=True) extractors call the Anthropic SDK directly + validate via Pydantic TypeAdapter: classification, Q&A/form, entity mentions, relationships. cocoindex has no ExtractByLlm (absent through 1.0.18) and no built-in retry — the KH-owned tenacity wrapper (_anthropic_retry) and the memo self-heal wrapper (extract_with_memo_self_heal) own that surface	extraction.py, prompts.py
4. Embedding	LiteLLMEmbedder("text-embedding-3-large", dimensions=1024) → vector(1024); rows land in record_embeddings(owner_kind, owner_id) — the single embeddings home (DR-036), no inline vector columns	flow.py
5. Post-fan-out passes	Entity resolution (_run_stage_5_resolution), then the Q&A dedup proposer — both run flow-scope after the per-item fan-out settles, both containment-wrapped so a pass failure is logged, not walk-aborting	stage_5.py, qa_dedup_proposer.py
6. Postgres UPSERT	mount_table_target(..., managed_by=ManagedBy.USER) — cocoindex writes rows only, never DDL (Canonical migrations own the schema). Mounted targets: q_a_extractions, source_documents, entity_mentions, entity_relationships, content_chunks, reference_items, record_embeddings (the former content_items mount is removed — DR-034)	flow.py

Around the stages:

• Per-item fan-out + containment — coco.mount_each runs one component per source item (ingest_file for files, ingest_url for URLs). An unexpected exception in one item’s ingest is tallied and logged, never promoted to a whole-walk failure.
• Producer chaining — on a clean walk completion, source_documents deltas are fed to trigger_producer_post_walk (producer/trigger.py); a failed walk never chains a producer run.
• Observability — flow start/end emit a pipeline-run webhook carrying the Canonical-generated run op_id, per-stage counts, Anthropic-retry / taxonomy-miss / per-item-failure / memo-heal counters, and extractor_version (IMAGE_SHA). The run context (op_id + the four observability counters) is published per-walk by app_main onto a FlowRunContext holder provided under a coco.ContextKey (detect_change=False) by the lifespan — see op_id stamping + memo fingerprint. The two emissions land on the SAME pipeline_runs row (Inv-16 — one row per invocation; the terminal emission UPDATEs the in_progress row started by the flow-start emission, it does not insert a sibling). The nightly workflow threads IMAGE_SHA into the sidecar docker run (-e IMAGE_SHA=${COCOINDEX_IMAGE_TAG}, Inv-8 forensic key) the same way the Coolify composes already do — without it every nightly-produced run row lacks result.extractor_version and extractor-version-cross-ref round-trip checks hard-fail.
• Idle mode — with COCOINDEX_SOURCE_PATH unset or missing, app_main logs and returns cleanly; nothing walks, nothing burns tokens.
• Lifespan — kh_pipeline_lifespan provisions the asyncpg pool (from COCOINDEX_DB_DSN) under DB_CTX and generates the entity-alias snapshot; KH_PIPELINE_APP = coco.App(AppConfig(name="kh_pipeline"), app_main) is the module-level App both entrypoints use.
• Blank-form gate on q_a_extractions (id-370) — both walk branches (flow.py:2291 Path-A mount_each, flow.py:3321 off-mount P4 ingest_once) skip the q_a_extractions declare_row when an extracted answer_text is NULL or whitespace-only. A blank form’s questions still extract to a NULL answer (a sanctioned result — prompts.py “OR null”; QAPair.answer_text: str | None), but the row is no longer minted because q_a_extractions feeds the answered-pairs-only promotion funnel. The skip preserves the enumerate idx feeding the uuid5(qa:{source_document_id}:{idx}) PK, so answered pairs keep their identity across re-walks of a mixed document. The companion migration 20260729185524_id370_promotion_candidates_answered_predicate.sql restores the same answered predicate on branch 1 of q_a_extractions_promotion_candidates() (symmetry with branch 3) and deletes the pre-gate NULL-answer rows (DR-093). See specs/id-370-unanswered-question-routing/.

Derived-row PK seeds — em:/er: registry-keyed on source_document_id

The entity_mentions and entity_relationships derived rows mint deterministic uuid5 PKs seeded on the STORED source_document_id — never rel_path. The seed formulas (in _ingest_content_branch):

• entity_mentions.id = uuid5(_KH_PIPELINE_DOC_NS, f"em:{source_document_id}:{canonical_name}:{entity_type}")
• entity_relationships.id = uuid5(_KH_PIPELINE_DOC_NS, f"er:{source_document_id}:{source_c}:{predicate}:{target_c}")

Document identity is content-hash-first (the admission resolver — DR-024 clause (i), restated by DR-026): the same bytes staged at a new rel_path resolve to the stored source_documents row, so seeding on source_document_id makes byte-identical re-stagings compute the SAME derived-row PKs and land as idempotent ON CONFLICT (id) upserts. Both ingestion paths (ingest_once and the engine content branch) now yield identical PKs per document — closing the F4 rel_path-keyed gap the {138.16} audit noted for the engine branch (id-398 / id-396 TECH §4 D1 upsert-absorb ruling).

The legacy rel_path-seeded engine-path rows (minted before id-398) are deleted outright by migration 20260729210500_id398_emer_registry_keyed_reseed.sql per DR-093 — delete, don’t backfill. Stage-5 may have rewritten canonical_name on entity_mentions, so the original seed input is not reliably recomputable; the next walk re-mints all derived rows under the fixed registry-keyed PKs. No FK references either table; DML only, no api-surface regen owed.

op_id stamping + memo fingerprint (id-400)

op_id is a PLAIN ROW FIELD stamped on every row the walk upserts. The S265 semantic — a no-op re-ingest does NOT re-stamp op_id; a full_reprocess walk DOES — requires the per-walk run context to stay OUT of the memoised components’ call fingerprints. The engine’s memo fingerprint covers function inputs + code only; coco.ContextKey values default to detect_change=False (“resources not affecting computation”) and are invisible to the fingerprint.

id-400 (D-397-A Option C — S265 restored) routes the run context (op_id plus the four observability counters, whose mutable state pickles into the fingerprint) through a FlowRunContext holder provided under the FLOW_RUN_CTX ContextKey by kh_pipeline_lifespan and published per-walk by app_main (resolve_flow_run_context() in flow_context.py; the holder is deliberately NEVER cleared at walk end — under live=True per-item components re-run on fs events after app_main returned and must still see the last walk’s run context). ingest_file / ingest_url resolve it INSIDE the memoised body — it is no longer a flow_* keyword argument of either component.

Consequences:

• An UNCHANGED item memo-HITS every later walk; its rows keep their last-materially-changed op_id.
• A changed item (byte-change) re-runs the body and stamps the CURRENT walk’s op_id.
• A full_reprocess walk bypasses memos entirely and re-stamps every row with the current op_id.
• The URL-branch FK-deferred backlink (which previously converged via the memo-busting walk-2 re-run) now converges via the flow-scope _reconcile_feed_article_backlinks post-pass — see URL backlink reconciliation.

This retires the prior ID-66.19 channel that threaded the run context as explicit kwargs via a named closure in app_main; that shape made them memo inputs (the engine fingerprints all args + kwargs), so the fresh per-walk op_id busted the outer memo on every walk and _upsert_source_document re-stamped every row — the {75.17}/bl-239 finding, ruled a defect of the CHANNEL (TRIAGE §3.1). The ID-66.19 thread constraint stays honoured: the holder crosses the _LoopRunner daemon-thread boundary via module/env scope (not contextvars), and ingest_file/ingest_url re-bind FLOW_META_CTX / _retry_counter_var / _taxonomy_miss_counter_var locally on the worker thread from the holder. The executable contract is scripts/tests/test_file_branch_memo_fingerprint.py (memo-HIT walk 2, re-stamp walk 3 byte-change, re-stamp walk 4 full_reprocess, context channel confirmed on cocoindex 1.0.18).

Curation pinning (id-400 / Inv-9)

Admin entity merges stamp entity_mentions.metadata.curation_pinned = true on the surviving target rows post-merge_entities (the merge route returns mentions_pinned in its response; a pin-step fault is surfaced via mentions_pin_error and never 500s — the merge itself has already committed). The ingestion walk honours the pin at three pipeline sites in stage_5.py and flow.py:

• Stage-5 write-back domain (_select_run_entity_mentions) excludes pinned rows at SQL — the walk may never UPDATE or DELETE a curated row, even when a full_reprocess legitimately re-stamped its op_id back into this run’s scope. Pinned canonicals still reach the resolver as PINNED existing seeds via the op-agnostic roster read (_select_existing_canonical_roster), so new mentions chain UNDER a curated canonical; the curated row itself is untouchable.
• Cross-op survivor rule (_run_stage_5_resolution) — a pinned cross-op key-holder wins UNCONDITIONALLY (whatever the confidence ranking says); the current-op survivor collapses into it via an op-scoped DELETE. The pin predicate is re-asserted AT the DELETE as defence-in-depth so a curated row is undeletable here even if the selection logic regresses.
• em-declare carry-forward — the em-declare loop re-declares pinned rows VERBATIM (stored canonical/metadata/op_id), so a pinned row survives full_reprocess and wins natural-key collisions.

Closes census #41 failure #1 (an admin merge reverted on a later walk). The marker rides the existing entity_mentions.metadata jsonb — no migration.

URL backlink reconciliation (id-400)

After url_handle.ready() in app_main, a flow-scope _reconcile_feed_article_backlinks post-pass backlinks any passed-but-unlinked feed_articles rows whose reference_items reference landed THIS walk. The per-item step-7 backlink FK-defers on a walk-1 race (the reference_items row flushes only after its component returns); pre-id-400 convergence relied on the retired memo-busting per-walk op_id kwarg re-running the item on walk 2. The pass is CONTAINED best-effort (mirrors the producer-trigger post-pass): a reconcile fault is logged and never fails the walk whose writes already landed. A successful link logs cocoindex.url_backlink_reconciled with the linked_count.

Module roles

Module	Role
flow.py	The pipeline itself: app_main (6 stages), target mounts, counters/webhook rollup, lifespan + KH_PIPELINE_APP.
extraction.py	Pydantic extraction shapes, the four Path-A extractors, _anthropic_retry, post-memo stamping (stamp_extraction_base), memo self-heal. Single source of truth for ANTHROPIC_MODEL (re-exported by flow.py). Owns the _extraction_async_client() factory — the single Anthropic-client construction site for the four extractors + pair_resolver, which scrubs empty-string credential env before construction so the deploy-env-driven LLM-tier mechanism (${VAR:-} passthrough in the platform composes) is safe — see reference/deployment-architecture.md §2.2.
server.py	aiohttp HTTP wrapper for the deployed container: health probe, on-demand walk trigger, staging/extract/producer routes, and the id-400 walk registry + GET /walk-status/{requestId} attribution surface. Boot enters the cocoindex lifespan only (coco.start_blocking()) — a container boot/restart can never auto-walk the corpus (bl-221); walks fire only on explicit POST /walk.
__main__.py	Local-dev entrypoint (python3 -m scripts.cocoindex_pipeline): KH_PIPELINE_APP.update_blocking(live=True) with continuous fs-watch.
_coco_api.py	Insulation façade for off-surface cocoindex symbols — a version bump is a one-file fix.
mock_llm.py	id-389 tier-1 LLM substrate — an in-image mock Anthropic /v1/messages server (python3 -m scripts.cocoindex_pipeline.mock_llm). Serves deterministic, schema-valid extraction payloads so a staging or CI corpus walk exercises the full wiring path at $0 and with no live Anthropic key. See § LLM tier substrate (id-389) below.
Supporting	adapters.py / extract.py (conversion + Trafilatura cleaner + quality gate), url_source.py / url_validation.py (URL source + SSRF), stage_5.py (entity resolution + curation pinning), qa_dedup_proposer.py, producer/ (post-walk producer), writer_fence.py (corpus-writer mutual exclusion), canonicalisation.py, entity_context.py, prompts.py (the four extractor system-prompt constants mock_llm.py exact-matches against). flow_context.py owns the FlowRunContext holder + FlowRunMeta / counter bindings — see op_id stamping + memo fingerprint.

HTTP surface (server.py:build_app)

Route	Auth	What
GET /health	none	Liveness reflecting the cocoindex worker thread: 200 while alive, 503 after a worker crash.
POST /stage	—	Multipart byte-drop into the watched corpus dir (ID-62). destPath is a corpus-relative file path, not a URL — a ?... query suffix is REJECTED with a named 400 and nothing is written, with exactly one allowlisted directive: ?failStage5=<embedder|pair_resolver> is stripped off the path before the write, armed one-shot, and consumed by the next POST /walk (id-414 AC-5/AC-6). The response echoes the path actually written (suffix stripped) and includes failStage5 when the directive was honoured.
POST /walk	Bearer PIPELINE_TRIGGER_SECRET	On-demand corpus walk: single-flight lock → writer fence → pull-sync materialise (Supabase Storage → corpus dir) → update_blocking(live=False) — one non-live pass, then idle. The walk runs INSIDE the fence hold and never acquires separately; a busy fence skips the pass (no walk, retry later). Accepts a full_reprocess flag. Each accepted pass is registered (id-400) with a requestId and one terminal status (completed / failed / fence_busy) — see GET /walk-status.
GET /walk-status/{requestId}	Bearer PIPELINE_TRIGGER_SECRET (same as /walk)	id-400 (NM-5) walk attribution surface: returns one registry entry’s lifecycle (accepted → running → completed / failed / fence_busy) and, on completion, the opId app_main minted for that pass (read back from the FlowRunContext holder, which retains the last walk’s run context by design). The registry is bounded observability (oldest-evicted past 50 entries), not a ledger — pipeline_runs is the ledger. Unknown requestId → 404 (evicted or never accepted). An id-414 Stage-5 failure-injection pass surfaces stage5FailureInjected on the entry carrying the mode the next walk consumed.
POST /extract	Bearer EXTRACT_API_TOKEN (fails closed if unset)	Pure-cleaner seam: HTML in → {text, verdict, warnings} out, no fetch. The one Traefik-reachable route; 20 MB per-route body cap.
POST /producer-run	Bearer PIPELINE_TRIGGER_SECRET	Manual forced producer pass over every concept, bypassing the post-walk delta gate; independent single-flight lock.

Boot-required environment

Two distinct databases — easy to conflate (see the deploy runbook §0):

• COCOINDEX_DB_DSN — Postgres DSN for the Canonical-owned asyncpg pool (row writes).
• COCOINDEX_DB — filesystem path for the cocoindex engine’s internal LMDB state store (memoisation). Not a DSN.
• COCOINDEX_SOURCE_PATH — the watched corpus dir; unset ⇒ idle mode.

/stage destPath query-suffix contract (id-414)

destPath is a corpus-relative FILE PATH, not a URL, so a ?... tail on it means nothing to the filesystem. Before this contract the handler joined the tail into the target and wrote files literally named name.xlsx?fullreprocess=1; extension routing then failed to recognise them and the items were dropped per-item. That is 19 of the 159 ingests the S522 nightly dropped while still reporting completed.

The contract is reject, not strip, with exactly one allowlisted directive:

• Reject. _stage_handler’s failure model (Inv-5) — “a client-correctable mis-wire is a NAMED 400, never a silent accept” — rules the case. A caller who wrote ?fullreprocess=1 meant something by it. Silently stripping would write a file the caller never asked for and return 200, the silent-accept shape that made the S522 drop invisible. Rejecting is also strictly stronger against the requirement “can never land as a filename”: nothing is written at all. An unknown ? suffix, a suffix with no path, a directive missing =, or an unknown mode is a named 400 listing the valid modes; no bytes are written.
• Allowlisted directive. ?failStage5=<embedder|pair_resolver> is stripped off the path before the write, armed one-shot, and consumed by the next POST /walk. The directive overrides that mode’s credential env vars (OPENAI_API_KEY for embedder; ANTHROPIC_API_KEY + ANTHROPIC_AUTH_TOKEN for pair_resolver) with an invalid sentinel for exactly one walk; _run_walk consumes it via the stage5_failure_injection() context manager and restores the original environment in a finally, so a failed or fence-busy walk cannot leak the injected credential into a later pass. The injected value is a non-empty invalid credential, deliberately NOT an unset/empty var — extraction._extraction_async_client() deletes empty-string ANTHROPIC_* vars (its id-389 guard), and a bare AsyncAnthropic() raises at construction rather than producing the anthropic.AuthenticationError the Inv-12/13 contract documents.
• Blast radius. Stage 5 is a per-RUN resolution pass over the whole walk’s entity mentions, NOT a per-document step, so an armed directive fails Stage 5 for EVERY document in that walk, not only the fixture that armed it. “Fail Stage 5 for one fixture” is not expressible at any seam at this layer. That is why the directive is opt-in, one-shot, and restored in a finally.
• Scope. The contract covers ? only. A # fragment (or any other URL-ish punctuation) is still written verbatim — # is a legal filename character on POSIX and no caller has ever sent one, so widening the guard would be speculative.

The 2xx response body echoes the path ACTUALLY written (suffix stripped) and includes a failStage5 field when the directive was honoured; GET /walk-status/{requestId} surfaces stage5FailureInjected on the registry entry of the walk that consumed it. The TS wire helper is __tests__/integration/cocoindex/test-helpers.ts::injectStage5Failure; its docstring was rewritten in the same PR to match behaviour.

Deploy

Deploy/ops are runbook territory — do not duplicate here:

• Deploy runbook → runbooks/onprem-b1-deploy.md (Cloud-Run-era predecessor archived: runbooks/_archive/cocoindex-deploy.md)
• Corpus writer-fence (mutual exclusion + LMDB non-rebuildability) → runbooks/corpus-writer-fence.md
• On-prem B1 topology (IONOS VPS + Coolify) → runbooks/onprem-b1-deploy.md
• Staging-stack ops (incl. idle-mode boot triage) → runbooks/onprem-b1-deploy.md; stand-up history (archived S491) → runbooks/_archive/staging-coolify-cocoindex.md

Testing

python3 -m pytest scripts/tests/ (deps: pip install -r requirements.txt). Directory gotchas (worktree CWD, sandbox skip baseline) live in the in-repo scripts/CLAUDE.md.

LLM tier substrate (id-389)

The four Path-A extractors in extraction.py reach the Anthropic API through four zero-arg AsyncAnthropic() clients. The pinned anthropic SDK (0.79.0) honours the ANTHROPIC_BASE_URL environment variable natively, so a corpus walk can be redirected at a non-Anthropic endpoint with no code change. id-389 formalises a three-tier model that keeps wiring iteration $0 and credential-independent:

Tier	ANTHROPIC_BASE_URL	Auth	Purpose
1. mock (default)	http://mock-llm:<port> (CI) or http://mockllm-platform-staging:8080 (staging compose)	none — the mock never validates the key	Wiring iteration: rows land, statuses flow, webhooks fire, fence held — at $0 and with no live key. Canned output cannot adjudicate extraction QUALITY.
2. openrouter	https://openrouter.ai/api (the Anthropic-skin /v1/messages endpoint)	ANTHROPIC_AUTH_TOKEN Bearer (= PRODUCER_AUTH_TOKEN); ANTHROPIC_API_KEY forced EMPTY	Real Claude extraction billed via OpenRouter credits for extraction-QUALITY runs while the direct Anthropic key stays disabled. Mechanics are the producer/agent_loop.py slice C/E recipe.
3. anthropic (final parity)	absent (SDK default = https://api.anthropic.com)	ANTHROPIC_API_KEY (real key)	One run before promoting a fix — never the scheduled default.

Two load-bearing invariants live in extraction.py:

• No hardcoded base_url. All four AsyncAnthropic() calls are zero-arg, so the env var is the single redirect seam. A code-level base_url= override would defeat the tier model silently.
• An empty ANTHROPIC_BASE_URL breaks the SDK. anthropic 0.79.0 takes '' over its default — so the variable must never be set to a blank value. On the mock and openrouter tiers it carries a real URL; on the anthropic tier it is simply OMITTED (never blanked) so the SDK falls back to its default.

The mock server (mock_llm.py)

mock_llm.py runs from the SAME pinned pipeline image as the sidecar — no separate artefact, no pip-at-boot — and is version-locked to the extraction code it mocks. The contract with extraction.py:

• Discrimination is an EXACT string match of the request’s system-block text against the four prompts.py constants (the same objects the extractors send via _cached_system_block). A prompt edit breaks the match loudly — a 400 naming the mock — instead of silently serving the wrong shape.
• Canned payloads are CONSTRUCTED from the real pydantic classes and revalidated through extraction.py’s own TypeAdapters at import time, so the mock fails at boot — not mid-walk — if the schemas drift.
• Streamed responses end with stop_reason: "end_turn", so _guard_not_truncated passes (the client.messages.stream(...) path bl-222 requires the full Anthropic SSE event grammar, which the mock speaks).
• A system-less request gets a plain "pong" message — the id-389 credential preflight ping — while an UNRECOGNISED system prompt is a loud 400.

The mock is compose-internal / runner-local only — a mock that answers /v1/messages unauthenticated must never be reachable off-box (no Traefik labels, no published ports).

Credential preflight (the S501 burn-class guard)

The cocoindex-nightly.yml lane prepends a credential preflight step that runs one cheap /v1/messages ping against the REAL endpoint whenever a live credential is in play (the anthropic and openrouter tiers). A dead, disabled, or unreachable key fails HERE — at minute 1, before the buildpacks build and a fence-gated walk — instead of after a 75-minute burn (the S501 failure mode this step exists to kill). The mock tier has no live credential; its own /health gate is its preflight.

OpenRouter tier-2 auth invariant

On the openrouter tier the sidecar’s ANTHROPIC_API_KEY is forced EMPTY for two reasons (the producer slice E double rationale):

1. A non-empty Anthropic-shaped X-Api-Key silently pins OpenRouter’s provider routing — the cheap-model variant would be served, not the real-model one.
2. The real Anthropic key would be transmitted to a third party.

The extraction model IS a deploy-env knob now — EXTRACTION_MODEL (id-389 AC-3, S559; extraction.py::_resolve_extraction_model). Blank means unset and falls back to the production default claude-opus-4-6; S559 ratified tier-2 extraction on z-ai/glm-5.2 through this knob. The earlier claim that a model-override knob was “deliberately not smuggled in” as a DR-060 memo hazard is superseded by DR-150: the effective LLM identity ("{base_url}:{model}", extraction.py::resolve_llm_identity) is a required argument of every memoized extractor, so memo entries are identity-scoped and a tier or model switch can never serve another identity’s memoized output.

One-way coupling to state, not gloss over: the producer’s own model knob falls back to extraction.py’s ANTHROPIC_MODEL when PRODUCER_MODEL is unset — setting EXTRACTION_MODEL on a deployment that leaves the producer’s unset moves BOTH lanes. The deployed platform composes set the producer’s explicitly, so the lanes stay independent there.

DR-150’s recorded rider still stands and is the operator-relevant half: the OUTER ingest_file memo keys on file identity/bytes only, so an env-only tier flip on unchanged bytes memo-HITs and never reaches the identity-scoped inner extractors — a silent no-op walk. Forcing a real re-extraction requires POST /walk with {"full_reprocess": true} or an engine-store wipe; the inner identity key protects changed-bytes / full_reprocess / self-heal paths only.

Tier selection

• Scheduled cocoindex-nightly runs default to the mock tier (inputs is empty on schedule, so the llm_tier fallback applies).
• workflow_dispatch input llm_tier (mock | openrouter | anthropic) opts a single run into a different tier — anthropic for one-run final parity before promoting a fix.
• Staging compose — mockllm-platform-staging is a sibling service of the cocoindex sidecar in deploy/coolify/docker-compose.platform-staging.yaml; the sidecar’s ANTHROPIC_BASE_URL defaults to it. Production compose is deliberately untouched (absent = SDK default = real API). See reference/deployment-architecture §2.2.