Orchestrator grounding — ID-28.20 RESEARCH re-authoring (S265)

Orchestrator grounding — ID-28.20 RESEARCH re-authoring (S265)

Purpose. 28.20’s mandate is “validate API reality against the installed codebase BEFORE spec’ing.” Before dispatching the Planner, the Orchestrator did exactly that. This memo records what the installed cocoindex 1.0.3 actually exposes, so the Planner starts from verified facts (and re-verifies) rather than from the fictional spec.

Installed package identity

• cocoindex 1.0.3, dist name cocoindex, Homepage https://cocoindex.io/.
• Location: ~/Library/Python/3.14/lib/python/site-packages/cocoindex.

Finding 1 — bind_target is fictional (CONFIRMED — matches 28.20 mandate)

flow.py:757-765 calls content_text.bind_target(ci_target, key_fields=…, op_id=flow["op_id"]) under # type: ignore. There is no DataSlice class and no cocoindex.flow module in 1.0.3 at all — the top-level surface has no bind_target anywhere. flow["op_id"] is likewise a placeholder subscript. Consequence as stated in the mandate holds: op_id columns write NULL, Stage-6 does not UPSERT.

Finding 2 — the 28.20 MANDATE’s prescribed fix is ALSO wrong API (NEW — second-order spec error)

The 28.20 details prescribe the “REAL cocoindex contract” as collect → export (“every written column incl. op_id is a FIELD of the collected row”). That is the public CocoIndex dataflow API (@flow_def / data_scope / add_collector / collector.export). The installed 1.0.3 package does not expose collect, export, add_collector, or flow_def anywhere.

Installed 1.0.3 is the reactive Component / Target-state-reconciliation API: top-level exports include App, AppConfig, mount_target, mount_each, declare_target_state, TargetState, TargetHandler, ContextKey, fn, use_context. flow.py is already written against THIS surface (coco.App, @coco.fn, coco.ContextKey, cocoindex.connectors.postgres, cocoindex.connectorkits.target.ManagedBy) — only the write call (bind_target) was fictional.

This is a recursive instance of the exact failure 28.20 corrects: the corrective subtask’s own mandate prescribes a non-existent API. The Planner MUST NOT implement collect → export. It must encode the real declare_row contract below and note the mandate correction in RESEARCH.md.

Finding 3 — the REAL write contract: TableTarget.declare_row(row=…)

cocoindex.connectors.postgres public API: mount_table_target, declare_table_target, table_target, TableTarget, TableSchema, ColumnDef, PgType, ValueEncoder, create_pool, RowFetcher.

mount_table_target(db: ContextKey[asyncpg.Pool], table_name: str, table_schema: TableSchema,
                   *, pg_schema_name=None, managed_by=ManagedBy.SYSTEM) -> TableTarget[RowT]
  # "Sugar over table_target() + coco.mount_target() + wrapping."

TableTarget.declare_row(*, row: RowT) -> None
  # "Declare a row to be upserted to this table.
  #  row: dict, dataclass, NamedTuple, or Pydantic model. Must include all primary key columns."

TableTarget.declare_vector_index(*, column, metric, method, …)   # pgvector index → Stage-4 embedding
TableTarget.declare_sql_command_attachment(*, name, setup_sql, teardown_sql=None)

ManagedBy = {SYSTEM, USER}. USER ⇒ cocoindex writes rows but does NOT touch DDL (KH migrations own schema) — already the intent in flow.py:732-749.

Canonical end-to-end pattern (from the bundled turbopuffer connector _target.py):

@coco.fn
def process_doc(doc: Doc, target: NamespaceTarget) -> None:
    target.declare_row(row=Row(id=doc.id, vector=doc.embedding, attributes={...}))

→ the write happens inside a @coco.fn that receives the transformed value + the target handle and calls declare_row. op_id is simply a field of the row dict (sourced from FLOW_META_CTX, which stamp_extraction_base() / bind_flow_meta() already populate — the hook the mandate cites exists). No identifier-binding API, no collect/export.

Open empirical question for the Planner (the deep bit, not yet pinned by the Orchestrator): how a transformed-slice value reaches a concrete row argument to declare_row in the reactive App/mount model — i.e. the row-scope iteration / mount_each / per-source-item @coco.fn wiring that turns content_text.transform(...) outputs into rows. Establish this empirically against the installed examples/connectors + run a live single-file probe before finalising the write-path spec.

Amendment targets already located (for the Planner, re-verify)

• docs/specs/id-28-cocoindex-flow-scaffolding/PRODUCT.md — behaviour-level + largely SOUND. Inv-11/12 (op_id stamping + round-trip) are correct contracts; only the mechanism was fictional. Likely no PRODUCT change beyond a note; confirm.
• TECH.md §P-2 (L297-301) — flow.bind_target(...) sketch → rewrite to declare_row.
• TECH.md §P-4 (L375, L442) — “augment bind_target() to pass op_id=flow['op_id']” → rewrite: op_id as a row field on the declare_row row.
• TECH.md §P-5 (L470) — “UPSERT encapsulated inside mount_table_target / bind_target” → correct.
• TECH.md §P-8 / Inv-23/24 + P-OQ2/3 — retry/DLQ reality (cocoindex native LLM HTTP-429 retry vs KH tenacity wrapper around the direct anthropic call in @coco.fn vs zero per-row postgres retry).
• flow.py:700-765 — the live fictional call sites.

In-scope items to assess criticality (per 28.20 + brief)

Real Stage-6 declare_row write path (CRITICAL — nothing writes without it) · op_id as row field (folds former Inv-11 gap + non-functional 28.9) · real source binding (T8 shipped empty, PLAN O-Q8) · Stage-4 embedding (declare_vector_index is the native hook) + Stage-5 entity-resolution stubs · _emit_upsert_log() live wiring (folds former ID-44.1; low criticality) · retry/observability SPEC accuracy (ID-44.2) · test-hardening: importlib.reload() per-file isolation for test_cocoindex_flow_*.py (+ fold backlog ID-177 idle_mode↔server ContextKey clash if convenient).

Source docs to re-ground against

docs/plans/phase-0-investigation/architecture/02-data-flow.md §3 (6-stage topology) + §5 (op_id hybrid) · docs/plans/phase-0-investigation/0.9-decision-graph.md §11.4.1 N7 · docs/research/ cocoindex-1.0.3-extractbyllm-spec-reality-investigation.md.