Skip to content
Canonical

id-377 research — declarative surface inventory (agent sweep)

Declarative / declared-as-data surface inventory

Section titled “Declarative / declared-as-data surface inventory”

Read-only agent sweep of scripts/ (tests excluded), commissioned for {377.4}. Verbatim findings; the companion design note (declarative-write-surfaces.md) records which of these v1 handles.

Verdict on the flow.py _insert_*_row raw-SQL mirrors — already visible

Section titled “Verdict on the flow.py _insert_*_row raw-SQL mirrors — already visible”

All six are plain inline string literals (implicit concatenation folds to a single ast.Constant) passed as the first positional arg to conn.execute(...):

Functiondefconn.executeTableINSERT cols
_insert_content_chunk_rowflow.py:2953flow.py:2975content_chunks8 ($7::vector)
_insert_record_embedding_rowflow.py:2997flow.py:3012record_embeddings4, ON CONFLICT (owner_kind, owner_id, model)
_insert_entity_mention_rowflow.py:3025flow.py:3046entity_mentions9 ($8::jsonb)
_insert_entity_relationship_rowflow.py:3069flow.py:3085entity_relationships6
_insert_qa_extraction_rowflow.py:3101flow.py:3127q_a_extractions12 ($11::jsonb)
_upsert_source_documentflow.py:2671flow.py:2809source_documents26, ON CONFLICT DO UPDATE SET 21

Caveat: the five _insert_* helpers take conn: Any as a PARAMETER (acquired once at flow.py:3260 in ingest_once, passed down). Attribute-name matching on {execute, fetch, …} still fires; a detector that first proves the receiver is an asyncpg connection will not.

Other cocoindex declarative write APIs in use

Section titled “Other cocoindex declarative write APIs in use”
  • localfs.declare_file(...) — a SECOND declarative write surface: 6 sites in producer/bundle_writer.py (:367, :690, :1051, :1090, :1141, :1366). Same declare-and-reconcile model as declare_row, but for markdown bundle FILES, not Postgres columns (orphan-delete semantics at bundle_writer.py:39-53). Relevant only if lineage ever covers file artefacts.
  • TableTarget.declare_vector_index(...) deliberately NOT used (rationale flow.py:4150-4158 — out-of-band DDL). No KTable/collector .export(), no declare_dir_target.
  • _coco_api.py is a PEP 562 lazy re-export façade: runtime imports are invisible (__getattr__ + _SYMBOL_SOURCES dict :55-78); the real from cocoindex... import lines exist only inside if TYPE_CHECKING: (:121-137). Name-based collection (not import-graph resolution) is the only shape that sees through it.

Declared-as-data DB access beyond TableSchema/mount/declare_row + _SQL_* consts

Section titled “Declared-as-data DB access beyond TableSchema/mount/declare_row + _SQL_* consts”
  • Column lists as comma-joined string constants interpolated into SQL (l_records.py): _QA_COLUMNS :390 (13 q_a_pairs cols), _SOURCE_DOCUMENT_COLUMNS :425 (16 source_documents cols), _QA_WON_COLUMNS :539 (a constant interpolating another constant). Interpolation sites :397, :414, :433, :438, :545. [v1: resolved by the order-respecting module-const f-string resolution.]
  • Genuinely dynamic SQL (l_records.py): :1110-1121 conditional tuple-unpack + computed placeholder index; :1315-1316/:1322-1323 f"{CONST} LIMIT $2" [v1: the latter resolve; the former stay caveated]. Query selection additionally gated by client-authored concept-feeder.json (:781-790 → :1100, :1168, :1318).
  • supabase-py with ClientOptions(schema="api") — bid_worker.py:104: every from_("t") resolves to api.t (view mirror), not public.t. Payloads built as list comprehensions (:490-501 → :503 upsert 8 cols; :526-545 → :546 insert 15 cols); on_conflict= carries column names as a comma string (:505); .order("created_at") :199 is a read-side column mention in a modifier only.
  • Row-mapping subscript reads with no SQL in-file: url_source.py (11 sites), server.py (5), canonicalisation.py:376-377 (rows arrive as a parameter; originating SELECT at flow.py:4504); canonicalisation.py:299, 319-327 reads the entity_aliases shape from a JSON snapshot fixture.
  • Stored procs — visible SQL, zero column tokens: flow.py:2655 (resolve_or_mint_source_identity, mints/updates source_documents), writer_fence.py:137/:161, bid_worker.py:783 (claim_next_job → processing_queue). [The pg_proc-scan follow-up’s surface.]
  • Named SQL const outside l_records: url_source.py:64-69 _PASSED_URLS_SQL, consumed by name at :196. [v1: resolved.]
  • Non-SQL routes into the write path: verify_driver.py:206-212 POSTs to /stage; server.py:1449-1452 route registry; server.py:834-836/:1191 engine kick via thread dispatch.
  • Confirmed absent tree-wide: executemany, copy_records_to_table, CSV/COPY, pandas.to_sql, psycopg, sqlalchemy, raw PostgREST, psql/supabase subprocesses.
  • Structural finding: sd_target is mounted (flow.py:3907) but NO sd_target.declare_row exists — every source_documents write goes through raw SQL in _upsert_source_document (rationale flow.py:2720-2784), which writes 26 columns vs the schema’s declared 22 (adds logical_path, admission_status, retention_class, origin_type). “Mounted” ≠ “written declaratively”; declared-column sets can be deliberate SUBSETS with omission semantics (GENERATED ALWAYS / PG defaults, flow.py:1420-1437).

Hard cases for a static resolver (25)

Section titled “Hard cases for a static resolver (25)”
  1. **kwargs absorption erasing the param name across 4 hops — producer/trigger.py:137
  2. Dispatch through a function-valued parameter — producer/trigger.py:158
  3. Lazy in-body imports breaking the module import graph — producer/embed.py:133-135 (also flow_def.py:374-376, trigger.py:117-119)
  4. *targets splat with arg↔param binding set by argument ORDER at a mount_each the engine replays — flow.py:4030 (order bound :4059-4069)
  5. Mount inside a closure stored in a module global, executed elsewhere — server.py:1152 (stored :1171, run :1191)
  6. Cross-module schema-constant reference — server.py:1152 (flow.RECORD_EMBEDDINGS_SCHEMA) [v1: handled via attribute-name candidates]
  7. declare_row(row=row) payload built in a loop into a dict — flow.py:2480 (dict literal :2466-2478) [v1: handled via scope-dict fallback]
  8. ** spread inside a jsonb column’s value — flow.py:2408
  9. Column list as comma-joined string constant in SQL — l_records.py:397 [v1: handled]
  10. Constant interpolating another constant — l_records.py:539-542 [v1: handled]
  11. Conditional tuple-unpack SQL + computed placeholder index — l_records.py:1110-1121 [caveated]
  12. Module-level named SQL const passed by name — url_source.py:196 [v1: handled]
  13. Mounted target never declared onto; raw SQL writes a LARGER column set — sd_target flow.py:3907 (22) vs :2809 (26)
  14. Declared columns deliberately a SUBSET; omission carries semantics — flow.py:1420-1437
  15. List-comprehension payload to supabase .insert/.upsert — bid_worker.py:546
  16. Column names only in an on_conflict= string — bid_worker.py:505
  17. Column names only in a chain modifier — bid_worker.py:199
  18. Stored-proc call: visible SQL, zero column tokens — flow.py:2655 [caveated: sqlFunctionSourceSites]
  19. Column reads with rows arriving as a parameter — canonicalisation.py:376-377
  20. Schema shape read from a JSON fixture — canonicalisation.py:299, 319-327
  21. Connection as a function parameter, not a module pool — flow.py:2953
  22. PEP 562 __getattr__ façade; real imports only under if TYPE_CHECKING: — _coco_api.py:55-78 vs :121-137
  23. ColumnDef encoder callable changing the wire type — flow.py:1469-1473 (_encode_pgvector)
  24. Star-splatted parameter tuple built earlier — qa_dedup_proposer.py:446
  25. One physical declare_row serving three owner_kinds from three modules — flow.py:1505, fed by flow.py:2250, flow.py:3724, producer/embed.py:137