ID-110 — Re-home url_import onto reference_items: RESEARCH (surface reconciliation + recommendation)

ID-110 — Re-home `url_import` onto `reference_items`: RESEARCH

{110.1} RESEARCH artefact. Evidence-led reconciliation of four surfaces, a conflict/overlap map, a recommended target design, migration/back-compat analysis, and framed open questions for Liam. This is not a behaviour spec — it frames the decision a follow-on {110.2} PRODUCT / {110.3} TECH would commit to.

Task: ID-110 “Re-home url_import (manual single-URL ingest) onto reference_items” (bl-298, from the ID-107 {107.4} deferral).
Predecessor deferral: ID-107 {107.4} — DEFERRED parent-ratified Option C (S344).
Authored: S-current (13/06/2026), fresh Planner dispatch.
Language: UK English; dates DD/MM/YYYY.
De-identification: all references are to “the client” / “the tenant” / “a workspace” generically; no client-specific literals.

0. Headline finding (read this first)

The {107.4} deferral blocker is now substantially obsolete. When {107.4} was deferred (S344), the stated blocker was threefold: (1) reference_items requires source_document_id NOT NULL + a PullMD/Docling body the /api/ingest/url route cannot produce; (2) url_source.py:59 defers url_import to post-v1 (hardcodes rss_feed); (3) no app-side reference_items writer exists. All three statements were accurate at the content_items-half framing. But the ID-75 Slice-1 landing path has since shipped and is live:

The async reference-write path is real and battle-tested — _ingest_url_body (scripts/cocoindex_pipeline/flow.py:2626-2801) fetches via PullMD/Docling, mints the sd:/ri: uuid5 pair, and declares both rows. The body problem is solved on the pipeline side.
The DB schema already admits url_import — reference_items.ingestion_source CHECK IN ('rss_feed','url_import') (supabase/migrations/20260606121451_id75_reference_items_layer.sql:18-19). No migration is needed to permit the value.
ID-75 already ratified the disposition (PRODUCT BI-9/BI-23; TECH §6.3, OQ-T1 RATIFIED): url_import is the reserved acquisition-route value for the manual-URL re-point (the {42.12} re-point this Task IS), and url_source.py:60 hardcoding rss_feed was always the feed-route-only v1 stance — url_import “joins when a manual-URL route ships (post-v1)” (url_source.py:58-59, verbatim).

So ID-110 is not a fresh greenfield contract. It is the execution of the already-specced, already-ratified {42.12} re-point that ID-75 scoped out of its own implementation window (BI-23 reconcile flag) and that {107.4} deferred because the async-vs-sync entry-point mismatch was unresolved. The remaining genuine decision is how the synchronous app route produces (or defers production of) the body — see §4.

1. Problem & context

/api/ingest/url (the manual single-URL “add content” option on the platform item page) TODAY inserts a content_items row stamped ingestion_source: 'url_import' (app/api/ingest/url/route.ts:191, .from('content_items').insert(...) at :207-211). Under the ID-75 O4/D4 reframe, this is definitionally wrong: content_items = client-adopted knowledge; external material (a single URL the user pasted) is evidence, not knowledge, and “external gets one reference per URL” (ID-75 PRODUCT DP-2). An externally-sourced URL belongs on reference_items, the global workspace-less reference layer, exactly as the RSS-feed route already lands there via cocoindex.

{107.4} correctly identified the re-point as the {42.12} work, attempted it, and deferred (Option C, S344) because the synchronous app route and the asynchronous cocoindex feed path are different entry points with different body-production models — and at that moment no resolution of the body problem for the sync route had been chosen. That is the gap ID-110 closes.

The reframe in one line: url_import should write one reference_items row + one source_documents provenance row per normalised URL (the BI-1 evidence pair), never a content_items row.

2. Surface-by-surface current state (verified)

All file:line and migration evidence below was confirmed against this HEAD on 13/06/2026.

Code-intelligence orientation (cited verbatim, not paraphrased)

gitnexus_query({query: "url import reference items ingest pipeline write", repo: "knowledge-hub"}) — top non-test process symbols and standalone definitions returned:

Function:app/api/ingest/url/route.ts:POST (:26-499) — the manual-URL route (Surface 1).
Function:scripts/cocoindex_pipeline/flow.py:_ingest_content_branch (:1966-2388) — the file-corpus content branch (sibling of the URL branch).
Test surfaces (live proof the URL write path works): Method:scripts/tests/test_cocoindex_url_write_path.py:TestUrlLandingDeclaresEvidencePair.test_landing_declares_exactly_sd_and_ri_with_field_contract (:225-300); scripts/tests/test_cocoindex_flow_failure_mode.py:TestUrlPerItemFailureIsolation.* (PullMD 5xx isolation, FK-deferral convergence — :1960-2166); scripts/tests/test_url_source.py:TestCrossWorkspaceCollapse.*.

The Python pipeline is outside ast-dataflow’s TypeScript corpus; Python grounding below used gitnexus + targeted grep sweeps (file:line citations throughout), per .ast-dataflow/CLAUDE.md. ast-dataflow string-literal-uses url_import over the TS corpus is implied by the grep sweep (string literal lives in route.ts, layer-inference.ts, schemas.ts comments only).

grep -rln "url_import" (Python + SQL): scripts/cocoindex_pipeline/url_source.py, supabase/migrations/20260428174512_add_ingest_source_to_content_items.sql, supabase/migrations/20260428180945_backfill_ingest_source.sql, supabase/migrations/20260606121451_id75_reference_items_layer.sql. grep -rln "url_import" app/ lib/ --include=*.ts: app/api/ingest/url/route.ts, lib/layer-inference.ts, lib/validation/schemas.ts (comment only — see §6).

Surface 1 — the LIVE manual-URL route (`app/api/ingest/url/route.ts`)

Confirmed. The route (POST, :26-499) runs an 18-step synchronous pipeline:

Auth getAuthorisedClient(['admin','editor']) → authFailureResponse(auth) (:30-31).
Rate limit 10/min (:39).
Body parse IngestUrlBodySchema (:44); fields = url, content_type?, user_tags?, skip_dedup?, content_owner_id? (lib/validation/ingest-schemas.ts:9-27).
SSRF validation validateUrl(url) (:68; lib/extraction/url-validation.ts).
URL-already-exists soft check against content_items.source_url (:74-90).
extractFromUrl(url) (lib/extraction/url) — fetches + extracts a body (:93-94).
Quality gate (<100 chars ⇒ 422) (:98-106).
generateEmbedding (:131).
Dedup soft-block (:153-172).
content_items INSERT with ingestion_source: 'url_import' as const (:191), source_url, source_domain, content, embedding, dedup_status, plus a metadata.ingestion_source: 'url_import' mirror (:195); .from('content_items') at :207-211.
Date extraction → expiry_date/lifecycle_type (:230-271).
classifyContent (:274-285), generateSummary (:288-299), quality score (:301-341).
Layer inference — inferLayer({ ingestionSource: 'url_import', ... }) (:348-358); lib/layer-inference.ts:197 special-cases url_import, and layer-inference.ts:21 types ingestionSource: 'manual' | 'url_import' | 'upload' | 'bid_library'.
Topic + guide-section suggestions (:376-460).

What it produces: a content_items row + classification + summary + quality score + layer/topic suggestions. It does NOT create any source_documents row — confirmed: the TS source_documents writers are app/api/upload/route.ts (file upload), app/api/items/[id]/route.ts, the source-documents/* management routes, and lib/mcp/tools/content.ts; the URL ingest route is not among them. This is the crux of the body problem (§4).

It does NOT write reference_items. Confirmed: grep -rln "reference_items" app/ lib/ --include=*.ts returns one hit — lib/intelligence/pipeline.ts:438 — and that hit is a dead comment, not a writer. Verbatim (pipeline.ts:435-441): “the legacy TS promotion into content_items is retired. runPipeline nudges the worker after the run (D-3).” The if (passed) block is now just result.articlesPassed++ — the ID-75 WP-E retirement shipped. There is no live app/lib TS reference_items writer.

Surface 2 — the `{107.4}` contract (LEDGER subtask, not a docs-site spec)

Confirmed. ID-107 has no specs/id-107* dir; the contract is subtask 4 of Task 107 in the ledger (bun scripts/ledger-cli.ts show task 107, subtask id “4”). Verbatim intent: re-point /api/ingest/url to write reference_items.ingestion_source=url_import, mirroring the cocoindex url_source.py write path; this is the {42.12} re-point; preserve the manual-URL ingest UX (poller/nudge as applicable); coordinate ordering with {107.2} (same file touched by the rename sweep).

Deferral (verbatim, S344): “DEFERRED (parent-ratified Option C, S344) — re-home url_import → reference_items is a blocked cross-lane feature: reference_items needs source_document_id NOT NULL + a PullMD/Docling body the /api/ingest/url route cannot produce; ID-75 url_source.py:59 defers url_import to post-v1 (hardcodes rss_feed); no app-side reference_items writer. url_import stays on content_items.ingestion_source (works post-rename). Filed ID-75 follow-up backlog this session.” (→ bl-298 → ID-110.)

Reconciliation note: The deferral’s three blockers were accurate but framed at the moment before the body-production decision was made. The async body path now exists (§Surface 4); the schema already admits the value (§Surface 4); so the blocker reduces to the single sync-vs-async body-production design question (§4).

Surface 3 — the historic SI web-feed spec (PRE-coco, CONTEXT ONLY)

${KH_PRIVATE_DOCS_DIR}/.../specs/intelligence-workspaces/p0-web-si-web-feed-handler-spec.md. This is the Firecrawl-era SI web-feed handler (predates cocoindex/PullMD). Mined for original intent only:

Original intent (web-feed handler)	Survives in the live ID-75 path?	Obsolete assumption (flag)
Admin creates a `source_type='web'` source; pipeline polls it, extracts via Firecrawl, synthesises a `ParsedFeedItem`, proceeds through dedup/scoring/storage (§1 Outcome).	Partially. The discover→score→gate→ledger half survives as the poller (ID-75 BI-12 keeps it untouched).	The storage half (“promotes to a `content_item`”, §3.4) is OBSOLETE — O4 retired auto-promotion into `content_items`; passed articles now land `reference_items`.
Extraction via Firecrawl `.scrape()` at the poll layer (§3.2).	Superseded. PullMD (HTML) / Docling (PDF) at the cocoindex landing tier are now the body producers.	Firecrawl-as-body-producer is OBSOLETE for the reference body; Firecrawl/Turndown survives only as a scoring-input extraction tier (the `{42.11}` re-point).
URL dedup by `feed_articles.external_url` (normalised), content-hash dedup at `content_items` (§3.4, D4).	URL-dedup survives (`feed_articles UNIQUE (workspace_id, external_url)`; reference dedup is now `reference_items UNIQUE (source_url)`).	The content_items content-hash dedup as the second layer is OBSOLETE for references — the `ri:`/`sd:` uuid5 identity + `UNIQUE(source_url)` is the new idempotency key.
”Monitor-only after first ingest” semantic; re-poll discards unchanged pages (§3.4).	Survives in spirit — the reference layer is update-in-place under a stable PK (BI-2).	The Firecrawl-credit-cost optimisation framing is OBSOLETE (different fetcher).
`validateWebUrl()` SSRF-ish gate (§3.5).	Survives — SSRF validation is a live concern (ID-75 ported `url-validation.ts` → `url_validation.py`).	The web-feed-specific gate shape is superseded by the shared `validateUrl`.

Net: Surface 3 confirms the discovery/scoring/SSRF intent that ID-75 preserved, and its storage-into-content_items conclusion is dead under O4. It contributes no live design to ID-110; it is useful only to confirm we are not re-litigating settled ground.

Surface 4 — the LIVE ID-75 reference path (the thing url_import folds into)

ID-75 is CLOSED (18/18 subtasks, S323) and REOPENABLE. The live artefacts:

Schema — reference_items (supabase/migrations/20260606121451_id75_reference_items_layer.sql):

id uuid PRIMARY KEY — PIPELINE-MINTED uuid5('ri:'+normalised URL), no DEFAULT (:5-6).
body text NOT NULL — “PullMD/Docling markdown: the canonical body of record” (:8).
source_url text NOT NULL; CONSTRAINT reference_items_source_url_key UNIQUE (source_url) — one reference per URL (:7, :23).
source_document_id uuid NOT NULL REFERENCES public.source_documents(id) ON DELETE RESTRICT (:16-17) — the body problem in schema form.
ingestion_source text NOT NULL CHECK (ingestion_source IN ('rss_feed','url_import')) (:18-19) — url_import ALREADY ADMITTED; no migration needed to permit the value.
embedding vector(1024) NULL, published_at, primary_domain, primary_subtopic, layer (validated, v1 constant 'research'), op_id.
NO workspace FK, NO junction table — RATIFIED-DO-NOT-BUILD (:25).
RLS: SELECT-only for authenticated; no app-side write policies — “writes are pipeline-only via the asyncpg owner connection (BI-16)” (:38-46).

RPCs (20260606130224_id75_reference_search_rpcs.sql): two-step reference_search(query, embedding, limit) + reference_get_verbatim(reference_id).

Async writer — _ingest_url_body (scripts/cocoindex_pipeline/flow.py:2626-2801), mounted via ingest_url (:2546-2623) over FeedUrlSource.items(). Steps:

SSRF gate validate_url(item.url) — reject ⇒ structured log cocoindex.url_ssrf_rejected + ingestion_quality_log row + ZERO rows (:2657-2676).
PDF sniff (_url_is_pdf) ⇒ _fetch_url_bytes → _docling_to_markdown, extraction_method='docling'; else _pullmd_fetch(url, content_epoch) (epoch-keyed memo), extraction_method=pullmd_<x_source>, pullmd_share_id captured (:2678-2697).
extract_classification(markdown) (content_type discarded) + embed_content_text (:2701-2706).
Mint sd_id = uuid5(NS, "sd:"+url), ri_id = uuid5(NS, "ri:"+url); "ci:" never minted from a URL (:2709-2712).
sd_target.declare_row — storage_path = source_url = normalised URL, URL-derived filename/mime_type/file_size, content_hash, extraction_method, pullmd_share_id (:2714-2730).
ri_target.declare_row — full BI-3 contract; ingestion_source = item.ingestion_source (:2732-2760).
Backlink UPDATE feed_articles SET reference_item_id=$1 WHERE external_url=ANY($2) AND passed (:2762-2801; FK-deferral tolerance on walk 1).

Source — FeedUrlSource (scripts/cocoindex_pipeline/url_source.py): enumerates feed_articles WHERE passed=true only, groups N workspace rows → 1 UrlItem per normalised URL. Crucially, INGESTION_SOURCE_RSS_FEED = "rss_feed" is hardcoded (:60, :96, :166) with the explicit comment (:58-59): “every enumerated URL arrives via the RSS ledger. url_import joins when a manual-URL route ships (post-v1).” — i.e. the hardcode is a route-scoping decision, not an architectural exclusion.

Body production precedent on the app side (load-bearing for Option A): app/api/upload/route.ts:411-430 DOES create a source_documents row from the app side (file uploads): .from('source_documents').insert({ storage_path, ... }) then updates it with extracted text at :615-624. So an app-side source_documents writer pattern already exists — the {107.4} “the route cannot produce one” was about the URL route’s current wiring, not an architectural impossibility.

SSRF/validation parity already shipped: ID-75 ported lib/extraction/url-validation.ts → scripts/cocoindex_pipeline/url_validation.py:73 validate_url(...), and normaliseUrl → scripts/cocoindex_pipeline/url_normalise.py normalise_url(...), guarded by a shared parity fixture (TECH §D-8). ID-110 reuses both directly.

3. Conflict / overlap map

Each surface mapped against the dimensions that decide the re-point. “the route” = /api/ingest/url today; “ID-75 ref path” = the live cocoindex _ingest_url_body.

Dimension	S1: `url_import` route (TODAY)	S4: ID-75 ref path (RSS, LIVE)	S2: `{107.4}` target	S3: SI web-feed (historic)
Target table	`content_items` (`route.ts:207`)	`reference_items` + `source_documents`	`reference_items` + `source_documents`	`content_items` (OBSOLETE)
Who produces the body	`extractFromUrl` (Readability/etc.) — body lives in `content_items.content`, no `source_documents` row	PullMD (HTML) / Docling (PDF) → `source_documents.body`-of-record + `ri.body`	(unresolved — the deferral)	Firecrawl `.scrape()` (OBSOLETE)
`ingestion_source` value	`'url_import'`	`'rss_feed'` (`url_source.py:60`)	`'url_import'` (CHECK already admits)	n/a (content_items)
Sync vs async	Synchronous (single HTTP request, ≤60s `maxDuration`)	Asynchronous (poller-nudged `/walk`, snapshot enumeration)	(the mismatch IS the deferral)	Synchronous poll, async pipeline
Idempotency / dedup key	`content_items.source_url` soft check + content-hash dedup	`ri:`/`sd:` uuid5(normalised URL) + `UNIQUE(source_url)`	uuid5(normalised URL)	`feed_articles.external_url` + content-hash
RLS / writer	App route writes via the authed user’s client	Pipeline-only (asyncpg owner conn; no app write policy on `reference_items`)	(must reconcile: app can’t write `reference_items` under current RLS)	App route writes content_items
Workspace scope	`content_items` IS workspace-associable (junction)	`reference_items` is global, no workspace FK (BI-7)	global (inherits ref-layer)	content_items (workspace)
Entry trigger	User pastes one URL on the item page	Relevance-gated feed article passes	One user-supplied URL	Admin creates a web source

Where they CONFLICT (the four real tensions):

C1 — Sync vs async (the central conflict). The route returns suggestions synchronously to the user; the ID-75 path is fire-and-forget async enumeration. A manual URL has no feed_articles ledger row and no relevance gate. Folding requires either (a) the route writes reference_items itself synchronously, or (b) the route seeds something the async path can enumerate.
C2 — RLS writer model. reference_items has no app-side write policy — writes are pipeline-only via the asyncpg owner connection. An app route writing reference_items directly either needs a service-role/owner path or a new write policy (a deliberate posture change ID-75 explicitly avoided).
C3 — Body of record. The route’s extractFromUrl produces text into content_items.content; it does not create the source_documents provenance row (source_document_id NOT NULL) nor a PullMD pullmd_share_id. Either the route gains a source_documents-writing step (upload-route precedent) or it defers body production to the PullMD path.
C4 — Synchronous UX expectations. The route returns suggested_layer, topic_suggestion, guide_section_suggestions, dedup_status, warnings to the caller immediately (route.ts:472-492). The reference layer has no per-row layer/topic/guide suggestion UX today (layer is the constant 'research'; no topic inference on references). Re-homing changes what the user sees back.

Where they OVERLAP (the reuse surface):

O1 — Identity & idempotency. Both want one row per normalised URL. The ref path’s uuid5('ri:'+normalise(url)) + UNIQUE(source_url) is directly reusable; the route’s existing URL-exists check (route.ts:74-90) maps onto it.
O2 — SSRF + normalisation. validateUrl (TS) / validate_url (Py) and normaliseUrl / normalise_url are parity-guarded and shared; the route already calls validateUrl (route.ts:68).
O3 — Classification + embedding. Both classify + embed the body; the ref path discards content_type (references carry none) — the route would drop its content_type / layer / topic steps.
O4 — Schema readiness. reference_items.ingestion_source already admits 'url_import' — zero schema change to permit the value (the only possible schema work is a new app-write RLS policy under Option A, see §4).

4. The body problem (the central blocker) — resolution options

Constraint restated: reference_items.source_document_id is NOT NULL REFERENCES source_documents(id) ON DELETE RESTRICT (migration :16-17). A reference row cannot exist without a source_documents row carrying the canonical body. The manual-URL route today produces neither a source_documents row nor a PullMD/Docling body — it puts Readability text into content_items.content.

Four resolution options, with feasibility / ID-75 reuse / back-compat / latency-UX / SSRF assessment each.

Option A — App route synchronously writes `sd` + `ri` itself (PullMD in-request)

The route calls PullMD (or reuses its existing extractFromUrl body) synchronously, mints sd:/ri: uuid5, and writes both rows from TS.

Feasibility: Moderate. The upload route precedent (app/api/upload/route.ts:411-430) proves an app-side source_documents writer is viable. But it duplicates the Python landing logic (uuid5 seeds, body-of-record contract, extraction_method semantics) in TS — a parity-drift hazard mirroring the normaliseUrl seam ID-75 had to fixture-guard.
ID-75 reuse: Schema (no change), ingestion_source value, identity scheme — reused. RPCs unaffected. But the writer is re-implemented, not reused (Python vs TS).
Back-compat: Clean — new rows land references; legacy content_items untouched.
Latency / UX: PullMD in-request adds a network round-trip inside the ≤60s window; acceptable for a single URL but slower than today’s Readability extraction. UX can still return synchronously.
RLS / SSRF: Requires resolving C2 — either a service-role/owner write path from TS (the route currently writes as the authed user) or a new reference_items INSERT policy (a posture change ID-75 deliberately avoided). SSRF reuse: validateUrl already in the route.
Verdict: Viable but re-opens the pipeline-only-writer posture and duplicates landing logic. Highest UX fidelity, highest architectural cost.

Option B — App route seeds a ledger-equivalent row; the existing cocoindex path mints `sd`+`ri` async

The route writes a feed_articles-equivalent row (or a minimal manual-URL ledger row) with passed=true + ingestion_source='url_import' provenance, then the existing FeedUrlSource enumeration (relaxed to admit manual rows) lands the reference asynchronously — the route nudges /walk.

Feasibility: High for the body production (zero new writer — _ingest_url_body already does everything). But it forces a synchronous UX into an async model: the user pastes a URL and the reference appears seconds-to-minutes later, not in the response. Requires url_source.py to emit ingestion_source='url_import' for manual rows (today hardcoded rss_feed at :60) — exactly the post-v1 join its own comment anticipates.
ID-75 reuse: Maximal — the entire landing path, identity, RPCs, SSRF, PDF route, failure isolation reused unchanged; only the enumeration source widens to include manual rows and stamp the route’s value.
Back-compat: Clean. Reuses the proven FK-deferral / idempotency machinery.
Latency / UX: Async — breaks the route’s synchronous suggestion contract (C4). Needs a UX rethink: “your URL is being processed” + a poll/nudge, or a redesigned reference-detail view. This is the biggest product cost.
RLS / SSRF: No posture change — writes stay pipeline-only (the asyncpg owner conn). SSRF runs in _ingest_url_body step 1 (already live). C2 dissolved.
Open sub-question: does a manual URL get a feed_articles row (it has no feed_source_id, no workspace necessarily), or a new lightweight manual-ingest ledger table the source also enumerates? feed_articles requires workspace_id + feed_source_id semantics that a manual paste lacks cleanly.

Option C — Hybrid: synchronous body fetch + async/owner write

Route fetches the body synchronously (immediate validation/quality feedback to the user), then hands a fully-formed payload to a small owner-connection writer (a thin TS service-role path or a dedicated RPC) that performs the sd+ri declare with the ID-75 identity contract — so the user gets immediate “accepted + here’s the title/summary” feedback while the canonical write uses the pipeline-aligned path.

Feasibility: Moderate. Best UX/architecture balance but the most moving parts — needs a new owner-write seam (a reference_ingest RPC mirroring the declare_row contract, or a service-role TS writer). A SECURITY DEFINER reference_ingest(...) RPC is the cleanest reconciliation of C2 (keeps the table write-policy-free; the function is the gated writer, mirroring how the q_a / reference RPCs are the only authenticated surface).
ID-75 reuse: High for schema/identity/RPC-pattern; the writer RPC is net-new but small and mirrors the declare_row field contract.
Back-compat: Clean.
Latency / UX: Synchronous accept + immediate metadata; preserves most of C4. PullMD in-request latency as Option A.
RLS / SSRF: A SECURITY DEFINER RPC is the idiomatic KH reconciliation of C2 (matches reference_search/reference_get_verbatim already being the only authenticated reference surface). SSRF in the route + defence-in-depth in the RPC/writer.
Verdict: Likely the recommended balance — see §5.

Option D — Keep `url_import` on `content_items`; define `adopted_from_reference` promotion instead

Leave the manual-URL route writing content_items (it IS a user deliberately adding a URL to their KB — arguably an adoption act, not passive feed evidence), and instead formalise the adopted_from_reference promotion edge for it.

Feasibility: High (near-zero work — keep status quo + clarify semantics).
ID-75 reuse: None needed.
Back-compat: Trivially clean.
Latency / UX: Unchanged.
Semantic tension: This contradicts the ratified O4 frame that ID-75 PRODUCT BI-9 / BI-23 explicitly extends to the {42.12} manual-URL surface (“at which moment that surface inherits BI-9/BI-11 — lands a reference row, never a content_items row”). It is defensible ONLY if Liam reframes manual-URL paste as an adoption act distinct from feed evidence — a genuine product question (OQ-3 below). But note: a manual paste is a raw external URL with no authored derivation, so under the strict O4 definition it is evidence, not adopted knowledge. adopted_from_reference is reserved for derived records that cite a reference, not for the raw landing.
Verdict: The honest “do nothing structural” option; included for completeness. Recommended only if Liam rejects the O4 extension to manual URLs.

Body-problem summary

Option	Body producer	Writer	C2 (RLS)	C4 (sync UX)	ID-75 reuse	New work
A	PullMD in-request (TS)	TS app, owner/service-role	re-opens	preserved	schema/identity	TS writer + RLS posture
B	PullMD async (Python)	existing `_ingest_url_body`	unchanged	broken	maximal	enumeration widen + UX rethink
C	PullMD in-request	SECURITY DEFINER RPC	clean (RPC-gated)	mostly preserved	high	small `reference_ingest` RPC
D	Readability (status quo)	TS app (content_items)	n/a	preserved	none	semantic reframe only

5. Recommended target design

Recommendation: Option C (hybrid synchronous-accept + owner-gated reference write via a SECURITY DEFINER reference_ingest RPC), folding into the ID-75 reference contract.

Rationale against the four surfaces:

It honours the ratified O4 frame (S1’s content_items write is the defect; S4’s reference contract is the destination; ID-75 BI-9/BI-23 already scoped this exact re-point) — so it is the faithful execution of {42.12}, not a new direction.
It dissolves C2 the KH-idiomatic way: a SECURITY DEFINER RPC is already the only authenticated surface over reference_items (the two search RPCs). A reference_ingest(url, ...) RPC (or a thin service-role TS writer) keeps the table write-policy-free — preserving the ID-75 pipeline-only-writer posture — while giving the synchronous app route a gated entry point.
It preserves the synchronous UX (C4): the user gets immediate accept + title/summary; layer is the reference constant 'research'; topic/guide suggestions are dropped (they are content_items affordances, not reference affordances — a deliberate, documented UX reduction).
It reuses ID-75’s identity scheme, schema (no DDL beyond possibly the RPC), SSRF/normalise parity, and the ingestion_source='url_import' value the CHECK already admits.

If the synchronous-write parity-duplication is judged too costly, fall back to Option B (maximal reuse, accept the async-UX rework) — but only after Liam rules on the UX change (OQ-2). Option A is the fallback if an RPC writer is undesirable. Option D only if Liam rejects the O4 extension to manual URLs (OQ-3).

Fold-into-ID-75 (reopen) vs fresh ID-110 contract: Author a fresh ID-110 PRODUCT+TECH contract that EXTENDS the ratified ID-75 reference layer — do NOT reopen ID-75. Grounds: ID-75 is CLOSED with 18/18 subtasks and a ratified spec; its {42.12} reconcile flag (BI-23) explicitly anticipated this as separately-specced follow-on work (“MUST be verified against this spec when they are specced”). Reopening ID-75 would re-litigate a closed, ratified frame; instead ID-110’s spec cites ID-75 PRODUCT BI-9/BI-11/BI-23 + TECH §6.3 as its binding contract and adds only the manual-URL-specific surface (synchronous entry point, RPC/writer, UX reduction, manual-URL provenance). The reference layer itself is reused as-is.

ingestion_source semantics: stamp 'url_import' on the manual-URL reference row (the CHECK admits it; ID-75 TECH §6.3 reserves it for exactly this route). 'rss_feed' stays the feed route’s value. Manual-URL UX preservation requirement: the route’s existing synchronous accept + returned metadata (title, summary, dedup warning) must survive; topic/guide-section suggestions and the content_type/layer-inference branch are dropped (references carry no content_type; layer is constant). The existing poller/nudge is irrelevant to the manual route (it has no feed ledger) — under Option C the RPC writes directly; under Option B a nudge would be needed.

6. Migration & back-compat

Schema: No migration required to permit the value — reference_items.ingestion_source CHECK IN ('rss_feed','url_import') already admits it. The only possible DDL is Option-C’s reference_ingest SECURITY DEFINER RPC (one migration, mirroring the q_a/reference RPC grant pattern: REVOKE EXECUTE FROM anon, GRANT EXECUTE TO authenticated, service_role, SET search_path = public, extensions).

Legacy content_items url_import rows (~55): Per ID-75 PRODUCT BI-10 + TECH §6.2 the cited counts are 55 url_import + 40 rss_feed legacy content_items rows. ID-75 did not retro-convert them — they are retained read-only as bid-era legacy lineage, and re-landing per O4 is the ID-45/T7 fresh-corpus re-ingest (“near-zero migration cost”, TECH §6.2). Recommendation for ID-110: leave the ~55 legacy rows untouched (consistent with ID-75’s disposition; do not double-handle); ID-110 changes only the forward write path. (Count NOT re-queried — read-only mandate + no worktree DB link; the BI-10/TECH §6.2 figure is the cited source. Flagged as OQ for verification at PRODUCT time if a precise current count matters.)

/api/items Zod enum: Already narrowed in {107.3} — ItemCreateBodySchema.ingestion_source = z.enum(['manual','upload','upload_autosplit']) (lib/validation/schemas.ts:343); url_import already 400s on that surface. The comment block at :336-342 already documents that url_import “is stamped internally by /api/ingest/url (IngestUrlBodySchema, set in code) and is NOT web-form-reachable”. After the re-point, /api/ingest/url no longer stamps content_items.url_import at all — so the IngestUrlBodySchema (which has no ingestion_source field — it is url, content_type, user_tags, skip_dedup, content_owner_id) needs no enum change; the route just stops writing content_items.

Forward closed-set interaction (ID-107): The forward content_items CV-13 set is the 6-value set (manual, upload, upload_autosplit, mcp_create, bid_outcome_integration, + reserved adopted_from_reference); url_import/rss_feed re-home to reference_items per ID-107 {107.1} details + ID-75 TECH §6.3. adopted_from_reference is presently a comment-only reserved value (schemas.ts:340 references it in prose; no enforced enum/CHECK uses it yet — the DB CHECK is DEFERRED to ID-45/T7 per ID-107 description). ID-110 removing the last live content_items.url_import writer makes the re-home complete in code (only legacy rows remain), tightening the case for the eventual CHECK.

layer-inference.ts url_import special-case: lib/layer-inference.ts:197 special-cases url_import, and :21 types ingestionSource to include it. Once the route stops calling inferLayer (references use the 'research' constant), this special-case becomes dead for the live path — ID-110 should either remove the url_import branch from inferLayer or document it as legacy-only (low-priority cleanup; flag for the follow-on TECH).

7. Open questions for Liam

Reopen ID-75 vs fresh ID-110 contract? Options: (a) reopen closed ID-75 and add manual-URL subtasks; (b) fresh ID-110 PRODUCT+TECH that cites/extends the ratified ID-75 contract. Recommended default: (b) — ID-75’s BI-23 explicitly anticipated {42.12} as separately-specced follow-on; reopening re-litigates a ratified closed frame. ID-110 binds to ID-75 BI-9/BI-11/BI-23 + TECH §6.3 without modifying them.
Body problem: synchronous app-write vs async feed-seed? Options: (a) Option C — synchronous accept + owner-gated reference_ingest SECURITY DEFINER RPC (preserves sync UX, dissolves RLS conflict, reuses identity/schema); (b) Option B — seed a ledger row, the existing cocoindex path lands it async (maximal reuse, but breaks the synchronous suggestion UX); (c) Option A — TS app writes sd+ri directly (re-opens pipeline-only-writer posture). Recommended default: (a) Option C. If RPC-writer is undesirable, (b) with an explicit UX rework decision; (c) as last resort.
Is manual-URL paste “evidence” (→ reference) or “adoption” (→ content_items)? The O4 frame says a raw external URL is evidence (→ reference_items); ID-75 BI-9/BI-23 extends this to the manual route. But one could argue a user deliberately adding a URL to their KB is an adoption act. Recommended default: evidence → reference_items (faithful to ratified O4; a raw URL has no authored derivation, so adopted_from_reference — reserved for derived records that cite a reference — does not fit a raw landing). Confirming this closes Option D.
Fate of the ~55 legacy content_items url_import rows? Options: (a) leave read-only legacy lineage, re-land at ID-45/T7 (consistent with ID-75 BI-10); (b) retro-convert now. Recommended default: (a) leave them — matches ID-75’s disposition; avoids double-handling; ID-45/T7 owns fresh-corpus re-landing.
Workspace-scoping of the manual-URL reference. reference_items has no workspace column by ID-75 design (BI-7, RATIFIED-DO-NOT-BUILD; workspace relevance is retrieval-time). A manual paste happens in a user/workspace context, raising whether manual references need workspace attribution the feed route doesn’t. Recommended default: NO workspace column — honour BI-7; capture the originating user/workspace as provenance only (e.g. op_id / an audit field, mirroring how the feed route keeps workspace_ids as attribution-only and never writes them to the row). Adding a workspace FK to reference_items would breach a ratified anti-pattern and should be a separate deliberate decision, not a side-effect of ID-110.
(Sub-question of OQ-2, surface at PRODUCT time) If Option B is chosen: does a manual URL get a feed_articles row (which needs workspace_id + feed_source_id it lacks cleanly) or a new lightweight manual-ingest ledger the source also enumerates? Recommended default: a small manual-ingest ledger table rather than overloading feed_articles — only relevant if OQ-2 lands on Option B.

8. Dependency / scope map

Verdict on tier: A follow-on PRODUCT+TECH (the TECH+PLAN or full chain tier) is warranted — the change is behaviourally meaningful (re-point, UX reduction, RLS reconciliation) and touches a migration (if Option C) + a route rewrite + the Python source (if Option B). Estimated effort: M (~3-5h) assuming Option C; S-M if Option D (reframe-only).

ID-75 artefacts cited (not reopened):

PRODUCT.md BI-9 (acquisition-route extensibility), BI-11 (legacy retirement obligation inherited by the manual route), BI-23 ({42.12} reconcile flag) — the binding contract.
TECH.md §6.3 (CV-13 re-homing; url_import reserved for the manual route), §WP-A (schema), §WP-B (RPC pattern to mirror for reference_ingest).
Migration 20260606121451_id75_reference_items_layer.sql (schema, CHECK), 20260606130224_id75_reference_search_rpcs.sql (RPC grant pattern).

Likely implementation subtasks a {110.4} PLAN would produce (illustrative, Option C):

(If Option C) Migration: reference_ingest SECURITY DEFINER RPC mirroring the declare_row field contract + q_a/reference grant pattern; staging-first; types regen.
Rewrite app/api/ingest/url/route.ts: drop content_items insert + content_type/layer/ topic/guide steps; add synchronous body fetch (PullMD or reuse extractFromUrl) + SSRF (already present) + reference_ingest call stamping ingestion_source='url_import'; preserve auth, rate-limit, dedup-by-source_url (now against reference_items), and the synchronous accept/metadata response (reduced shape).
Response-shape + caller updates (the item page “add URL” UI): handle the reference-landing response (no layer/topic suggestions); surface the new reference.
lib/layer-inference.ts cleanup: remove/legacy-flag the url_import branch (:21,:197).
Tests: route writes reference_items (not content_items), idempotent re-add (same PK), SSRF rejection, dedup-by-URL, zero content_items.url_import writes remain; behaviour-not- implementation per test-philosophy.md.
(If async / Option B) url_source.py enumeration widening + manual-ledger seeding + nudge.

Cross-Task touchpoints (sibling-only dependency note): ID-110’s subtasks are sibling-only; the ID-75 dependency is a citation of a closed Task’s ratified spec, not a subtask dependency — correctly modelled at the Task level (ID-110 depends-on conceptually on ID-75’s shipped schema, which is already merged). No cross-Task subtask dependency is needed; if PLAN decomposition surfaces one, escalate (split/merge) rather than bend the constraint.

9. Claims I could NOT fully verify (honest unknowns)

Precise current count of legacy content_items.url_import rows. Cited as 55 (ID-75 PRODUCT BI-10 / TECH §6.2). NOT re-queried — read-only mandate + no DB link in the worktree (supabase/.temp/project-ref empty, as expected). Flag for a cheap read-only verification at PRODUCT time if the exact count is load-bearing.
Whether extractFromUrl (lib/extraction/url) output is acceptable as the reference body-of-record, or whether PullMD must be called for parity with the feed path. The feed path’s body-of-record is explicitly the PullMD/Docling markdown (schema comment :8); the route’s Readability text may diverge in quality. Resolving this is part of OQ-2 (Option A/C body producer) — flagged for TECH.
Whether a TS service-role/owner write path or a SECURITY DEFINER RPC is the preferred C2 reconciliation — both are idiomatic; the RPC is recommended but the final mechanism is a TECH decision.
EXECUTOR-VERIFY-type concern: if Option B, whether the engine’s snapshot enumeration cleanly handles a manual-row source alongside the feed source (the ID-75 _LiveDirItems / mount_each machinery is feed-tuned). Flag for TECH if Option B is chosen.

10. Surfaces that differed from the dispatch pointers

lib/intelligence/pipeline.ts reference_items hit is DEAD, as the brief suspected — confirmed: :438 is a comment inside the retired WP-E block (if (passed) is now just result.articlesPassed++). The brief’s “verify whether that hit is live or dead” → DEAD (no writer).
An app-side source_documents writer DOES exist (app/api/upload/route.ts:411-430) — the brief framed “no app-side reference_items writer” (true) but the body-problem analysis benefits from knowing the upload route already writes source_documents from TS, which materially strengthens Option A/C feasibility. This nuances the {107.4} “the route cannot produce one” blocker: it is wiring-absent, not architecturally impossible.
The reference_items CHECK already admits url_import — the brief noted the migration has source_document_id NOT NULL but the ingestion_source CHECK already including url_import (:18-19) means the value-permission half of the deferral blocker was already resolved by ID-75’s own migration. The deferral was about the writer/body, not the value.
url_source.py:60 hardcodes rss_feed with an explicit “url_import joins post-v1” comment (:58-59) — confirming the hardcode is a route-scoping decision ID-75 deliberately left for this Task, not an exclusion.

11. Provenance pointers

Deferral source: ledger Task 107 subtask 4 (bun scripts/ledger-cli.ts show task 107).
Live route: app/api/ingest/url/route.ts (POST :26-499; content_items insert :191,207).
Live reference write path: scripts/cocoindex_pipeline/flow.py:_ingest_url_body (:2626-2801), ingest_url (:2546-2623); source scripts/cocoindex_pipeline/url_source.py.
Schema: supabase/migrations/20260606121451_id75_reference_items_layer.sql; RPCs 20260606130224_id75_reference_search_rpcs.sql.
Binding ratified contract: ID-75 PRODUCT.md (BI-9/BI-11/BI-23), TECH.md (§6.3, OQ-T1).
Historic context (obsolete storage half): SI web-feed spec specs/intelligence-workspaces/p0-web-si-web-feed-handler-spec.md.
App-side source_documents writer precedent: app/api/upload/route.ts:411-430,615-624.
Items Zod enum (post-{107.3}): lib/validation/schemas.ts:343; ingest body lib/validation/ingest-schemas.ts:9-27; layer special-case lib/layer-inference.ts:21,197.

Changelog

Date	Session	Note
13/06/2026	S-current	`{110.1}` RESEARCH authored (fresh Planner). Four surfaces reconciled with verified file:line/migration/RPC/ledger evidence + cited gitnexus/grep orientation. Conflict/overlap map (C1-C4 conflicts, O1-O4 overlaps). Body-problem options A-D. Recommended Option C (synchronous-accept + owner-gated `reference_ingest` RPC) folding into — NOT reopening — the ratified ID-75 reference layer via a fresh ID-110 contract. Six OQs for Liam. Headline: the `{107.4}` blocker is substantially obsolete — the async body path shipped, the CHECK already admits `url_import`, and ID-75 BI-23 anticipated this exact `{42.12}` re-point.