ID-110 — Re-home url_import onto reference_items: TECH (implementation plan)

ID-110 — Re-home url_import onto reference_items: TECH

{110.3} TECH artefact. Implementation plan for the Option-C re-point of /api/ingest/url (POST) so a pasted URL lands the ID-75 evidence pair (one reference_items row + one source_documents row per normalised URL) via an owner-gated reference_ingest SECURITY DEFINER RPC, and stops writing content_items. Anchored to the ratified {110.2} PRODUCT behaviour (§1–§7) and the binding {110.1} S348 ratification (Option C, fresh ID-110 extending ID-75, manual-URL = evidence, ~55 legacy rows read-only, no workspace column).

• Authored: 13/06/2026, FRESH Planner dispatch (Q-PLANNER-2 — a different instance from the {110.2} PRODUCT author; PRODUCT read in full as input).
• Resolves the three TECH-time Open Questions (OQ-B body producer, OQ-D similarity-dedup retention, OQ-N view-reference destination) — definite decisions, each grounded in code-intel reads, recorded in §0.
• Language: UK English; dates DD/MM/YYYY. No client-specific literals.

---

Context

What is being built

The manual single-URL route app/api/ingest/url/route.ts (POST handler :26-499) TODAY runs an 18-step synchronous pipeline that inserts a content_items row stamped ingestion_source: 'url_import' (insertData :185-208, .from( 'content_items').insert(...) :210-214), then classifies / summarises / scores / infers layer / suggests topic+guide-sections, and returns those affordances (:472-492). Under the ratified ID-75 O4/D4 frame this is definitionally wrong: a pasted external URL is evidence, not adopted knowledge. ID-110 re-points the forward write path to the reference layer; the ~55 legacy content_items url_import rows stay read-only (re-land at ID-45/T7). PRODUCT §1–§7 is the behaviour contract; this spec is the how.

Code-intelligence orientation (cited verbatim — do not paraphrase)

gitnexus_query({query: 'url import reference_items ingest', repo: 'knowledge-hub', goal: 'find the synchronous URL ingest route write path and the reference write contract it must mirror'}) — returned no execution-flow process wiring the route to the reference layer: the top processes are generic auth/response stubs (proc_14_post “POST → CreateClient”, proc_185/186/187_post “POST → UnauthorisedResponse / ForbiddenResponse / ScheduleCleanup”, priority ≤ 0.108), and process_symbols resolved only Function:app/api/ingest/url/route.ts:POST (:26-499) and Function:app/api/items/route.ts:POST (:29-452). The definitions block surfaced the live URL write-path tests (scripts/tests/test_cocoindex_url_write_path.py:TestUrlLandingDeclaresEvidencePair. test_landing_declares_exactly_sd_and_ri_with_field_contract :225-300; test_cocoindex_flow_failure_mode.py:TestUrlPerItemFailureIsolation.* :1868-2166; test_url_source.py:TestCrossWorkspaceCollapse.*) and the E2E spec (e2e/tests/content-ingestion-url.spec.ts). gitnexus orientation: the route→reference re-point flow does not exist yet (confirming PRODUCT’s empty-process finding — this is net-new wiring over an existing route).

gitnexus_context({uid: 'Function:app/api/ingest/url/route.ts:POST'}) — incoming: {} (route entry, no internal callers). outgoing.calls: getAuthorisedClient, authFailureResponse, rateLimitResponse (lib/auth.ts), checkRateLimit (lib/rate-limit.ts), parseBody (lib/validation/index.ts), validateUrl (lib/extraction/url-validation.ts), detectContentType (lib/extraction/content-type-detect.ts), resolveContentOwnerId (lib/auth/owner-default.ts), sb (lib/supabase/safe.ts), safeErrorMessage (lib/error.ts), updateRequestContext (lib/logger/request-context.ts). outgoing.accesses: the local consts suggestedLayer and matches (the content_items-only affordances ID-110 drops, PRODUCT §3.3). The route’s auth + rate-limit + SSRF + parse seam is reused unchanged; the write tail is replaced.

gitnexus_context({name: 'UrlIngestForm', file_path: 'components/create-content/url-ingest-form.tsx'}) — incoming.calls: NewItemTabs (app/item/new/new-item-tabs.tsx, the only caller). outgoing.calls: IngestionSuccessCard (components/create-content/ingestion-success-card.tsx), IngestionProgress, DedupWarning (components/shared/dedup-warning.tsx), ClaudePromptButton, isValidUrl, plus Button/Input/Label. UI blast radius = exactly one form + its success surface (PRODUCT §3.4).

Python/SQL grounding (ast-dataflow is TS-only — grep sweep per .ast-dataflow/ CLAUDE.md):

• grep -rln "reference_ingest" supabase/ scripts/ app/ lib/ → empty — the RPC is net-new.
• grep -rln "reference_items" app/ lib/ --include=*.ts → only lib/intelligence/pipeline.ts (the :438 dead comment inside the retired WP-E block; PRODUCT §7.2) — no live TS reference writer exists.
• find app -path '*reference*' + grep -rln "reference_search|reference_get_verbatim" app/ lib/mcp/tools/ → no reference-detail page, route, or app/MCP TS consumer of the ID-75 reference RPCs (decisive for OQ-N, §0).
• grep "uuid" package.json → uuid is NOT a direct dependency; no TS uuid5 helper exists (decisive for the identity-minting seam, §2.1).
• PullMD is a Python-only httpx client: scripts/cocoindex_pipeline/adapters.py: _pullmd_http_get :169-249 (GET {PULLMD_SERVICE_URL}/api?url=<encoded> with Authorization: Bearer {PULLMD_API_TOKEN}, on-prem Coolify env). No TS PullMD client exists (decisive for OQ-B, §0).

Key existing surfaces (read directly, file:line)

• Route POST app/api/ingest/url/route.ts:26-499: auth ['admin','editor'] :30-31; rate-limit 10/min :39; parseBody(IngestUrlBodySchema) :46; SSRF validateUrl(url) :68; URL-exists soft check against content_items.source_url :74-92; extractFromUrl(url) :95; quality gate <100⇒422, <500⇒warn :98-117; generateEmbedding :131; dedup soft-block :146-176; content_items INSERT :185-218; classify/summarise/score/inferLayer/topic/guide :267-460; response :472-492.
• The async writer contract the RPC mirrors — scripts/cocoindex_pipeline/flow.py:_ingest_url_body :2626-2801. Step 4 mints source_document_id = uuid.uuid5(_KH_PIPELINE_DOC_NS, f"sd:{item.url}"), reference_item_id = uuid.uuid5(_KH_PIPELINE_DOC_NS, f"ri:{item.url}") (:2710-2712); step 5 sd_target.declare_row (storage_path = source_url = item.url, URL-derived filename/mime_type/file_size, content_hash, op_id, extraction_method, pullmd_share_id — :2715-2730); step 6 ri_target.declare_row (full field contract: title, body=markdown, summary, source_url, published_at, primary_domain, primary_subtopic, layer='research', embedding, source_document_id, ingestion_source, op_id; workspace_ids NEVER written — :2733-2760).
• Namespace constant scripts/cocoindex_pipeline/flow.py:1601: _KH_PIPELINE_DOC_NS = uuid.UUID("fbfaf1ff-1ee4-583c-9757-1674465b2ec1").
• Schema supabase/migrations/20260606121451_id75_reference_items_layer.sql: reference_items.id uuid PRIMARY KEY (no DEFAULT — pipeline-minted); body text NOT NULL; source_url text NOT NULL + UNIQUE (source_url); source_document_id uuid NOT NULL REFERENCES source_documents(id) ON DELETE RESTRICT; ingestion_source ... CHECK IN ('rss_feed','url_import') (:18-19 — admits the value, no migration to permit it); layer validated by trg_validate_reference_items_layer; RLS: SELECT-only for authenticated, NO INSERT/UPDATE/DELETE policy (:38-46).
• source_documents columns (20260416102457_pre_squash_reconciliation.sql: 4028-4050): id uuid DEFAULT gen_random_uuid() NOT NULL (overridable by passing an explicit id), NOT NULL filename, original_filename, mime_type, file_size (integer), content_hash, version (default 1), storage_path, status (default 'uploaded', CHECK in uploaded|processing|processed|failed). source_url text NULL added by the ID-75 migration; extraction_method text NULL + CHECK added by ID-42 (20260526074944_id42_pullmd_provenance.sql:21-26): extraction_method IS NULL OR = ANY (ARRAY['pullmd_readability','pullmd_playwright','pullmd_cloudflare', 'pullmd_reddit','pullmd_trafilatura','docling']) — does NOT admit 'readability' / 'unpdf' (load-bearing for OQ-B).
• RPC grant pattern to mirror 20260606130224_id75_reference_search_rpcs.sql: SET search_path = public, extensions; SECURITY DEFINER; ALTER FUNCTION ... OWNER TO postgres; REVOKE EXECUTE ... FROM PUBLIC and FROM anon; GRANT EXECUTE ... TO authenticated, service_role.
• uuid5 availability 20260416102457_pre_squash_reconciliation.sql:69: CREATE EXTENSION IF NOT EXISTS "uuid-ossp" WITH SCHEMA "extensions" ⇒ extensions.uuid_generate_v5(namespace, name) is available server-side.
• App-side source_documents writer precedent app/api/upload/route.ts:411-440 (file uploads insert a source_documents row from TS via the service client).
• TS normaliseUrl lib/intelligence/content-extractor.ts:96; Python port scripts/cocoindex_pipeline/url_normalise.py:normalise_url, parity-guarded by scripts/tests/fixtures/url_normalisation_parity.json against __tests__/validation/url-normalisation-parity.test.ts.
• Dead-code sites lib/layer-inference.ts:21 (ingestionSource union includes 'url_import'), :197 (the url_import special-case) — PRODUCT §7.1.

---

0. Resolved Open Questions (OQ-B / OQ-D / OQ-N) — definite decisions

OQ-B — Body producer: REUSE extractFromUrl in-request; the RPC accepts a pre-fetched body + maps extraction_method to a CHECK-admitted value (NULL when the source is Readability/unpdf). PullMD-in-request is rejected for v1.

Decision: the synchronous route keeps its existing extractFromUrl(url) call (route.ts:95, lib/extraction/url.ts:47 — Readability for HTML, unpdf for PDF) as the body producer. It does NOT call PullMD in-request.

Grounding (why PullMD-in-request is rejected):

1. No TS PullMD client exists. PullMD is a Python-only httpx client (adapters.py:_pullmd_http_get :169-249) that requires PULLMD_SERVICE_URL + PULLMD_API_TOKEN env wired on the IONOS on-prem Coolify host (per ID-42 TECH §WP-A). The Vercel-hosted Next.js route has no such client and no such env; building one is a net-new AGPL-boundary surface (adapters.py:25 “HTML/pullmd AGPL boundary”) well beyond a re-point’s scope. PRODUCT OQ-B’s “PullMD in-request” default explicitly conditioned on “unless TECH measures the latency as unacceptable” — the blocker here is not latency but absence of the client + token surface on the app tier.
2. extractFromUrl already produces a usable body — Readability markdown-ish text (extractionMethod: 'readability') or unpdf text ('unpdf'), with SSRF re-validation after redirects, 15s timeout, 20 MB cap (url.ts:47-127). reference_items.body is NOT NULL; this output satisfies it.
3. source_documents.extraction_method CHECK does not admit 'readability'/'unpdf' (ID-42 enum, §Context). The RPC therefore writes extraction_method = NULL for the manual-URL path (the CHECK is IS NULL OR ..., so NULL is admitted), with the true producer recorded in source_documents.extraction_metadata JSONB (e.g. {"extractor":"readability","via":"app_sync_url_import"}) for provenance without a CHECK violation. Do NOT widen the CHECK enum to add readability/unpdf — that couples the reference provenance vocabulary to the legacy app extractor and is out of scope.

Documented parity caveat (carry into the PR + journal): the feed path’s body of record is PullMD/Docling markdown (schema :8); the manual-URL body is Readability/unpdf text — a quality divergence between feed references and manual references that is accepted for v1. Filed as a follow-on (see §Follow-ups): a future slice may route the manual URL through PullMD once an app-tier client/token seam exists, upgrading manual references to body-of-record parity (the ri:/sd: uuid5 identity is stable, so a re-fetch UPSERTs in place — BI-2 — with no row churn). This caveat is the explicit cost of choosing Option C without a TS PullMD client.

OQ-D — Reference similarity-dedup retention: DROP the similarity soft-warning for references; exact-URL idempotency (PRODUCT §2.1) is the sole dedup contract. The route still returns a dedup_status/warnings shape, but no similarity duplicate_matches.

Decision: keep exact-URL idempotency as the hard contract (the UNIQUE(source_url) constraint + deterministic uuid5 PK); do NOT carry the content-similarity soft-warning (checkForDuplicates/formatDedupWarning, route.ts:146-176) onto the reference path.

Grounding:

1. checkForDuplicates is a content_items-scoped dedup (it queries the adopted- knowledge corpus for near-duplicate bodies). References are global evidence; a manual URL near-duplicating an adopted item is not a meaningful reference-layer collision — the meaningful collision is the same URL, which §2.1 handles deterministically. RESEARCH O3 records the feed path discards content-similarity dedup for references and relies solely on ri:/sd: uuid5 identity + UNIQUE(source_url); the manual route should match the feed path it folds into, not invent a divergent reference dedup UX.
2. Avoids cross-layer leakage. Running the content_items similarity check on a reference ingest would surface adopted-knowledge ids in a reference response — a layer-boundary smell (PRODUCT §3.3 already strips content_items affordances).
3. PRODUCT §2.3 explicitly admits this (“MAY carry duplicate_matches… resolve its retention in {110.3} TECH”) and §3.2 requires only a dedup_status/warnings shape be preserved. The route returns dedup_status: 'clean' and a warnings array (low-content / embedding-failure warnings still apply, §4.4), but no duplicate_matches for the reference landing.

Net: PRODUCT §2.3’s “soft block, not hard reject” is satisfied vacuously (there is no similarity block); the only block is the exact-URL short-circuit of §2.2, which returns url_already_exists rather than erroring. DedupWarning (the UI component) is no longer fed for the URL path (it stays for the upload path).

OQ-N — “view this reference” destination: No reference-detail page exists; the success card surfaces the landed title + summary + a copyable reference id and OMITS the “view item” link. A proper reference-detail destination is filed as a follow-on (out of ID-110 scope).

Decision: the IngestionSuccessCard reference variant shows title, summary, and a copyable referenceId (with a copy-to-clipboard affordance), and renders NO navigation link to a /item/<id> page or any reference-detail page.

Grounding:

1. No reference-detail page or route exists — find app -path '*reference*' returns only the unrelated app/api/notifications/preferences/route.ts; no app/reference* dir; grep for reference_search/reference_get_verbatim consumers in app/ / lib/mcp/tools/ returns nothing (the ID-75 RPCs are unconsumed by any user-facing surface today). Linking “view item” to /item/<reference-id> would 404 — the content_items item page does not exist for this URL (PRODUCT §3.4 forbids it).
2. PRODUCT §3.4 recommended default is exactly this — “surface the landed title/summary + a copyable id and omit a dead ‘view item’ link… promote a proper destination as a follow-on if absent.” This decision confirms the default against the verified absence of any destination.

Follow-on filed (see §Follow-ups): a reference-detail view (page or modal over reference_get_verbatim) so manual references are navigable. Out of ID-110 scope — ID-110 must not 404 and must not ship a dead link.

---

Proposed changes

The work is four code slices + one migration, mapped one-to-one against PRODUCT’s numbered invariants in §Testing. Sequencing: migration first (staging-first, types regen), then route, then UI, then dead-code cleanup.

1. Migration — reference_ingest SECURITY DEFINER RPC (PRODUCT §1.1–§1.5, §4.6, §5.1–§5.3)

New migration supabase migration new id110_reference_ingest_rpc (do NOT use MCP execute_sql; supabase db push foreground per CLAUDE.md). The RPC is the single owner-gated write seam that lands the sd+ri pair atomically, keeping reference_items write-policy-free (§5.1).

Signature & contract:

SET search_path = public, extensions;

CREATE OR REPLACE FUNCTION public.reference_ingest(
  p_source_url        text,           -- caller passes the ALREADY-normalised URL
  p_title             text,
  p_body              text,           -- extractFromUrl output (NOT NULL on ri)
  p_summary           text,           -- nullable
  p_primary_domain    text,           -- nullable (classifier output)
  p_primary_subtopic  text,           -- nullable
  p_embedding         vector(1024),   -- nullable; caller passes JSON.stringify(array)
  p_published_at      timestamptz,    -- nullable; original pub time, never ingest time
  p_filename          text,
  p_mime_type         text,
  p_file_size         integer,
  p_content_hash      text,
  p_extraction_metadata jsonb DEFAULT '{}'::jsonb,  -- {"extractor":"readability",...} (OQ-B)
  p_op_id             uuid DEFAULT NULL
)
RETURNS TABLE (
  reference_id   uuid,
  source_document_id uuid,
  title          text,
  summary        text,
  source_url     text,
  primary_domain text,
  primary_subtopic text,
  already_existed boolean
)
LANGUAGE plpgsql
VOLATILE                              -- writes (contrast STABLE search RPCs)
SECURITY DEFINER
SET search_path = public, extensions
AS $$
DECLARE
  v_sd_id uuid := extensions.uuid_generate_v5(
    'fbfaf1ff-1ee4-583c-9757-1674465b2ec1'::uuid, 'sd:' || p_source_url);
  v_ri_id uuid := extensions.uuid_generate_v5(
    'fbfaf1ff-1ee4-583c-9757-1674465b2ec1'::uuid, 'ri:' || p_source_url);
  v_existing uuid;
BEGIN
  -- §2.1/§2.2 idempotency: if the ri already exists, return it (already_existed=true),
  -- write nothing. Deterministic PK + UNIQUE(source_url) make this a no-op converge.
  SELECT id INTO v_existing FROM public.reference_items WHERE id = v_ri_id;
  IF v_existing IS NOT NULL THEN
    RETURN QUERY SELECT ri.id, ri.source_document_id, ri.title, ri.summary,
      ri.source_url, ri.primary_domain, ri.primary_subtopic, true
      FROM public.reference_items ri WHERE ri.id = v_ri_id;
    RETURN;
  END IF;

  -- §4.6 atomicity: both rows land in one function invocation (one implicit txn);
  -- sd first (FK target), then ri. A failure on either raises and rolls back both.
  INSERT INTO public.source_documents (
    id, filename, original_filename, mime_type, file_size, content_hash,
    storage_path, source_url, status, extraction_method, extraction_metadata, op_id)
  VALUES (
    v_sd_id, p_filename, p_filename, p_mime_type, p_file_size, p_content_hash,
    p_source_url, p_source_url, 'processed', NULL,  -- extraction_method NULL (OQ-B)
    p_extraction_metadata, p_op_id)
  ON CONFLICT (id) DO NOTHING;             -- belt-and-braces idempotency on sd

  INSERT INTO public.reference_items (
    id, title, body, summary, source_url, published_at, primary_domain,
    primary_subtopic, layer, embedding, source_document_id, ingestion_source, op_id)
  VALUES (
    v_ri_id, p_title, p_body, p_summary, p_source_url, p_published_at, p_primary_domain,
    p_primary_subtopic, 'research', p_embedding, v_sd_id, 'url_import', p_op_id)
  ON CONFLICT (id) DO NOTHING;

  RETURN QUERY SELECT ri.id, ri.source_document_id, ri.title, ri.summary,
    ri.source_url, ri.primary_domain, ri.primary_subtopic, false
    FROM public.reference_items ri WHERE ri.id = v_ri_id;
END;
$$;

ALTER FUNCTION public.reference_ingest(text,text,text,text,text,text,vector,
  timestamptz,text,text,integer,text,jsonb,uuid) OWNER TO postgres;
REVOKE EXECUTE ON FUNCTION public.reference_ingest(...) FROM PUBLIC;   -- full sig
REVOKE EXECUTE ON FUNCTION public.reference_ingest(...) FROM anon;
GRANT  EXECUTE ON FUNCTION public.reference_ingest(...) TO authenticated, service_role;

Design decisions (REJECTED-alternatives live here per ID-34):

• Caller passes a pre-normalised p_source_url; the RPC re-derives the uuid5 PKs server-side via extensions.uuid_generate_v5. This keeps identity-minting in ONE place (the DB), avoids adding the uuid npm dependency (it is not a direct dep — §Context), and dissolves the TS↔Python uuid5 parity-drift hazard RESEARCH §4 Option-A flagged. The route MUST call normaliseUrl (content-extractor.ts:96) before passing the URL, so the same normalisation the feed path applies (normalise_url) gates the PK. uuid5 parity is empirically confirmed: Postgres uuid_generate_v5 and Python uuid.uuid5 both implement RFC-4122 v5 (SHA-1) and produce identical output for identical namespace+name (verified 13/06/2026 — uuid5(ns,'ri:https://example.com/a') = ac261849-c4e5-5a28-970b-4a063146ad2a from Python; the Postgres function uses the same algorithm). The Executor MUST add a parity assertion (a test that the RPC’s minted id for a fixed URL equals the Python uuid5 for the same ri:+url) so a future Postgres uuid impl drift is caught.
• REJECTED — a new reference_items INSERT RLS policy (RESEARCH Option A): would re-open the ID-75 pipeline-only-writer posture (§5.1, BI-16). The SECURITY DEFINER RPC is the KH-idiomatic reconciliation (mirrors reference_search/reference_get_verbatim being the only authenticated reference surface) and keeps the table policy-free.
• REJECTED — a thin service-role TS writer (RESEARCH Option A variant): duplicates the declare_row field contract in TS and re-implements identity-minting; the RPC centralises both. (The upload-route source_documents TS writer precedent at :411-440 proves it is possible, but it does not write the ri half nor the deterministic PK.)
• REJECTED — async feed-seed (RESEARCH Option B): breaks the synchronous accept UX (PRODUCT §3.1) and needs an enumeration widen of url_source.py (§5.3 forbids touching it). Out of scope.
• status = 'processed' on the source_documents row (not 'uploaded'/'processing') because the body is already extracted synchronously — no async processing follows. CHECK admits it.
• ON CONFLICT (id) DO NOTHING on both inserts is belt-and-braces against a race between the pre-check SELECT and the INSERT (two concurrent identical-URL requests); the deterministic PK makes the loser a no-op rather than a unique-violation 500.
• Atomicity (§4.6): a PL/pgSQL function body runs in the caller’s transaction; an exception on the ri insert rolls back the sd insert — no orphaned provenance row. The FK source_document_id NOT NULL REFERENCES ... ON DELETE RESTRICT is satisfied because the sd insert precedes the ri insert in the same txn.

No schema change to reference_items/source_documents columns — the CHECK already admits url_import (§Context); only the RPC is net-new DDL. Types regen after db push (supabase/CLAUDE.md) so Database['public']['Functions']['reference_ingest'] is typed for the route call.

2. Route rewrite — app/api/ingest/url/route.ts (PRODUCT §1, §2, §3, §4)

Replace the write tail (steps 5, 10–18) while keeping the front matter (auth :30-31, rate-limit :39, parse :46, SSRF :68) byte-for-byte:

1. §2.2 URL-exists check — re-point :74-92 from content_items.source_url to the reference layer. Compute const normalised = normaliseUrl(url) (import from @/lib/intelligence/content-extractor), then sb(supabase.from('reference_items') .select('id, title').eq('source_url', normalised).limit(1).maybeSingle(), 'reference_items.byUrl'). If present, return { url_already_exists: true, existing_item: { id, title } } (the UI’s existing branch, url-ingest-form.tsx:152-157, is unchanged). (reference_items has no archived_at; drop the .is('archived_at', null) filter.)
2. §1.4/§1.5 body + provenance fields — keep extractFromUrl(url) :95 (OQ-B) and the quality gate :98-117 (§4.4). Derive filename (last path segment or host), mime_type (application/pdf when extractionMethod === 'unpdf', else text/html), file_size (Buffer.byteLength(extracted.content) — integer), content_hash (crypto.createHash('sha256').update(extracted.content).digest('hex'), node:crypto), extraction_metadata = { extractor: extracted.extractionMethod, via: 'app_sync_url_import', ...(extracted.pageCount && { page_count }) }.
3. §3.2 classify + embed — keep generateEmbedding :131 for the ri.embedding. For primary_domain/primary_subtopic, call the existing classifyContent producing the classification values to PASS to the RPC (not writing content_items). Simplest: run the classifier’s pure extraction to get domain/subtopic, or pass null for both and let them be backfilled by a future reference-classify pass — decision: pass the classifier output when cheaply available, else null (both columns are nullable on reference_items; the feed path fills them via extract_classification). Drop inferLayer, suggestTopic, suggestGuideSections, date-extraction, quality-score (all content_items-only).
4. §1.1–§1.3 write — call supabase.rpc('reference_ingest', { p_source_url: normalised, p_title, p_body: extracted.content, p_summary, p_primary_domain, p_primary_subtopic, p_embedding: embeddingValue (JSON.stringify), p_published_at, p_filename, p_mime_type, p_file_size, p_content_hash, p_extraction_metadata, p_op_id }) wrapped in sb()/ tryQuery(). The RPC returns { reference_id, source_document_id, ..., already_existed }. summary for the response: PRODUCT §3.2 wants a summary; extractFromUrl gives an excerpt/ogDescription — use extracted.excerpt || extracted.ogDescription || null as p_summary (a full generateSummary pass is content_items-only; references carry the feed-declared summary — for a manual URL the excerpt is the closest equivalent).
5. §3.1/§3.2/§3.3 response — return { id: reference_id, title, source_url: normalised, summary, primary_domain, primary_subtopic, warnings, dedup_status: 'clean' }. Drop content_type, suggested_layer, topic_suggestion, guide_section_suggestions, duplicate_matches (§3.3 + OQ-D). HTTP 2xx synchronous (§3.1).
6. §4.1/§4.3/§4.5/§4.7 — auth gate, SSRF 400, rate-limit, and the catch 500 via safeErrorMessage are unchanged.

maxDuration = 60 stays; the dropped classify/summary/score/topic/guide steps make the synchronous path faster than today, so latency is not a concern (OQ-B note).

3. UI — IngestionSuccessCard + UrlIngestForm (PRODUCT §3.4, §4.2, §4.7, OQ-N)

• components/create-content/ingestion-success-card.tsx — add a reference variant (or prop-gate on a new kind: 'reference'). For a reference landing: render title, summary, domain/subtopic badges, warnings; render a copyable referenceId (copy-to-clipboard via the existing toast import); do NOT render the suggestedLayer Select control (:69-96) or the contentType classification badge or a /item/<id> “view item” link (OQ-N). The Link/ExternalLink “view item” affordance is omitted for the reference variant.
• components/create-content/url-ingest-form.tsx — handleSubmit (:116-196) maps the reduced response onto the success card’s reference variant (id→referenceId, no suggestedLayer/contentType). The existing-item branch (:152-157) and error states (:159-196, §4.7 WCAG — message string + icon, not colour-only) are unchanged.
• DedupWarning (components/shared/dedup-warning.tsx) is no longer fed for the URL path (OQ-D); leave the component for the upload path.
• Semantic-token discipline (components/CLAUDE.md) applies to any copy/affordance change — no raw hex/Tailwind palette literals.

4. Dead-code cleanup — lib/layer-inference.ts (PRODUCT §7.1, §7.2)

Run gitnexus_impact({target: 'inferLayer', direction: 'upstream'}) before editing. Remove the url_import branch (:197) and its presence in the ingestionSource union (:21) — OR, if other live callers still pass url_import (verify via impact), leave the branch and add a // legacy-only — no live caller after ID-110 comment. The §7.2 dead comment at lib/intelligence/pipeline.ts:438 needs no change (documented as not-a-writer). Verification (§7.2): grep -rln "reference_items" app/ lib/ --include=*.ts after the re-point returns only the route/RPC call site plus the dead pipeline.ts:438 comment.

---

Testing and validation

Behaviour-first per test-philosophy.md (assert observable contract, not internals). bun run test (never bun test) for TS; python3 -m pytest scripts/tests/ is untouched (§5.3 — feed path byte-unchanged). Each PRODUCT invariant maps to a concrete check:

PRODUCT invariant	Test / verification
§1.1 one ri + one sd, zero ci	Integration: POST a fresh URL; assert exactly one reference_items row + one source_documents row for normalise(u), and zero content_items with that source_url.
§1.2 ingestion_source='url_import'	Assert landed reference_items.ingestion_source === 'url_import'.
§1.3 deterministic uuid5 PKs + FK	Assert ri.id === uuid5(NS,'ri:'+normalise(u)), sd.id === uuid5(NS,'sd:'+url), ri.source_document_id === sd.id; uuid5 parity assertion (RPC-minted id == Python uuid.uuid5 for the fixture URL); no ci:-seeded row.
§1.4 full ri field contract, no workspace col	Assert non-null title, body, layer==='research'; assert no workspace column written (the table has none — assert the insert payload omits it).
§1.5 sd provenance fields	Assert sd.storage_path === sd.source_url === normalise(u); extraction_method is NULL with extraction_metadata.extractor recorded (OQ-B).
§2.1 exact-URL idempotency	POST the same URL twice; assert one reference_items row; second response non-error with already_existed/url_already_exists.
§2.2 URL-exists re-pointed	Pre-seed a reference_items row; POST that URL; assert url_already_exists: true + existing_item.id == the reference id (not a content_items id).
§2.3 / OQ-D no similarity warning	Assert a near-duplicate (different URL, similar body) still lands a reference row and the response carries no duplicate_matches; dedup_status: 'clean'.
§3.1 synchronous	Assert single POST returns 2xx with id+title in the body (no polling).
§3.2 response fields	Assert non-empty id,title,source_url; id resolves to the landed reference_items row.
§3.3 dropped affordances	Assert response JSON has no suggested_layer/topic_suggestion/guide_section_suggestions/content_type/duplicate_matches keys.
§3.4 / OQ-N success card	Component test: reference variant shows no layer-suggestion control, no content-type badge, no /item/<id> link; shows title/summary + copyable reference id.
§4.1 auth	Non-admin/editor caller → authFailureResponse; zero rows.
§4.2 url-exists UI	Existing-item branch resets to idle, surfaces existing reference (not error).
§4.3 SSRF	SSRF-failing URL → 400, zero rows (existing validateUrl test re-pointed).
§4.4 low-content	URL yielding <100 chars → 422, zero reference rows.
§4.5 rate limit	11th request/min rate-limited.
§4.6 atomicity	Induced mid-write failure (e.g. force the ri insert to raise) leaves neither sd nor ri row — RPC txn rollback test.
§4.7 error display	5xx drives the form to its error state with a visible message (icon + string, not colour-only).
§5.1 no app write policy	Assert reference_items has no INSERT/UPDATE/DELETE policy after ID-110; only reference_ingest writes.
§5.2 RPC grants	Assert anon cannot EXECUTE reference_ingest; authenticated can; SECURITY DEFINER + pinned search_path (migration-revoke-guard.yml lints the anon REVOKE).
§5.3 feed path unchanged	git diff shows url_source.py/flow.py byte-unchanged; Python suite green.
§7.1 inferLayer dead	No live path passes ingestionSource:'url_import' to inferLayer (grep + impact).
§7.2 single TS writer	grep -rln "reference_items" app/ lib/ --include=*.ts → only the new call site + the dead pipeline.ts:438 comment.

bl-119 flag (PRODUCT §6.2): the bl-119 Phase-5 UI test surfaces that ID-110 invalidates and that MUST be re-pointed (not left asserting the dead content_items contract): e2e/tests/content-ingestion-url.spec.ts (asserts the old layer/topic suggestions + content_items landing — its deleteByUrl :127-150 and ingestResponsePromise :179-184 helpers assume the content_items response shape) and any url-ingest-form unit test asserting suggestedLayer/content_type in the response. Authoring the bl-119 suite remains bl-119’s scope; ID-110 must update/skip the E2E spec so CI is green, and flag the rest for bl-119.

---

Risks and mitigations

• uuid5 parity drift (RPC vs Python). Mitigated by the §1 parity assertion + the shared namespace literal fbfaf1ff-1ee4-583c-9757-1674465b2ec1 pinned in the migration with a comment citing flow.py:1601. If a future Postgres uuid-ossp upgrade changes output, the assertion fails loudly.
• extraction_method CHECK violation. Writing 'readability'/'unpdf' would violate the ID-42 CHECK at insert time. Mitigated by writing NULL + recording the producer in extraction_metadata (OQ-B); the migration MUST NOT add those values to the CHECK.
• Normalisation mismatch. If the route passes a non-normalised URL, the PK + UNIQUE(source_url) diverge from the feed path. Mitigated by calling normaliseUrl before the RPC (a parity-guarded function) and passing the normalised value as both the PK seed and source_url.
• Schema parity staging↔prod. Apply the migration staging-first (.env.local targets staging) via supabase db push foreground; schema-parity/migration-revoke-guard side workflows gate the anon REVOKE + drift.
• Silent supabase-call failure. All RPC/select calls go through sb()/tryQuery(); the RPC’s no-silent-failure posture (raise on insert error → route 500 via safeErrorMessage) preserves the §4.7 error contract.
• proxy.ts publicRoutes. No new route is added (the existing /api/ingest/url keeps its auth gate, §4.1); no publicRoutes change needed.

Follow-ups

• PullMD-in-request for manual references (OQ-B caveat). Once an app-tier PullMD client
  • PULLMD_SERVICE_URL/PULLMD_API_TOKEN seam exists, route the manual URL through PullMD for body-of-record parity with the feed path. The stable ri:/sd: uuid5 identity means a re-fetch UPSERTs in place (BI-2). File as backlog.
• Reference-detail destination (OQ-N). Build a reference-detail view (page or modal over reference_get_verbatim) so manual references are navigable; then wire the success card’s “view reference” link. File as backlog.
• bl-119 Phase-5 UI suite — re-point the URL-path UI tests to assert the reference contract (owned by bl-119; ID-110 only flags + green-keeps the E2E spec).
• Reference classification backfill. If the route passes null primary_domain/primary_subtopic for manual URLs, a future reference-classify pass could backfill them (the feed path classifies inline). Low priority.

Changelog

Date	Session	Note
13/06/2026	S-current	{110.3} TECH authored (FRESH Planner, Q-PLANNER-2 — different instance from {110.2}). Resolves OQ-B (reuse extractFromUrl; extraction_method=NULL + extraction_metadata provenance; PullMD-in-request rejected — no TS client; documented body-of-record parity caveat), OQ-D (drop similarity soft-warning; exact-URL idempotency is sole dedup; checkForDuplicates is content_items-scoped), OQ-N (no reference-detail page exists — copyable id, omit dead link; follow-on filed). Net-new reference_ingest SECURITY DEFINER RPC mints ri:/sd: uuid5 server-side via extensions.uuid_generate_v5 (uuid5 parity empirically confirmed RFC-4122 v5 SHA-1), atomic sd+ri insert, ON CONFLICT idempotency, ID-75 grant pattern (REVOKE anon+PUBLIC, GRANT authenticated/service_role). One-to-one invariant→test map (§1.1–§7.2). bl-119 E2E re-point flagged. Code-intel cited verbatim (gitnexus empty-process; POST/UrlIngestForm context; grep sweeps; schema + CHECK + grant reads).