Search — Workflows

Search — Workflows

Last verified: Session 210 (29 April 2026) — refresh covering S196-S198, S205, S207, and S208 (live preview, MCP filter widening, cadence-compliance scorer, saved presets, persona-branched cold-start prompts, from_bid URL stickiness).

Overview

Canonical search has six interlocking workflows: live preview while typing, full hybrid search submission, filter post-filtering, saved-preset apply / save / restore, cold-start persona prompt activation, and MCP section-level chunk search (with review-cadence filtering). Each shares the same underlying RPC pipeline but with different invocation cadences and rate limits.

Workflow 1: Live preview while typing (inline SearchBar)

Trigger: User types into <SearchBar variant="inline"> while focused. Owner: useDebouncedPreview (hook), GET /api/search/preview (route), PreviewSearchSchema (Zod).

Flow

User types → 300ms debounce → ≥3 chars? → AbortController-aware fetch
  → ilike on title+content → server clamp limit ≤ 20 → title-first sort
  → response { results, count } → TanStack Query cache (30s staleTime)
  → SearchBar dropdown re-renders preview section

Detailed Steps

1. Capture keystrokes.

   • File: components/browse/search-bar.tsx
   • Variant gate: isInline && showRecent — preview never fires for hero or compact variants and only while the dropdown is open.
   • Stable empty-array reference returned when disabled (EMPTY_RESULTS: PreviewResult[] = [] module-level constant) to avoid downstream useMemo ref churn.

2. Debounce.

   • File: hooks/browse/use-debounced-preview.ts
   • setTimeout of PREVIEW_DEBOUNCE_MS = 300 from lib/search-history.ts.
   • Sub-3-char queries clear the debounced state via setTimeout(0) so the state update happens outside the effect body (react-hooks/set-state-in-effect).

3. Issue request.

   • URL: GET /api/search/preview?q=<encoded>&limit=8
   • Cache key: queryKeys.search.preview(debouncedQuery) from lib/query/query-keys.ts.
   • signal from TanStack Query’s queryFn cascades to the underlying fetch so query-key changes auto-cancel in-flight requests.

4. Validate + rate-limit (server).

   • File: app/api/search/preview/route.ts
   • Auth: getAuthenticatedClient() → authFailureResponse(auth).
   • Rate limit: checkRateLimit('search-preview:<userId>', 60, 60_000).
   • Validation: parseSearchParams(PreviewSearchSchema, request.nextUrl.searchParams) — q non-empty, limit positive int.

5. Wildcard escape.

   • Function: escapeIlike(raw) (exported from app/api/search/preview/route.ts for unit tests).
   • Escapes \ first, then %, then _. Critical: without this, "50%" matches every row.

6. Run query.

   • supabase.from('content_items').select('id, title, content_type, primary_domain, layer').or(title.ilike.%${escaped}%,content.ilike.%${escaped}%).limit(min(limit,20))
   • Wrapped in sb() (lib/supabase/safe.ts) — fails fast on Postgres error.

7. Sort + map.

   • In-memory: title matches first, then content-only matches.
   • Strip layer from response (selected for forward compat, not exposed today).

8. Return + render.

   • Response shape: { results: { id, title, content_type, primary_domain }[], count }.
   • SearchBar re-renders the dropdown’s preview section between Recent and Popular topics. aria-live="polite" + aria-busy={previewLoading} on the wrapper. Popular topics hidden while preview has data or is loading.

Error Handling

Error Condition	Handling	User Feedback
Empty query / sub-3 chars	Hook gate; no fetch	No spinner, no preview section
Wildcard injection (50%)	escapeIlike() on server	Matches literal % instead of every row
429 rate-limited (60/min)	rateLimitResponse(rl.resetAt)	Stale TanStack cache rendered until window resets; no toast
Network error / 500	TanStack useQuery enters error state	Empty preview section; user can retype
User keeps typing	TanStack auto-aborts in-flight via signal; new request supersedes	Old results disappear; new “Searching…” shows

Database Operations

Operation	Table	Columns	RLS
SELECT	content_items	id, title, content_type, primary_domain, layer, content	All authenticated

---

Workflow 2: Full hybrid search submission

Trigger: User submits a query (Enter key, “See all results”, click on prompt search card, or setSearchQuery(q) from external context). Owner: useSearch / useBrowseData (hooks), POST /api/search (route), hybrid_search (RPC), SearchBodySchema (Zod).

Flow

User submits → addRecentSearch → setSearchQuery → useBrowseData flips to
  Search Mode → POST /api/search → SearchBodySchema validate → rate-limit
  (30/min) → generateEmbedding (OpenAI) → hybrid_search RPC → optional layer
  post-filter → response { results, count } → useBrowseData applies post-filters
  → ContentGrid/ContentList renders

Detailed Steps

1. Submit handler.

   • For inline SearchBar: form onSubmit → addRecentSearch(trimmed) → onSearch(trimmed) callback fires setSearchQuery(query) on BrowseContent.
   • For hero / compact: router.push('/browse?q=<encoded>') — Browse page reads q from useSearchParams().
   • For prompt cards (kind: 'search'): addRecentSearch + onSelectQuery.

2. useBrowseData flips to Search Mode.

   • File: hooks/browse/use-browse-data.ts
   • isSearchMode = Boolean(searchQuery) flips the useInfiniteQuery enabled: false and useQuery enabled: true.
   • Browse Mode pagination is suspended; Search Mode caps at SEARCH_RESULT_LIMIT = 20.

3. Send POST.

   • URL: POST /api/search
   • Body: { query, threshold: 0.35, limit: SEARCH_RESULT_LIMIT }.
   • For useSearch() (drawer surface), an AbortController ref aborts the previous request before each new one.

4. Validate + rate-limit (server).

   • File: app/api/search/route.ts
   • Auth: getAuthenticatedClient() → authFailureResponse(auth).
   • Rate limit: checkRateLimit('search:<userId>', 30, 60_000).
   • Validation: parseBody(SearchBodySchema, raw) — query 1-2000 chars trimmed, threshold 0-1, limit clamped 1-100.

5. Generate embedding.

   • File: lib/ai/embed.ts
   • Singleton OpenAI client.
   • Model: text-embedding-3-large, 1024-dim.
   • Char-truncate at MAX_EMBEDDING_CHARS = 24_000.
   • On failure, return 503 EMBEDDING_FAILED { error, code }.

6. Call hybrid_search RPC.

   • Params: query_embedding: JSON.stringify(embedding) (raw arrays serialise wrong), query_text, similarity_threshold, limit_count.
   • The RPC body bakes in default exclusions: embedding IS NOT NULL, archived_at IS NULL, governance_review_status != 'draft', superseded_by IS NULL (unless include_superseded: true).
   • Scoring formula combines cosine similarity (70%), title / suggested_title / ai_keywords (10-15%), summary / author / 30-day-recency boosts, and a 3% × win-rate multiplier from the polymorphic citations table for items with ≥2 citations.

7. Layer post-filter.

   • In-memory filter on r.layer === layer if layer param present in request body.

8. Apply post-filters in useBrowseData.

   • applyPostFilters(searchResult.data, filters) runs the Browse filter set (Domain, Subtopic, Content Type, Platform, Author, Date range, etc.) against the in-memory result array.

9. Render results.

   • ContentGrid / ContentList renders cards / rows based on viewMode from localStorage.kb-view-mode.
   • verifierIds are resolved via useDisplayNames() for the verified-by badge.

State Transitions

Current State	Event	Next State	Side Effects
Browse Mode (filters)	setSearchQuery(q)	Search Mode	useInfiniteQuery.enabled: false; useQuery.enabled: true
Search Mode	setSearchQuery(undefined)	Browse Mode	TanStack queries swap; URL ?q= removed, filters preserved
Search Mode	clearFilters()	Search Mode (no filters)	URL filter params removed; ?q= retained; from_bid retained
Idle	useSearch().search('')	Idle	mutation.reset() returns { results: [], count: 0 }

Error Handling

Error Condition	Handling	User Feedback
Auth failure (401/403/500)	authFailureResponse(auth)	Standard auth redirect / error
429 rate-limited (30/min)	rateLimitResponse(rl.resetAt)	TanStack error → useSearch surfaces “Search failed”
OpenAI embedding down (503)	EMBEDDING_FAILED code in body	”Search is temporarily unavailable. Please try again shortly.”
RPC error	500 with safeErrorMessage()	Generic “Search query failed” toast
In-flight request superseded	AbortController.abort()	No surface; new request renders

Database Operations

Operation	Function / Table	Columns	RLS / SECURITY
RPC	hybrid_search	embeds, ranks, joins citations/form_responses/form_questions/workspaces for win-boost	SECURITY DEFINER
SELECT	content_items	full row via RPC	implicit via SECDEF

---

Workflow 3: Saved filter preset apply / save / restore

Trigger: User clicks a <PresetBar> chip, “Save” button, or “Manage” button. Owner: useFilterPresets (hook); presets persist in localStorage.kb-filter-presets. No server side.

Flow

User clicks chip → useFilterPresets.applyPreset(presetId) → resolve preset
  → preserve from_bid URL param → router.push('/browse?<params>&from_bid=…')
  → /browse re-reads useSearchParams → useBrowseData refetches with new filters

Detailed Steps

Apply

1. Click chip in <PresetBar>.

   • File: components/browse/preset-bar.tsx
   • Active preset toggle: clicking the active chip calls onClearFilters instead of applyPreset.

2. Resolve preset.

   • useFilterPresets.applyPreset(presetId) looks up by ID across the 5 system presets + user presets sorted by createdAt asc.

3. Preserve from_bid.

   • File: hooks/browse/use-filter-presets.ts:applyPreset
   • from_bid is read from searchParams.get('from_bid') and re-appended to the new URL after composing the preset’s params (SD-5 / risk R-4).

4. Navigate.

   • router.push('/browse?<preset.params>&from_bid=<sticky>').
   • useBrowseData re-reads useBrowseFilters and refetches.

Save

1. Click “Save” in <PresetBar> (visible when canSave === true).
2. <SavePresetDialog> opens.
3. User submits a name (1-40 chars).
4. useFilterPresets.savePreset(name).
   • File: hooks/browse/use-filter-presets.ts
   • Reads the normalised current URL params via normaliseParams() — strips sort, order, cursor, q, sorts the remaining keys.
   • Creates { id: 'u_<crypto-uuid8>', name, params, isSystem: false, createdAt: ISO }.
   • Writes to localStorage.kb-filter-presets (best-effort; silent fail if storage is full or unavailable).

Manage / Rename / Delete / Restore

1. Click “Manage”.
2. <ManagePresetsDialog> lists user presets.
3. Inline rename → useFilterPresets.renamePreset(presetId, newName) — no-op for system presets (presetId.startsWith('system-')).
4. Delete → deletePreset(presetId) — tombstone in dialog state allows undo via restorePreset(preset). System presets immutable.

State Transitions

Current State	Event	Next State	Side Effects
Active preset (matched)	Click matching chip	Cleared	onClearFilters removes filter params (preserves q & from_bid)
Inactive preset	Click chip	Active preset (matched)	applyPreset overwrites filter params
Filters set	Click “Save”	New user preset persisted	localStorage update
Multiple user presets	Click “Manage”	Manage dialog open	Rename / delete actions enabled

Database Operations

None — presets are localStorage-only.

---

Workflow 4: Cold-start persona prompts activation

Trigger: User opens /browse with no q, no filters, not unread-only. Owner: <SearchPromptCards> + shouldShowColdStartPrompts() + usePrimaryFocus + useTopDomains.

Flow

/browse loads → useBrowseData returns items + filters → 5-condition gate
  → SearchPromptCards renders persona-branched 3-card grid → click card
  → discriminated-union dispatch (search / filter / chipComposite)
  → setSearchQuery OR setFilters OR setFilterPanelOpen(true)

Detailed Steps

1. Visibility gate.

   • File: lib/browse-cold-start.ts:shouldShowColdStartPrompts(items, filters, isSearchMode, isLoading, totalCount)
   • Returns true iff: no q, no filters, not unread-only, not loading, totalCount > 0.

2. Persona resolution.

   • File: hooks/use-primary-focus.ts
   • Reads user_metadata.primary_focus from Supabase auth — bid_writing, account_management, or marketing.
   • Viewers always see fallback set regardless of primary_focus.

3. Card data.

   • File: components/browse/search-prompt-cards.tsx
   • Discriminated union per spec §1.20:
     • kind: 'search' — { id, title, description, exampleQuery }
     • kind: 'filter' — { id, title, description, filterPreset: AllowedFilterPreset } (whitelisted keys: domain, content_type, include_qa, source, date_from, freshness, layer)
     • kind: 'chipComposite' — { id, title, description, panelTarget: 'domain', moreLabel }
   • Compile-time enforcement via Pick<BrowseFilters, AllowedPresetKey> + satisfies ReadonlyArray<PromptCard> on each persona’s card array.

4. Top-3 domain chips.

   • File: hooks/browse/use-top-domains.ts
   • 24 h cached count of content_items.primary_domain — top 3 by count.

5. Click dispatch.

   • kind: 'search' → addRecentSearch(exampleQuery) + parent’s onSelectQuery(exampleQuery) → triggers full hybrid search via setSearchQuery.
   • kind: 'filter' → parent’s onApplyFilter(filterPreset) → setFilters(preset) writes URL params (no recent-search write).
   • kind: 'chipComposite' chip click → onApplyFilter({ domain: [chipName] }).
   • kind: 'chipComposite' “More domains…” click → onOpenFilterPanel('domain') → setFilterPanelOpen(true) (no URL mutation).

Error Handling

Error Condition	Handling	User Feedback
primary_focus missing	usePrimaryFocus() returns null → fallback cards	Generic 4-card fallback set
useTopDomains fail	Returns empty array; chip card hides gracefully	”More domains…” link still present

Database Operations

Operation	Function / Table	Columns	RLS
SELECT	content_items (via useTopDomains)	primary_domain, count	All authenticated

---

Workflow 5: MCP section-level chunk search (with review-cadence filtering)

Trigger: Claude / LLM calls the find tool with granularity: "chunk" (the consolidated retrieval tool that absorbed the former standalone search_content_chunks MCP tool under ID-71). Owner: lib/mcp/tools/search.ts (registration) + the search_content_chunks RPC (database — the RPC name is unchanged; only the tool entry-point consolidated). S208 §5.5 Phase 4 widened both the tool params and the RPC signature with review-cadence filters.

Flow

Claude → MCP server → find tool handler (granularity: "chunk") → createMcpClient
  (RLS-scoped) → generateEmbedding → search_content_chunks RPC (with optional
  filter_overdue_review + filter_review_due_within_days) → JOIN content_items
  for cadence predicates → ORDER BY similarity DESC → return ranked chunks
  → formatChunkSearchResults → Markdown + structuredContent JSON

Detailed Steps

1. Receive MCP tool call.

   • File: lib/mcp/tools/search.ts (the find tool’s chunk-granularity branch)
   • Schema (Zod), on the chunk-granularity branch:
     • query: string (required)
     • limit?: number (default 10, max 30)
     • content_item_id?: uuid (scope to one document)
     • overdue_review?: boolean (S208) — true = restrict to review_overdue items, false = exclude them
     • review_due_within_days?: number (1-365) (S208) — items whose next_review_date <= CURRENT_DATE + N days

2. Auth.

   • createMcpClient(extra.authInfo) — per-user Supabase client from OAuth bearer token. RLS applies.

3. Embed query.

   • getGenerateEmbedding() lazy-imports the singleton wrapper.

4. Call RPC.

   • supabase.rpc('search_content_chunks', { query_embedding, similarity_threshold: 0.3, limit_count: limit, filter_content_item_id, filter_overdue_review: args.overdue_review ?? undefined, filter_review_due_within_days: args.review_due_within_days ?? undefined })
   • ?? undefined keeps the JSON-RPC payload free of null values (matches the existing filter_content_item_id convention).

5. RPC executes.

   • File: supabase/migrations/20260428212936_extend_search_content_chunks_review_filters.sql
   • LANGUAGE plpgsql STABLE SECURITY DEFINER, SET search_path = public, extensions.
   • JOIN content_chunks cc to content_items ci (existing JOIN — zero round-trip cost for the new predicates per Option A).
   • Cadence filter predicates AND-compose with existing predicates:
     • filter_overdue_review IS NULL OR (TRUE AND ci.governance_review_status = 'review_overdue') OR (FALSE AND (ci.governance_review_status IS DISTINCT FROM 'review_overdue'))
     • filter_review_due_within_days IS NULL OR (ci.next_review_date IS NOT NULL AND ci.next_review_date <= CURRENT_DATE + (N || ' days')::interval)
   • ORDER BY similarity DESC, LIMIT limit_count.

6. Format.

   • truncateResponse(formatChunkSearchResults(query, chunkResults)) → Markdown for human readers (≤10,000 chars).
   • toStructuredContent({ query, count, content_item_id, overdue_review_filter, review_due_within_days_filter, results }) → structured JSON for downstream tool calls. Filters surface as null when omitted (consistent with content_item_id convention).

State Transitions

None — read-only.

Error Handling

Error Condition	Handling	User Feedback (LLM)
Embedding generation fail	Try/catch in tool handler	”Chunk search failed: . Try simplifying your query.”
RPC error	Tool returns isError: true	Same wording
review_due_within_days < 1 / > 365	Zod validation	LLM gets validation error from MCP protocol

Database Operations

Operation	Function	Cadence-relevant columns	RLS / SECURITY
RPC	search_content_chunks	content_items.governance_review_status, content_items.next_review_date, joined to content_chunks	SECURITY DEFINER

---

Workflow 6: Diagnostic CLI search

Trigger: Operator runs bun run scripts/kb-search.ts "<query>" from terminal. Owner: scripts/kb-search.ts (standalone Bun CLI; not wired to the app runtime).

Flow

Operator command → parseArgs → loadEnvFile (.env.local + .env)
  → assertEnvFlag (--env=prod) → OpenAI embedding → hybrid_search RPC
  (default include_superseded=true) → optional domain post-filter
  → optional summary_data join (--full) → JSON or formatted text output

Detailed Steps

1. Argv parsing.

   • Flags: --limit N, --domain "NAME", --threshold N (default 0.25, vs app’s 0.35), --full (joins content_items.summary_data), --json, --include-superseded / --exclude-superseded, --env=prod (assert only — does NOT swap env values).

2. Env loading.

   • findProjectRoot() walks up from script dir + cwd to find .env/.env.local.
   • .env.local loaded first (higher priority), then .env.
   • Reads SUPABASE_URL (or NEXT_PUBLIC_SUPABASE_URL), SUPABASE_PUBLISHABLE_KEY (or NEXT_PUBLIC_…), OPENAI_API_KEY.

3. --env=prod assertion.

   • assertEnvFlag(env, supabaseUrl) checks supabaseUrl.includes('rovrymhhffssilaftdwd').
   • On mismatch: error message instructing operator to override env vars inline.

4. Embed + RPC.

   • Same text-embedding-3-large model, same hybrid_search RPC, same stringified vector convention.
   • Diagnostic default: include_superseded: true (operators want every row).

5. Post-filter + summary join.

   • Domain post-filter is case-insensitive r.primary_domain.toLowerCase() === domain.toLowerCase().
   • --full issues a separate select id, summary_data from content_items where id in (...) for joined display.

6. Format.

   • --json → JSON.stringify(output, null, 2).
   • Default → ANSI-coloured aligned table with title / domain / keywords / snippet.

Error Handling

Error Condition	Handling	User Feedback
Missing env vars	process.exit(1)	”Missing SUPABASE_URL / SUPABASE_PUBLISHABLE_KEY in environment”
--env=prod but URL not prod-pointed	process.exit(1) with override hint	Operator sees inline-override instructions
OpenAI embedding fail	process.exit(1)	”Failed to generate embedding: “
RPC error	process.exit(1)	”Search RPC error: “
Summary data fetch fail	Logs warning, proceeds without summaries	Stderr warning (suppressed in —json mode)

---

Cadence-compliance scorer modifier (S208 §5.5 Phase 5)

Quality score (content_items.quality_score) influences search-result UI through Quality badges and the sort=quality_score Browse option. The modifier extends freshnessRaw() in lib/quality/quality-score.ts:

cadenceCompliancePenalty(nextReviewDate: string | null, now?: Date): number
// 0    when nextReviewDate is null OR > 30 days before due
// 0-10 graduated linear when 1-30 days before due
// 15   when 1-14 days overdue
// 25   when 15-30 days overdue
// 40   when > 30 days overdue

Preservation rule (§9.4): the penalty applies only when next_review_date is non-null — items without cadence produce identical scores to pre-Phase-5 quality calculations. Boundary at daysUntilDue === 0 falls into the overdue ≤14 tier (-15) per spec control flow.

Caller wires:

Caller	Wired?	Notes
components/content/content-card.tsx	Yes	Passes next_review_date + review_cadence_days explicitly
app/api/cron/quality-score/route.ts	Yes	Daily quality recomputation cron
components/item-detail/metadata-sidebar.tsx	Yes	Passes new fields to QualityScoreBreakdown so item-detail aligns
lib/mcp/tools/quality.ts	No	Reads persisted scores; no calculateQualityScore call
components/shared/quality-badge.tsx	No	Accepts precomputed QualityScoreResult

Search ranking is unaffected directly — the modifier flows into quality_score reads, which only the Browse sort=quality_score option consumes today.

Automated Processes

Process	Schedule	Route / Script	Purpose
Daily quality-score recompute	Daily 04:00 UTC	app/api/cron/quality-score/route.ts	Recalculate quality_score (S208 Phase 5 cadence-compliance modifier applied here)
Daily review-cadence flagger	Daily 03:45 UTC	app/api/cron/review-cadence/route.ts	Flag items as review_overdue when next_review_date < CURRENT_DATE (drives overdue_review MCP filter)
Embedding backfill (manual)	Operator-run	bun run scripts/backfill-chunks.ts	Embed chunks/items missing embedding; required before they can surface in hybrid_search / search_content_chunks
Summary regeneration (manual)	Operator-run	bun run scripts/batch-generate-summaries.ts	Refresh summary_data for --full CLI display

Integration Points

External System	Direction	Protocol	Purpose
Supabase	Read	REST + RPC	content_items / content_chunks SELECT, hybrid_search / search_content_chunks RPC
OpenAI	Request	HTTPS	Embedding generation (text-embedding-3-large, 1024-dim)
MCP clients	Request	Streamable HTTP	The find tool (its type / scope / granularity / similar_to branches collapsed the former search_knowledge_base, search_qa_library, find_similar_items, and search_content_chunks tools)
Browser localStorage	Read/Write	Web Storage	kb-recent-searches, kb-filter-presets, kb-view-mode, kb-hide-thumbnails

Current Limitations

• Search Mode caps at 20 results before filter post-filtering. Narrow filter combinations can produce empty result pages even when matching items exist beyond the cap.
• hybrid_search has no native offset. MCP tools over-fetch + slice; pages beyond the default 10 need explicit limit arguments.
• Live preview is title + content only. No ai_keywords, summary, author_name, or chunk-level matching — by design but means certain known-item searches require submitting the full pipeline.
• Saved presets are localStorage-only. No server-side sync — clearing storage erases user presets but leaves the 5 system presets.
• Cold-start prompts depend on primary_focus. Users who never set the preference see the fallback set permanently.
• Recent searches don’t sync across devices.
• /components/guide/guide-research-feed.tsx still calls the wrong method (GET /api/search?q=) — silent failure caught by surrounding try/catch. Tracked for cleanup.
• Cadence-compliance scorer wires only 3 of 5 callers. quality.ts MCP tool and quality-badge.tsx component read precomputed scores rather than recomputing — by design (avoid duplicate calculations) but means the cadence penalty is only applied at the cron daily recomputation cadence for those callers.