Product Documentation Quality Review -- Track 1

Product Documentation Quality Review — Track 1

Reviewer: Claude Opus 4.6 (adversarial verification agent) Date: 3 April 2026 Scope: 10 documents across 4 domains (search, bid-management, quality-governance, administration) Method: Every factual claim cross-referenced against codebase files, database types, schema reference, and MCP inventory

---

Overall Assessment

Document	Verdict	Accuracy
search/technical.md	FAIL	~45%
search/user-journeys.md	PASS WITH NOTES	~80%
bid-management/technical.md	FAIL	~55%
bid-management/user-journeys.md	PASS WITH NOTES	~75%
bid-management/workflows.md	PASS WITH NOTES	~80%
quality-governance/technical.md	PASS WITH NOTES	~70%
quality-governance/user-journeys.md	FAIL	~55%
quality-governance/workflows.md	PASS WITH NOTES	~75%
administration/technical.md	PASS WITH NOTES	~75%
administration/user-journeys.md	PASS WITH NOTES	~80%

Summary: 3 documents FAIL outright due to critical factual errors (wrong HTTP methods, hallucinated components, incorrect column names). 7 documents pass with notes but contain errors that would mislead developers. No document is fully accurate. The docs read plausibly but contain the classic Gemini pattern of confident, specific claims that are subtly wrong.

---

Document-by-Document Findings

search/technical.md

Verdict: FAIL Accuracy: ~45%

Factual Errors Found

1. Wrong HTTP method. Doc says GET /api/search. Actual: POST /api/search (app/api/search/route.ts line 15: export async function POST).

2. Wrong parameter names and types. Doc claims query params q, limit, content_types, domain. Actual POST body params are query, threshold, limit, layer (defined in SearchBodySchema at lib/validation/schemas.ts line 67-76). None of the four documented params match the actual four.

3. Hallucinated RPC function. Doc lists search_qa_library as an RPC function. It does NOT exist as a database RPC — it is an MCP tool registered in lib/mcp/tools/search.ts line 181. The supabase/types/database.types.ts has no search_qa_library function.

4. Wrong hybrid_search parameters. Doc says params are query_text, query_embedding, match_count, filter_domain. Actual params (from database.types.ts line 2958-2963): query_embedding, query_text, similarity_threshold, limit_count. Two of four are wrong.

5. Wrong file path for highlight utility. Doc says lib/highlight.tsx. Actual location: components/shared/highlight.tsx.

6. Wrong page component path. Doc says app/(authenticated)/browse/page.tsx. No (authenticated) route group exists. Actual: app/browse/page.tsx.

7. Missing route. app/api/search/suggestions/route.ts exists but is not documented.

8. Wrong useSearch return shape. Doc says { results, isSearching, performSearch }. Actual return (hooks/use-search.ts line 102-108): { results, count, isLoading, error, search }. All three field names are wrong.

Correct Claims Verified

• useSearch exists at hooks/use-search.ts — correct
• ContentLibraryResult exists at components/content/content-library-result.tsx — correct
• content_items table has embedding vector(1024) — correct
• Search consolidated into browse — correct
• /search redirects to /browse — correct (though redirect is 307, not 302 as implied)

---

search/user-journeys.md

Verdict: PASS WITH NOTES Accuracy: ~80%

Factual Errors Found

1. /library route exists but description is imprecise. Doc says “The Q&A Library (/library) restricts this search strictly to items where content-type is qa.” The route exists at app/library/page.tsx but the claim about restricting to qa content type should be verified against the component logic.

2. Dashboard header search claim unverified. Doc says “Dashboard (/ Header Search)” as an entry point. The dashboard exists at / but whether a header search widget redirects to /browse?q=... needs specific component verification.

Missing Information

• No mention of the search suggestions endpoint (/api/search/suggestions)
• No mention that search uses POST, not GET (carries forward from technical.md error)
• No mention of threshold parameter for tuning semantic similarity

---

bid-management/technical.md

Verdict: FAIL Accuracy: ~55%

Factual Errors Found

1. Hallucinated route. Doc lists GET /api/bids/[id]/extract-metadata. This route does NOT exist. The actual tender-related routes are /api/bids/[id]/tender/route.ts and /api/bids/[id]/tender/download/route.ts.

2. Massively incomplete route list. Doc lists 6 routes. Actual count: 29 route files under app/api/bids/. Missing routes include: export/docx, export/xlsx, outcome, outcome/integrate, questions/[qId], questions/match, readiness, responses/[rId], responses/[rId]/history, responses/[rId]/regenerate, responses/[rId]/restore, responses/draft-all, responses/draft-stream, responses/estimate, templates/, templates/[templateId]/analyse, templates/[templateId]/auto-map, templates/[templateId]/fields/, templates/[templateId]/fill, tender/, tender/download. This is a significant omission.

3. Wrong route path. Doc says /api/bids/[id]/match. Actual: /api/bids/[id]/questions/match/route.ts (nested under questions).

4. Wrong hook paths. Doc says hooks/use-bid-export.ts and hooks/use-draft-recovery.ts. Actual locations: hooks/bid/use-bid-export.ts and hooks/streaming/use-draft-recovery.ts.

5. Wrong page component paths. Doc says app/(authenticated)/bid/[id]/page.tsx and app/(authenticated)/bid/[id]/session/page.tsx. No (authenticated) group exists. Actual: app/bid/[id]/page.tsx and app/bid/[id]/session/page.tsx.

6. Hallucinated component. Doc lists MobileActionMenu under components/bid/.... No such component exists in components/bid/.

7. Wrong template_requirements column name. Doc says embedding. Actual column: requirement_embedding (from database.types.ts line 2098).

8. Hallucinated test file. Doc references __tests__/app/bid-pipeline.spec.ts. No such file exists. Related tests include __tests__/api/bid-drafting-pipeline.test.ts, __tests__/api/bid-questions-pipeline.test.ts, __tests__/lib/bid-state-machine.test.ts, and 22 other bid test files.

9. Reference to CopilotPlugin. Doc mentions “ContentTabs/CopilotPlugin bridging” in the user-journeys file. CopilotKit was removed in Session 109 (see CLAUDE.md session history). The only file referencing CopilotPlugin is the doc itself.

Correct Claims Verified

• bid_questions has section_name, question_sequence, confidence_posture, matched_content_ids — correct
• bid_responses has response_text, review_status, drafted_by — correct
• bid_response_history has version, change_reason — correct
• template_completions has fields_filled, storage_path — correct
• workspaces table with type discriminator — correct
• MCP tools list_active_bids, get_bid_detail, get_bid_question, show_bid_dashboard, cite_content, get_content_effectiveness — all verified in MCP inventory
• search_for_bid_response RPC exists — confirmed in database.types.ts line 3076

---

bid-management/user-journeys.md

Verdict: PASS WITH NOTES Accuracy: ~75%

Factual Errors Found

1. CopilotPlugin reference. Line 58 mentions “ContentTabs/CopilotPlugin bridging”. CopilotKit was removed in S109. The drafting streams via draft-stream route, not through CopilotPlugin.

2. content_citations population claim. Doc says “Citations to source KB articles are tracked in content_citations” during save. While the table exists, the claim that auto-save populates it during drafting needs verification — citations are created via the cite_content MCP tool, not automatically.

3. Role permissions for “Delete Bid Workspace”. Doc says only Admin can delete. This should be verified against the actual API route RLS policies.

Correct Claims Verified

• Entry points /workspaces and /bid/[id] — correct routes exist
• ReadinessChecklist component exists at components/bid/readiness-checklist.tsx
• BidExportMenu exists at components/bid/bid-export-menu.tsx
• useDraftRecovery uses localStorage — confirmed from hook implementation
• Tiptap editor in bid session — confirmed

---

bid-management/workflows.md

Verdict: PASS WITH NOTES Accuracy: ~80%

Factual Errors Found

1. Missing state in bid lifecycle. Doc lists: draft -> questions_extracted -> drafting -> in_review -> ready_for_export -> submitted -> won/lost/withdrawn. Actual states from lib/validation/schemas.ts line 524-536 include matching between questions_extracted and drafting. Full sequence: draft, questions_extracted, matching, drafting, in_review, ready_for_export, submitted, won, lost, withdrawn.

2. Match output terminology. Doc says Pass 1 output is “Matching constraints (strong, partial, needs_sme)”. The actual confidence_posture values on bid_questions are strong_match, partial_match, no_content (from schema reference line 321). Close but not exact.

3. ReadinessBadge described as separate component. Doc says “ReadinessBadge lights up green” as a side effect. ReadinessBadge is actually exported from components/bid/readiness-checklist.tsx (line 253), not a standalone component file.

Correct Claims Verified

• 3-pass AI pipeline (match, outline, stream) in lib/ai/draft.ts and lib/ai/match.ts — correct
• bid_response_history is append-only with version and change_reason — correct
• OpenAI for embeddings, Anthropic for drafting — correct architecture

---

quality-governance/technical.md

Verdict: PASS WITH NOTES Accuracy: ~70%

Factual Errors Found

1. Wrong review_assignments key columns. Doc says content_id, reviewer_id, due_date, priority. Actual columns (from database.types.ts lines 1565-1582): reviewer_id, assigned_by, assignment_type, filter_domains, filter_content_types, filter_freshness, due_date, status. There is NO content_id column (assignments scope by filters, not individual content items). There is NO priority column.

2. Wrong verification_history column names. Doc says content_id, verified_by, verification_type. Actual columns: content_item_id, performed_by, action_type. All three names are wrong.

3. VerificationBadge props wrong. Doc says props are verified, trustData. Need to verify against actual component but these prop names seem plausible though potentially inaccurate.

4. Missing governance_config columns. The technical doc table omits auto_flag_on_quality_drop, auto_flag_on_freshness_transition, auto_flag_cooldown_days, quality_score_threshold — all of which exist in the actual table (confirmed in database.types.ts lines 1145-1153) and are central to the governance bridge logic described in workflows.md.

Correct Claims Verified

• All 6 review API routes exist and paths match
• Governance API routes exist (GET/POST /api/governance)
• ReviewQueuePanel, AssignmentManager, ReviewCadenceCard exist at stated paths
• VerificationBadge and QualityBadge exist at stated paths
• governance_config has domain, posture, timeout_days — correct
• digests table columns match
• MCP tools delete_content_item, update_governance_status, get_quality_briefing, get_quality_actions all verified

---

quality-governance/user-journeys.md

Verdict: FAIL Accuracy: ~55%

Factual Errors Found

1. Wrong review action model. Doc describes Journey 1 step 2 as: “Reviewer approves -> status transitions to approved. A toast notification appears enabling an Undo action.” The actual review actions (from app/api/review/action/route.ts lines 14-22) are: verify, flag, skip, unverify, unflag. There is NO approve action. The system uses verification (verified_at/verified_by), not approval-based governance_review_status transitions.

2. Fabricated “Flag quality” button. Journey 2 describes a “Flag quality” button in the AI summary tab footer. The actual review action is flag which creates a review_needed entry in ingestion_quality_log — the UI mechanism needs verification but the description oversimplifies.

3. Wrong VerificationBadge states. Doc says trust levels are “Unverified (yellow), Verified (green check), Curated (green ellipsis icon - indicating verification + brief + detail + owner assigned)”. The progressive trust model is more nuanced and the specific visual representations should be verified against components/shared/verification-badge.tsx.

4. Governance review status model confusion. The entire framing implies a governance_review_status approve/reject workflow, but the actual review action route uses a verification model (verify/unverify/flag/unflag), which is a fundamentally different paradigm. The governance_review_status column exists on content_items but its transitions are NOT managed by /api/review/action.

Correct Claims Verified

• Entry points /review, /change-reports, /item/[id], /settings all exist
• Editor+ access for review queue — correct
• Change Reports at /change-reports — correct
• Role permissions table broadly correct

---

quality-governance/workflows.md

Verdict: PASS WITH NOTES Accuracy: ~75%

Factual Errors Found

1. Wrong governance review states. Doc workflow 3 lists transitions: approved -> pending -> approved/changes_requested. The actual governance_review_status values need to be checked, but changes_requested appears to be fabricated — no codebase reference to this status was found.

2. Wrong verification_history column names. Workflow 3 database operations table says columns content_id, verified_by, verification_type. Actual: content_item_id, performed_by, action_type.

3. content_history claim. Doc says edits create “Original snapshot preserved in content_history”. The content_history table exists with trigger-based population, but the doc implies it’s part of the review action flow, which is misleading — it’s a separate trigger mechanism.

Correct Claims Verified

• Quality-to-governance bridge via lib/quality/quality-score.ts — file exists
• Freshness bridge via lib/freshness.ts — file exists
• Cron routes: freshness-transitions, quality-score, classification-quality — all exist
• governance_config.quality_score_threshold and auto_flag_on_quality_drop — confirmed in DB types
• governance_config.auto_flag_on_freshness_transition — confirmed in DB types
• auto_flag_cooldown_days — confirmed in DB types

---

administration/technical.md

Verdict: PASS WITH NOTES Accuracy: ~75%

Factual Errors Found

1. Incomplete admin API routes. Doc lists only GET/POST /api/admin/users/. Actual routes include: /api/admin/users/route.ts, /api/admin/users/[userId]/route.ts, /api/admin/users/invite/route.ts. The [userId] and invite sub-routes are missing.

2. “chron” typo. Doc says “automated lifecycle chron jobs” (line 9). Should be “cron jobs”.

3. Settings sections undercount. Doc references “9 settings sections” via the sidebar. Actual components in components/settings/ include at least 13 distinct section components: profile, connected-apps, connections, content-organisation, taxonomy, tags, entities, guides, layers, team, governance, activity, developer-setup. The doc misses connected-apps, connections, content-organisation, developer-setup, and layers.

4. taxonomy_domains.provenance CHECK values. Doc says values are name, provenance as key columns. The actual CHECK values for provenance are baseline, client, recommended (not seeded as the schema quick reference says — there’s a discrepancy between the schema doc and actual DB).

Missing Information

• No mention of taxonomy-related API routes (domains/subtopics are managed via Supabase client queries, not dedicated API routes, which is worth noting)
• No mention of the notification system beyond the table listing
• No mention of intelligence-poll cron (though this is correctly scoped out as SI)

Correct Claims Verified

• SettingsSidebar, TaxonomySection, TagsSection, TeamSection, LayersSection all exist at stated paths
• user_roles has role column with admin/editor/viewer CHECK — correct
• user_roles has display_name — confirmed in database.types.ts
• layer_vocabulary has key, label, display_order — correct
• notifications has user_id, type, entity_type — correct
• 5 cron routes (excluding SI) — correct listing

---

administration/user-journeys.md

Verdict: PASS WITH NOTES Accuracy: ~80%

Factual Errors Found

1. Oversimplified settings access. Doc says “All roles (view adapted)” can access /settings. The actual role-based visibility of individual settings sections needs component-level verification.

2. Missing settings sections. Doc references “9 settings sections” (from Track 1 continuation prompt). Actual count is 13+ components as noted above.

Correct Claims Verified

• /settings page route exists
• TaxonomySection manages taxonomy_domains and taxonomy_subtopics — correct
• TeamSection manages user roles — correct
• Role promotion workflow (viewer to editor) — plausible
• Database trigger for auth.users to user_roles — confirmed architecture

---

Cross-Cutting Issues

1. Phantom (authenticated) Route Group

Multiple documents reference app/(authenticated)/... paths. This route group does NOT exist in the codebase. All authenticated pages live directly under app/ (e.g., app/browse/page.tsx, app/bid/[id]/page.tsx). This error appears in search/technical.md and bid-management/technical.md.

2. Systematic Column Name Errors

The verification_history table columns are wrong in every document that references them. The docs say content_id, verified_by, verification_type but the actual columns are content_item_id, performed_by, action_type.

3. Review Action Model Misunderstood

The governance docs describe an approve/reject workflow, but the actual system uses a verify/flag model. This is not a minor naming error — it represents a fundamental misunderstanding of how the review system works.

4. Hook Subdirectory Structure Ignored

Hooks are organised into subdirectories (hooks/bid/, hooks/streaming/, etc.) but docs list flat paths (hooks/use-bid-export.ts). This reflects incomplete research of the actual file structure.

---

Recommendations

1. Should existing docs be corrected?

Yes, mandatory. Three documents fail and should be rewritten with actual codebase verification. The remaining seven need targeted corrections. The error density is high enough that these docs cannot be trusted as authoritative references in their current state.

2. Should Track 2 proceed?

Yes, but with a different process. Track 2 should NOT use the same agent/model that produced Track 1. The systematic errors (phantom route groups, wrong column names, fabricated components) indicate the research phase was inadequate. Track 2 should ensure the research agent reads actual source files, not just reference documents.

3. Should client-facing docs be updated?

Not yet. Fix Track 1 first. Using these docs to update client documentation would propagate errors.

4. Does the documentation strategy need revision?

The strategy is sound; execution was poor. The four-phase process (research, draft, adversarial verify, finalise) is correct. The problem is that Phase 3 (adversarial verification) either was not performed or was performed superficially. The strategy document itself correctly warns against this.

---

Priority Actions

1. Critical — Fix search/technical.md: Wrong HTTP method (POST not GET), wrong params, wrong hook return shape, hallucinated RPC function. Nearly every technical detail is wrong.

2. Critical — Fix quality-governance/user-journeys.md: Wrong review action model (verify/flag not approve/reject). Fundamental misunderstanding of the system.

3. Critical — Fix bid-management/technical.md: Hallucinated route, hallucinated component, wrong hook paths, wrong column name, 23 missing routes.

4. High — Fix verification_history columns everywhere: content_id -> content_item_id, verified_by -> performed_by, verification_type -> action_type.

5. High — Remove all (authenticated) route group references: Replace with actual paths (e.g., app/browse/page.tsx).

6. High — Fix review_assignments columns: Remove phantom content_id and priority; document the filter-based scoping model.

7. Medium — Add missing bid state matching: Add to workflows state machine diagram.

8. Medium — Fix governance_config column documentation: Add the bridge-related columns that drive automation.

9. Medium — Remove CopilotPlugin references: CopilotKit was removed in S109.

10. Low — Fix template_requirements.embedding to requirement_embedding.