ID-117 {117.2} PRODUCT — Unified document/revision diff surface (binary + canonical-markdown + user-edit over one revision abstraction)

ID-117 {117.2} PRODUCT — Unified document/revision diff surface

Inputs (read in full): {117.1} RESEARCH.md (CORRECTED S384, supersedes the drifted S383 draft) and ID-59 PRODUCT.md INV-14..INV-17 (the landed user-edit Diff-UI contract ID-117 builds on top of). Grounding decision: S380 (dc46727f) — extend-ui-kit is the LOCKED binary-leg substrate for v1, NOT deferred. This PRODUCT does not re-litigate that lock.

Task: ID-117 (Track C, S380, priority high, 0 deps) — “Unified document/revision diff surface … superseding the legacy source_document_diffs engine and consolidating it with the {59.12} in-platform user-edit diff. ONE surface, three view-depths over a common revision abstraction.” Promoted from bl-267. NOT an ID-45 gate. Retires source_document_diffs.

Summary

One unified diff/revision-comparison surface that compares two snapshots of one logical record at one of three view-depths — (1) binary source diff (DOCX/PDF/XLSX rendered side-by-side via extend-ui-kit viewers), (2) canonical-markdown revision diff (the post-Docling markdown projection), (3) in-platform user-edit diff (the landed content_history / q_a_pair_history comparison) — over a common revision abstraction, and retires the 0-row source_document_diffs engine. The user gets one coherent “observe and intervene” surface (“one record, many views”) instead of three disjoint, partially-built ones.

Problem

Today the three legs are three disjoint surfaces (RESEARCH §3): leg 1 (binary source diff) EXISTS but is text-only — it diffs source_documents.extracted_text strings with no binary rendering of the actual DOCX/PDF/XLSX the client edits; leg 2 (canonical-markdown) has no distinct surface; leg 3 (user-edit diff) is landed-full ({59.12}/{59.16}) but limited to a bespoke unified line-diff. The legacy leg-1 store (source_document_diffs) is empirically 0 rows on dev/platform (RESEARCH §8) and was built pre-pivot, so it is a droppable legacy leaf — but the use-case it served (compare two binary source-doc versions) must be preserved on the new surface. Most client content is binary pre-pipeline, so a real binary viewer is the load-bearing gap.

Code-intelligence orientation (S384, verbatim — repo:'canonical'):

• gitnexus_query('source_document_diffs revision diff content_history') surfaced the legacy compute engine lib/source-documents/document-diff.ts (computeFullTextDiff @332–483, computeDocumentDiff @506–655; process proc_8_post “POST → DetectSectionHeader”, cross_community, module Source-documents), the legacy diff page app/documents/[id]/diff/page.tsx (DocumentDiffPage @141–364), and the fetcher/key seams lib/query/fetchers.ts (fetchItemHistoryList @107–114, fetchItemHistoryVersion @117–124, module Query) + lib/query/query-keys.ts (diff @169–170).
• gitnexus_context('RevisionDiffView') → components/item-detail/revision-diff-view.tsx @228–282 (module Item-detail), incoming calls: exactly TWO — CompareVersionsPanel (components/item-detail/version-history.tsx) and QARevisionHistory (components/qa/qa-revision-history.tsx); outgoing calls RevisionMeta, cn; 0 processes (pure render component). This two-caller seam is the central extension point — the engine is deliberately source-agnostic (revision-diff-view.tsx:12 header comment), so legs 2+3 are render-mode additions to it, and the binary leg is a new composition alongside it.
• Binary-leg substrate columns confirmed present on public.source_documents (supabase/migrations/20260617130000_squash_baseline.sql:7460): storage_path (NOT NULL), mime_type (NOT NULL), version, parent_id (the version chain), extracted_text (nullable). These are the inputs the binary leg needs; no new column is required.

Goals / Non-goals

Goals: (a) ONE surface comparing two revisions of one record at three view-depths over a common revision abstraction; (b) real client-side binary rendering (DOCX/PDF/XLSX) for the binary leg via extend-ui-kit; (c) the richer side-by-side + inline-word markdown view that ID-59 INV-15 deferred to v1.1; (d) clean retirement of source_document_diffs (storage + the legacy text-only diff page/API) once OQ-117-1 is settled.

Non-goals (v1): a new diff-storage table (diffs are computed on demand — inherits ID-59 INV-14); the reference_items / app/reference/[id] surface (flat, un-versioned, no reference_history — RESEARCH §7, OQ-117-6); editing binary originals in-browser (extend-ui’s DOCX/XLSX editors are experimental and out of scope — viewers only); change-overlay annotation ON the rendered binary originals (v1.1 — see INV-7 / OQ-117-2′).

Figma

Figma: none provided. (Binary-leg UX shape carried as OQ-117-2′ for Liam; the user-edit and markdown legs inherit the landed Warm Meridian treatment of RevisionDiffView.)

Behavior

A. The common revision abstraction (what “a diff” is)

1. One surface, two revisions, one record. Every comparison the surface renders is “two snapshots (older, newer) of one logical record, shown at a chosen view-depth”. A comparison is always between two revisions of the same record — the surface never compares two different records, and never compares more than two revisions at once.

2. Three view-depths over two physical substrates. The user (or the calling surface) selects one of three view-depths: binary (leg 1), canonical-markdown (leg 2), user-edit (leg 3). These resolve to two physical revision substrates — the source_documents version chain (parent_id) for binary/source records, and the content_history / q_a_pair_history tables for content-item / Q&A records. The canonical-markdown depth is a projection of one of those substrates at markdown granularity, not a third store.

3. Diffs are computed on demand, never stored. The surface never persists a computed diff (inherits ID-59 INV-14). Given two selected revisions, the comparison is derived live. No new diff-storage table exists after this Task; source_document_diffs is retired (§E).

4. Provenance panel per revision. For each of the two compared revisions the surface shows its provenance: version, change_type, change_summary, created_at (DD/MM/YYYY), the actor label (created_by), and edit_intent where the substrate carries one. Where a substrate has no such columns (the source_documents chain has no edit_intent / change_type / change_summary), the surface shows a synthesised provenance for the revision and omits — rather than fabricates — the missing fields.

   • Open question (OQ-117-4): for a re-ingest source_documents revision, what does the provenance panel show — a synthesised change_type: 'reingest' with edit_intent absent, or some other label? Recommended default: change_type: 'reingest', edit_intent omitted, actor = the uploading/ingesting user; confirm wording with Liam in TECH.

B. View-depth (1) — binary source diff (extend-ui-kit, LOCKED v1)

5. Binary source diff renders the actual files, side-by-side. For two source_documents versions (older, newer) in the same parent_id chain, the binary view-depth renders both original binary files — DOCX, PDF, or XLSX — side-by-side using extend-ui-kit viewers, so the user sees the document as the client actually authored it, not just its extracted text. This is the leg that replaces the legacy text-only diff page.

6. Format coverage and fallback. The binary leg supports PDF, DOCX, and XLSX in v1 (the minimum extend-ui-kit viewer set per RESEARCH §5 / OQ-117-2′). For a mime_type outside that set, or where a binary original cannot be fetched/rendered, the surface degrades to the canonical-markdown / text comparison of the two revisions (leg 2) with a clear inline notice that the binary original could not be rendered — it never shows a blank panel and never silently drops the comparison.

7. v1 binary-diff shape = visual-compare-only (OQ-117-2′ disposition). Because extend-ui’s “Document Split Viewer” is a packet-splitter, not a version-comparator (RESEARCH §5), and no extend-ui component exposes a doc-to-doc diff prop, the binary version-diff is composed from two file-viewer instances rendered side-by-side. For v1 the surface presents a visual side-by-side comparison only — the two rendered originals positioned for human eyeballing, with synchronised page/scroll navigation where the viewers support it — without inline change-marking on the rendered binary. A textual change summary (the extracted_text / markdown line-diff of the two versions, leg 2’s engine) is shown alongside the visual panes so the user still gets a “what changed” signal at text granularity.

   • v1.1 (deferred, defined here): a change-overlay that annotates the rendered binary originals themselves (e.g. highlighting changed regions over the DOCX/PDF render). This is explicitly NOT v1 because it requires bespoke text-diff-to-rendered-coordinate mapping that extend-ui does not supply.
   • Open question (OQ-117-2′): confirm visual-compare-only + alongside text summary for v1 (change-overlay → v1.1). This bounds {117.3}/{117.4} scope.

8. Binary fetch is signed-URL gated and workspace-scoped. The binary originals are fetched via short-lived signed URLs scoped to the requesting user; a user MUST NOT be able to render a source document (or its prior versions) belonging to a workspace they cannot already read. A signed-URL fetch failure surfaces as the leg-2 fallback (INV-6), not a broken viewer.

C. View-depth (2) — canonical-markdown revision diff

9. Canonical-markdown source is content_items.content via content_history. The markdown view-depth compares two content_history revisions of content_items.content (the cocoindex post-Docling projection) — the canonical markdown home. The upload-path source_documents.extracted_text projection is treated as legacy and is only used as the binary-leg text fallback (INV-6), not as the canonical-markdown depth.

   • Open question (OQ-117-3): confirm content_items.content + content_history is canonical for the markdown depth (post-ID-45 full-corpus re-ingest makes cocoindex the dominant path), with extracted_text legacy. Recommended: yes.

10. Markdown depth renders side-by-side with inline word-level highlighting. The canonical-markdown comparison delivers the richer rendering ID-59 INV-15 deferred to v1.1: a side-by-side old↔new markdown view with inline word-level change highlighting (additions/removals marked within changed lines), not only the unified line-diff. The existing word-level primitive (version-diff.tsx, diffWords) and the LCS line-diff (RevisionDiffView) are the building blocks; this leg is where they combine into the side-by-side + word-inline view.

11. Markdown depth shares the user-edit engine and presentation invariants. The canonical-markdown depth obeys the same presentation invariants as the user-edit depth (INV-13): more-than-colour signalling, Warm Meridian semantic tokens, UK English, and an explicit “no changes between these versions” empty state.

D. View-depth (3) — in-platform user-edit diff (the landed engine, preserved)

12. The landed user-edit Diff-UI is preserved, not replaced. The existing RevisionDiffView behaviour — compare two content_history rows (content) or two q_a_pair_history rows (Q&A) selected from version history, default to the latest two, unified line-diff with [+]/[-] gutters and [Added]/[Removed] labels — continues to work exactly as it does today (ID-59 INV-14..INV-17). ID-117 extends this engine with the new view-depths; it does NOT regress the user-edit depth. The two existing callers (CompareVersionsPanel, QARevisionHistory) keep working.

13. Shared presentation invariants (all three depths). Across every view-depth, the surface: distinguishes additions/removals/unchanged by more than colour alone (gutters, labels, or weight — WCAG 2.1 AA, colour never the sole signal); uses Warm Meridian semantic tokens only (e.g. bg-status-success/10, bg-status-error/10 — no raw Tailwind colours); renders dates as DD/MM/YYYY; uses UK English throughout; and renders an explicit “no changes between these versions” state for identical revisions rather than a blank panel. Loading and error states are explicit (skeleton/spinner while fetching; a clear error affordance on fetch failure), never a silent blank.

E. Retirement of source_document_diffs (keep RETIRE and UNIFY separate)

14. The legacy source_document_diffs storage + text-only diff surface is retired. After this Task there is no source_document_diffs table, no api.source_document_diffs view, no lib/source-documents/document-diff.ts compute engine, no app/documents/[id]/diff page, no app/api/source-documents/[id]/diff route, and no MCP get_document_diff tool. The binary view-depth (§B) is the replacement for the use-case those served (compare two binary source-doc versions). Retirement is safe because the table is 0 rows on dev/platform (RESEARCH §8) and S380 states the client DB is likewise empty (app pre-launch).

    • Open question (OQ-117-1) — MUST be answered before the drop migration is authored: (a) confirm source_document_diffs is 0 rows in the CLIENT DB too (unreachable from RESEARCH context); and (b) decide whether the legacy re-ingest review workflow — the status {pending_review, applied, dismissed} / reviewed_at / reviewed_by / reviewer_note / AI-suggest-apply-via-affected_content_item_id path — is RETIRED outright or RE-HOMED onto the unified surface. The storage drops cleanly only if the workflow is retired; if re-homed, the unified surface gains a review-action affordance and the impact-matching (source-document-impact.ts) is preserved. Framed for Liam below.

15. No user-facing review workflow regresses silently. If OQ-117-1 resolves to retire the re-ingest review workflow, the surface MUST NOT present a dangling “review / apply / dismiss” affordance that no longer does anything. If it resolves to re-home, the affordance moves onto the unified surface with equivalent behaviour (mark a revision’s changes applied/dismissed against the affected content item). Either way there is no orphaned half-wired review control.

16. Retirement cleans every coupling, including non-UI consumers. The retirement removes the source_document_diffs reference from the GDPR data-export path (scripts/export-user-data.ts) and the api. view generator (scripts/generate-api-views.ts) — so a user-data export after this Task neither references nor errors on the dropped table, and the api. view-set regenerates without it. (The task-view-vendor-drift / schema-parity side-workflows will fire on the view-set change; handling that re-vendor step is a TECH/PLAN concern, flagged here so it is not missed.)

F. Entry points, roles, and scoping

17. Entry-point / IA is one open decision, but the engine is shared regardless. Whether the unified surface lives at ONE route or stays inline-per-record (binary at a source-document route; markdown/user-edit inline in version history) is OQ-117-5; in either shape the same revision abstraction and the same render engine back all three depths, and the binary leg replaces the retired /documents/[id]/diff page.

    • Open question (OQ-117-5): one consolidated route vs inline-per-record with a shared engine. Affects {117.4} decomposition; recommended default: keep the binary leg at a source-document-scoped route (replacing the retired diff page) and the markdown/user-edit depths inline in the existing version-history surfaces, all sharing one engine. Confirm.

18. Roles. The diff surface (all depths) is read-only and visible to any role that can already read the underlying record’s version history (inherits ID-59 INV-19). Rendering a binary original requires the same read authorisation as reading the source document. IF the re-ingest review workflow is re-homed (OQ-117-1), its write actions (apply/dismiss) are restricted to admin / editor and route failures via authFailureResponse(auth).

19. Workspace scoping. Every depth is workspace-scoped to what the requesting user can already read: a user MUST NOT see, render, or diff a revision (or binary original) of a record in a workspace they cannot otherwise reach. The binary signed-URL fetch (INV-8) enforces this at the storage layer; the history/fetch queries enforce it at the row layer.

20. AI-invisible-infrastructure. Where AI participated upstream (e.g. an AI-suggested re-ingest revision, or AI-assisted similarity in any re-homed review action), the surface does not present it as a user-facing “AI feature” — the user reviews ordinary revisions and the compared content behaves as if the data was always there (inherits ID-59 INV-21, ai-visibility-policy.md).

Empirical verification (pre-ratification, OQ-3)

Symbol / dependency	Class	Method	Result
extend-ui-kit (extend-hq/ui registry: PDF/DOCX/XLSX viewers)	external-library (binary leg)	RESEARCH §5/§8: package.json + node_modules + corpus grep + registry WebFetch (S384)	ABSENT in repo, but resolvable as a shadcn-style registry install (npx shadcn@latest add @extend/<component> copies source into the repo — no compiled peer-dep version lock; MIT; consumes radix-ui ^1.4.3 already present). This is a build prerequisite, NOT a spec-blocking ABSENT — install is a {117.4} slice, not a deferral. No hard blocker vs React 19.2.6 / Next 16.2.6. Date: 21/06/2026.
source_documents.{storage_path,mime_type,version,parent_id,extracted_text}	internal-Canonical (binary-leg substrate)	migration baseline read (squash_baseline.sql:7460)	PRESENT — no new column needed (out of OQ-3 external scope; recorded for completeness).
RevisionDiffView / RevisionBlob (engine to extend)	internal-Canonical	gitnexus_context (S384)	PRESENT, 2 callers (out of OQ-3 external scope).

No external-library symbol the v1 surface relies on returned ABSENT / SIGNATURE_DRIFT in a spec-blocking way. extend-ui-kit’s absence is a registry-install build step (S380-locked), explicitly carried as a {117.4} install slice.

Open questions (collected — for TECH {117.3} and/or Liam)

• OQ-117-1 (retirement scope — BLOCKS the drop migration): confirm client-DB 0-rows AND decide retire-vs-rehome of the re-ingest review workflow. Liam decision needed (framed in the return message).
• OQ-117-2′ (binary-diff UX shape): confirm visual-compare-only + alongside text summary for v1, change-overlay → v1.1. Liam confirmation recommended (defaulted in INV-7).
• OQ-117-3 (canonical-markdown source): confirm content_items.content + content_history canonical, extracted_text legacy. Recommended yes (INV-9).
• OQ-117-4 (binary-revision provenance): synthesised change_type:'reingest', edit_intent omitted — confirm wording (INV-4). TECH-resolvable.
• OQ-117-5 (entry-point / IA): one route vs inline-per-record (INV-17). Affects {117.4}.
• OQ-117-6 (reference-page exclusion): confirm reference_items / app/reference/[id] is OUT of ID-117 scope (Non-goals). Recommended: confirmed out (future seam only, RESEARCH §7).

---

End of PRODUCT. Builds on ID-59 INV-14..INV-17 (landed user-edit Diff-UI); grounded on S380 (extend-ui-kit LOCKED for binary leg, v1) + {117.1} RESEARCH (CORRECTED S384). No implementation, no DDL, no ledger writes performed.