ID-127 {127.14} — IONOS box carry-across inventory (S410)

Carry-across inventory — current IONOS box (`77.68.122.71`)

Read-only inventory of the box-only / non-git-tracked state on the single oversized IONOS box (77.68.122.71, 8 vCPU / 16 GB / 464 GB, hostname ubuntu) ahead of the ID-127 two-server split + box cancellation. Captured 2026-06-24 for subtask {127.14}.

What this box currently hosts (confirmed live, supersedes any “Platform” framing — this is the CLIENT (Phew) pipeline + the Coolify control-plane, exactly per AMENDMENT §3):

Coolify control-plane v4.1.1 (Traefik 3.6.17) — the only control-plane.
kh-onprem-pipeline-production (uuid ybiczck7f7e1xbdev3bk89cr) — Phew prod cocoindex stack, wired to client Supabase rovrymhhffssilaftdwd.
kh-onprem-pipeline-staging (uuid ljurtiab99kb6dst38k76419) — Phew staging cocoindex stack, wired to client Supabase turayklvaunphgbgscat.
An empty canonical-platform Coolify project (uuid xcpfgvoricv3lw7g5wt7z00t) with zero apps — the ID-127 target shell; nothing built in it.

Target after split (AMENDMENT): Server A = this client pipeline (both Phew apps); Server B = Platform pipeline (ca-pipeline-platform-*, to be built) + the relocated Coolify control-plane. The prod/staging Coolify apps + their on-box state captured below carry across to Server A.

SECRET HANDLING. No raw secret VALUES are written here. Secrets are recorded as key-name + where-the-live-value-lives + carry-disposition only. Live values are in the Coolify app env (revealable via env_vars reveal=true) / host files / the GH Production

Staging environments / the password manager.

1. Coolify app envs — both apps

Captured via env_vars(reveal=true). All entries are flagged is_buildtime=true AND is_runtime=true on both apps (no runtime-only split in use). The cocoindex compose service ALSO hardcodes COCOINDEX_IMAGE_TAG inline in the resolved compose (mirrors the env key) — the env key is the source of truth the deploy job PATCHes.

Carry-disposition legend: carry-as-is = value must be byte-identical on the new box (e.g. CRON_SECRET so the Vercel poller bearer still matches); re-point = same Supabase project, value unchanged but contains a password (DSN); regenerate = host-local / should be rotated on rebuild; non-secret = config, recorded verbatim below.

1a. PRODUCTION app (`ybiczck7f7e1xbdev3bk89cr`)

Key	env uuid	Non-secret value (verbatim)	Secret?	Carry-disposition
`GHCR_OWNER`	`tt010oztln7kpg4hgnlf73c2`	`ai-solution-hub`	no	non-secret (carry-as-is)
`COCOINDEX_IMAGE_TAG`	`l92p743mjory7qaopxqjx58e`	`sha-b92aee5e3cc1405d4abcce61a7581472faccb136`	no	non-secret — but this is the per-client pinned tag; set fresh at standup (ID-45 finalises the client pin). The deploy job PATCHes this key.
`COCOINDEX_SOURCE_PATH`	`rqjlptckuuqvt4q8nz0wd98t`	`/cocoindex-state/corpus`	no	non-secret (carry-as-is; boot-never-walks makes it safe permanently)
`COCOINDEX_LMDB_MAP_SIZE`	`srtdk6p18e3f2h50ioxyao06`	`4294967296`	no	non-secret (carry-as-is)
`SUPABASE_URL`	`qyjgasix2a5q6yhxf8hslnip`	`https://rovrymhhffssilaftdwd.supabase.co`	no	non-secret — client prod DB (carry-as-is for Server A)
`NEXT_PUBLIC_SUPABASE_URL`	`r1nz7o8tilnjna4acmeqgt5s`	`https://rovrymhhffssilaftdwd.supabase.co`	no	non-secret (carry-as-is)
`NEXT_PUBLIC_APP_URL`	`owgxt8thsishnfifyrilhqwe`	`https://kh.phew.org.uk`	no	non-secret (carry-as-is)
`PIPELINE_RUN_WEBHOOK_URL`	`j2fyre2ezn0azm0seso1q8c0`	`https://kh.phew.org.uk/api/internal/pipeline-runs/record`	no	non-secret (carry-as-is)
`PULLMD_ADMIN_EMAIL`	`h1dh545eg1zoszklu3mynz3n`	`liam@aisolutionhub.co.uk`	no	non-secret (carry-as-is)
`SUPABASE_PUBLISHABLE_KEY`	`q797gdh0pcmemuhwk7evpa7g`	(`sb_publishable_…` — publishable, low-sensitivity)	low	re-point (client prod publishable key)
`NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY`	`n7rnd5riijgypi6naakwdbmq`	(same publishable key)	low	re-point
`ANTHROPIC_API_KEY`	`jpba83k2g1z8gpkiqd7uzkb9`	—	SECRET	carry-as-is (or rotate at rebuild); value in Coolify env + password mgr
`OPENAI_API_KEY`	`ws8hfqghy4r8g9j8knchej71`	—	SECRET	carry-as-is (or rotate)
`SUPABASE_SERVICE_ROLE_KEY`	`dlhlwnn6vfk81n002f5kzn8m`	—	SECRET	re-point (client prod service-role; same project)
`CRON_SECRET`	`lr3gi11r2jqjh6obciku9dhm`	—	SECRET	carry-as-is (the `/walk` bearer — must equal the Vercel poller’s `CRON_SECRET` for client prod)
`EXTRACT_API_TOKEN`	`qwm1wqif2caqmjktzcd3fxks` + DUP `oyl8brlnbfdh8vkyzeu6a63h`	—	SECRET	carry-as-is — ⚠ DUPLICATE KEY (two env rows, identical value; see DRIFT §)
`SENTRY_AUTH_TOKEN`	`igbq0oiu2ol1faob1ux6knmc`	—	SECRET	carry-as-is (or rotate)
`COCOINDEX_DB_DSN`	`fh9wn6ho2spn1m1xofqy04bw`	host=`aws-1-eu-west-2.pooler.supabase.com:5432` db=`postgres` user=`postgres.rovrymhhffssilaftdwd`	SECRET (pw in DSN)	re-point (same client prod Supabase; DSN contains the DB password)
`PULLMD_API_TOKEN`	`udhxtlrnasxykf2f75yym115`	(`pmd_…` static bearer)	SECRET	carry-as-is — also hardcoded inline in the pullmd compose service (see §8)
`PULLMD_ADMIN_PASSWORD`	`fkfqhuhgm5dukv78ywp8ffq1`	—	SECRET	carry-as-is (or regenerate)

1b. STAGING app (`ljurtiab99kb6dst38k76419`)

Same key set; client staging DB turayklvaunphgbgscat. Differences from prod called out.

Key	env uuid	Non-secret value (verbatim)	Secret?	Carry-disposition
`GHCR_OWNER`	`l42mr5sad96s7691sve0xwmr`	`ai-solution-hub`	no	non-secret
`COCOINDEX_IMAGE_TAG`	`p85lcabjurrjuncswa88vcpi`	`sha-b92aee5e3cc1405d4abcce61a7581472faccb136`	no	non-secret (same tag as prod currently)
`COCOINDEX_SOURCE_PATH`	`li4xnk9xtn8h8usj5ufa3d8i`	`/cocoindex-state/corpus`	no	non-secret
`COCOINDEX_LMDB_MAP_SIZE`	`d13n2r527x0tzi53jbjmzu9o`	`4294967296`	no	non-secret
`SUPABASE_URL`	`fzth09zkzhkcpszxar44fg4v`	`https://turayklvaunphgbgscat.supabase.co`	no	non-secret — client staging DB
`NEXT_PUBLIC_SUPABASE_URL`	`ir6acoe3gtta052knaxwjmzn`	`https://turayklvaunphgbgscat.supabase.co`	no	non-secret
`NEXT_PUBLIC_APP_URL`	`e144nwph6sfbmapdbv67nk14`	`https://knowledge-hub-git-staging-tw-group.vercel.app`	no	non-secret (note: legacy `knowledge-hub-` Vercel host, not `canonical-`)
`PIPELINE_RUN_WEBHOOK_URL`	`xtwqzveywy4hei6x1au4u070`	`https://knowledge-hub-git-staging-tw-group.vercel.app/api/internal/pipeline-runs/record?x-vercel-protection-bypass=…`	SECRET (bypass token in query)	carry-as-is — URL embeds a Vercel protection-bypass token; treat as secret
`PULLMD_ADMIN_EMAIL`	`yltrx0mng8n0h6qpatfm9oyo`	`liam@aisolutionhub.co.uk`	no	non-secret
`SUPABASE_PUBLISHABLE_KEY`	`v4u3c3fzzghw030uqfd4bsay`	(`sb_publishable_…`)	low	re-point
`NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY`	`eguc322gw8uex7cnmnst60pw`	(same)	low	re-point
`ANTHROPIC_API_KEY`	`fzm7msjy35becj5uhmmstadz`	—	SECRET	carry-as-is (distinct value from prod)
`OPENAI_API_KEY`	`djgvfscrcx7bkeu31b9d4r02`	—	SECRET	carry-as-is (distinct from prod)
`SUPABASE_SERVICE_ROLE_KEY`	`tqvg9rsyc9w0q6zwor4hhlm6`	—	SECRET	re-point (client staging service-role)
`CRON_SECRET`	`fhdmazernk6yxn30sfhqmha0`	—	SECRET	carry-as-is (distinct from prod; must match the staging poller bearer)
`EXTRACT_API_TOKEN`	`vx9xhzk5w02zxwoix42j0622` + DUP `z12z8zb6kzicgvldfqqedh9g`	—	SECRET	carry-as-is — ⚠ DUPLICATE KEY (two rows, identical value)
`SENTRY_AUTH_TOKEN`	`bgr4cqg87f22fnfnh8jan7pz`	—	SECRET	carry-as-is — identical value to prod’s `SENTRY_AUTH_TOKEN` (shared)
`COCOINDEX_DB_DSN`	`qrklzkwr549pgx4hedzhoyvv`	host=`aws-1-eu-west-2.pooler.supabase.com:5432` db=`postgres` user=`postgres.turayklvaunphgbgscat`	SECRET (pw in DSN)	re-point
`PULLMD_API_TOKEN`	`g11kadnpxqrgh0zkdtnpzbjk`	(`pmd_…` static bearer; distinct from prod)	SECRET	carry-as-is — also hardcoded inline in the pullmd-staging compose service
`PULLMD_ADMIN_PASSWORD`	`ampofpxrx47s6kiwexnczp5d`	—	SECRET	carry-as-is (or regenerate; distinct from prod)

Checklist keys NOT present as Coolify app env (they are inline in the compose environment: block, resolved from other vars, not standalone env rows): COCOINDEX_DB (=/cocoindex-state/lmdb, literal in compose), PORT (not set — server.py defaults to 8080), CACHE_DB/PULLMD_* for the pullmd service (inline in the pullmd service, see §8), PULLMD_SERVICE_URL (=http://pullmd:3000 literal). No standalone CACHE_DB env row exists.

2. Coolify scheduled tasks — both apps

App	Task uuid	Name	Container	Command	Frequency	Enabled
prod	`gqi55umn3m447gguf2vbp5iq`	`cocoindex-walk`	`cocoindex`	abs-interp `python3 -c …urllib POST /walk` (bearer `$CRON_SECRET`)	`0 2 * * *`	false
staging	`z1197cpafcghwlfsxa22xzlq`	`cocoindex-walk`	`cocoindex-staging`	(same command form)	`0 2 * * *`	false
staging	`ixptsepifshcwtr22zf6zrzo`	`cocoindex-walk-hourly-fallback`	`cocoindex-staging`	(same command form)	`0 * * * *`	TRUE

All three match the runbook exactly (uuids, containers, enabled flags). The full verified command string (absolute-interpreter-path form) is identical across all three:

1
/layers/google.python.runtime/python/bin/python3 -c 'import os,urllib.request as u;print(u.urlopen(u.Request("http://127.0.0.1:8080/walk",method="POST",headers={"Authorization":"Bearer "+os.environ["CRON_SECRET"]}),timeout=30).status)'

No additional scheduled tasks exist on either app. The prod hourly-fallback (runbook {75.15}) is not present (expected — prod hourly fallback was deferred). Carry-disposition: recreate these three tasks on Server A with the same command/cadence/enabled flags (non-secret config).

3. Coolify control-plane

Item	Value	Carry-disposition
Coolify version	4.1.1 (image `ghcr.io/coollabsio/coolify:4.1.1`)	relocate to Server B
Traefik (coolify-proxy)	3.6.17 (image `traefik:v3.6`)	new install on Server B
Realtime / sentinel / helper	`coolify-realtime:1.0.15`, `sentinel:0.0.21`, `coolify-helper:1.0.14`	new install
Coolify DB	`coolify-db` (postgres:15-alpine), volume `coolify-db`	relocate (control-plane DB)
Coolify redis	`coolify-redis` (redis:7-alpine), volume `coolify-redis`	relocate
Managed databases	ZERO (`list_databases` → `[]`) — confirms runbook claim	n/a
Managed services	ZERO	n/a
Storages/S3 backup config	No Coolify-managed S3 backups (no DBs to back up). The only S3 backup is the host-cron LMDB job (§6)	n/a
Server timezone	UTC (`Etc/UTC`)	carry-as-is
Docker auto-cleanup	`force_docker_cleanup=true`, `docker_cleanup_frequency="0 0 * * *"`, threshold 80%	⚠ this is what pruned the LMDB backup tools image — see §6 + DRIFT

`/data/coolify/source/.env` — KEY NAMES present (values NOT dumped; back up to password mgr)

1
APP_ID  APP_KEY  APP_NAME  DB_PASSWORD  DB_USERNAME  DOCKER_ADDRESS_POOL_BASE
2
DOCKER_ADDRESS_POOL_SIZE  PUSHER_APP_ID  PUSHER_APP_KEY  PUSHER_APP_SECRET
3
REDIS_PASSWORD  REGISTRY_URL  ROOT_USERNAME  ROOT_USER_EMAIL  ROOT_USER_PASSWORD

Carry-disposition: back up /data/coolify/source/.env to the password manager before cancellation (required to restore the Coolify control plane per the install.sh warning). APP_KEY + DB/Redis creds are control-plane-internal; a fresh Coolify install on Server B generates its own — this is a backup-for-safety item, not a value to copy verbatim, since Server B is a greenfield Coolify install.

4. Coolify per-app deploy config

Field	PROD (`ybiczck…`)	STAGING (`ljurtiab…`)	Carry-disposition
`watch_paths` (auto-deploy sentinel)	`null`	`deploy/coolify/.never-auto-deploy-sentinel`	see DRIFT — prod has NO sentinel
git source	`ai-solution-hub/canonical` (GitHub App, source_id 1)	same	re-point to same repo
git branch	`release/v1.0.0`	`staging`	carry-as-is (prod tracks a release branch, not `main`)
build pack	`dockercompose` (parsing v5)	same	non-secret
base_directory	`/deploy/coolify`	same	non-secret
compose location	`/docker-compose.production.yaml`	`/docker-compose.staging.yaml`	non-secret (git-tracked)
FQDN / Domains field	`null` (UNSET)	`null` (UNSET)	matches runbook (labels-only routing)
docker_compose_domains	`[]`	`[]`	non-secret
Traefik router (compose label)	`kh-cocoindex-prod-walk`, Host `kh-pipeline.aisolutionhub.co.uk`	`kh-cocoindex-staging-walk`, Host `kh-pipeline-staging.aisolutionhub.co.uk`	re-point hostnames per client/env at standup
Router path scope	`PathPrefix(/walk) ‖ /health ‖ /extract`	same	see DRIFT — `/extract` is now routed (runbook says only `/walk` + `/health`)
cert resolver	`letsencrypt` (httpchallenge, storage `/traefik/acme.json`)	same	non-secret
Resource limits	NONE (`limits_cpus=0`, `limits_memory=0`, cpu_shares 1024)	same	set real limits on the right-sized 4-vCore/4 GB boxes
health_check_enabled (Coolify)	false (compose has its own `/dev/tcp` healthcheck)	false	non-secret
Manual deploy webhook secrets (github/gitlab/gitea/bitbucket)	present (4 per app)	present (4 per app)	regenerate (Coolify auto-generates on app create)

Auto-deploy toggle state is not exposed by the Coolify API/MCP (runbook-known limitation); on prod the disable relies on the UI toggle (no watch_paths sentinel set), on staging the watch_paths sentinel is the disable mechanism. Verify the prod auto-deploy UI toggle state manually before cancellation so it can be reproduced on Server A.

Traefik proxy config (coolify-proxy) — carry-across reference

Coolify-proxy runs Traefik v3.6 with: entrypoints http :80 / https :443 (+ http3), ACME resolver letsencrypt (httpchallenge on the http entrypoint, storage /traefik/acme.json), providers.docker.exposedbydefault=false, file provider watching /traefik/dynamic/. Ports published on host: 80, 443 (tcp+udp), 8080 (Traefik dashboard/API, internal). This is a standard Coolify v4 proxy — reproduced automatically by a fresh Coolify install; no manual carry needed beyond the per-app compose labels (git-tracked).

5. Host hardening

Item	Live state	Carry-disposition
`ufw status numbered`	active; ALLOW IN (v4+v6) for `22, 80, 443, 8000, 6001, 6002` tcp; default deny incoming	reproduce identically on both new boxes (note: 8000/6001/6002 are Coolify ports — only needed on the control-plane box, Server B)
sshd drop-in `/etc/ssh/sshd_config.d/00-kh-hardening.conf`	present (233 B, `-rw-r--r--`): `PasswordAuthentication no`, `KbdInteractiveAuthentication no`, `ChallengeResponseAuthentication no`, `PubkeyAuthentication yes`, `PermitRootLogin prohibit-password`	recreate verbatim on both boxes (sorts before `50-cloud-init.conf`)
Other sshd drop-ins	`50-cloud-init.conf` (600), `60-cloudimg-settings.conf` — cloud image defaults	n/a (the `00-` override wins)
`unattended-upgrades`	enabled + active; `/etc/apt/apt.conf.d/20auto-upgrades` = Update-Package-Lists “1” + Unattended-Upgrade “1”	reproduce on both boxes
SSH key	`~/.ssh/kh_ionos_ed25519` (ed25519, no passphrase) — auth confirmed working	carry-as-is (or generate a fresh per-box key); the key file must be backed up
`/etc/` customisations	none beyond the above (no kh/cocoindex/coolify files in `/etc`)	n/a

6. LMDB backup ⚠ CURRENTLY BROKEN

Item	Live state	Carry-disposition
Backup scripts dir	`/root/kh-backup/` (NOT `/root/kh-secrets/` — runbook conflates) — contains: `lmdb-backup-cold.sh` (5320 B, exec), `lmdb-backup.sh` (7974 B, exec, the SUPERSEDED hot form), `restore-verify.sh` (2667 B, exec), `Dockerfile.tools` (1541 B)	these are committed in-repo (`deploy/onprem/backup/`) — carry via git; the host copies + the host placement are the carry-across item
Secrets env file	`/root/kh-secrets/lmdb-backup.env` (271 B, `-rw-------`, root:root). Keys: `BACKUP_S3_BUCKET`, `AWS_ENDPOINT_URL`, `AWS_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`	regenerate S3 creds on rebuild (re-place this file root-600 on Server A); values in password mgr
Host crontab (root)	`30 3 * * * APP_UUID=ybiczck7f7e1xbdev3bk89cr /root/kh-backup/lmdb-backup-cold.sh >> /var/log/lmdb-backup.log 2>&1`	recreate on Server A with the new prod app UUID
Tooling image `kh-lmdb-backup-tools:latest`	ABSENT from `docker images` — was host-built from `Dockerfile.tools` (base `debian:bookworm-slim` + `awscli`+`zstd`+`ca-certificates`); evidently pruned by Coolify’s nightly `force_docker_cleanup` (00:00, 80% threshold). The nightly backup has been FAILING	rebuild on Server A: `docker build -t kh-lmdb-backup-tools:latest -f /root/kh-backup/Dockerfile.tools /root/kh-backup` — and protect it from cleanup, OR pin it
Backup log `/var/log/lmdb-backup.log`	latest run 2026-06-24 03:30: `docker stop → cp data.mdb → docker start` succeeded, then `docker run kh-lmdb-backup-tools:latest` → `pull access denied … repository does not exist`. Same failure repeated across recent runs	n/a (log is host-local)
Live side effect	the cron does stop+start the prod cocoindex container nightly at 03:30 (≈seconds) but produces NO uploaded backup — so prod LMDB is currently unprotected	flag for the operator; fix is rebuilding the tools image
Host backup tooling	host has `zstd` + `liblmdb0` (lib only) but NOT `mdb_copy`/`mdb_stat` binaries and NOT `aws` — by design (the cold backup ships those in the container)	n/a

Last successful prod LMDB snapshot copy was Jun 6 03:30 (the on-host data.mdb mtime is Jun 6 — prod has not walked since), but it was never uploaded (tools image already gone). No backup object can be assumed to exist in the S3 bucket from any recent run.

7. Docker volumes + LMDB store sizes

docker volume ls (6 volumes, all local driver):

Volume	Purpose	On-host `_data` path	Size
`ybiczck7f7e1xbdev3bk89cr_cocoindex-state`	prod cocoindex state (LMDB + corpus)	`/var/lib/docker/volumes/ybiczck7f7e1xbdev3bk89cr_cocoindex-state/_data`	lmdb 272 K / corpus 20 K
`ybiczck7f7e1xbdev3bk89cr_pullmd-data-production`	prod pullmd SQLite store	`…/ybiczck7f7e1xbdev3bk89cr_pullmd-data-production/_data`	232 K
`ljurtiab99kb6dst38k76419_cocoindex-state-staging`	staging cocoindex state	`…/ljurtiab99kb6dst38k76419_cocoindex-state-staging/_data`	lmdb 2.1 M / corpus 476 K
`ljurtiab99kb6dst38k76419_pullmd-data-staging`	staging pullmd SQLite store	`…/ljurtiab99kb6dst38k76419_pullmd-data-staging/_data`	296 K
`coolify-db`	Coolify control-plane Postgres	(control-plane)	—
`coolify-redis`	Coolify control-plane Redis	(control-plane)	—

LMDB env layout (confirms runbook /mdb subdir): prod …/_data/lmdb/mdb/data.mdb = 262144 B (256 KiB, mtime Jun 6) + lock.mdb 8192 B; staging …/_data/lmdb/mdb/data.mdb = 2088960 B (≈2.0 MiB, mtime Jun 24 — actively walked) + lock.mdb. Carry-disposition: none — LMDB is a memo cache (loss = rebuild, never KH data loss; Supabase is canonical). Do not migrate; the new boxes rebuild memo state on first walk.

Total docker disk: images 17.96 GB (10.6 GB of which is the single cocoindex image), containers 40 MB, volumes 135 MB. Box disk overall: 22 G used / 464 G (5%).

8. PullMD state (feeds bl-359 — the separate decommission; inventory-only)

PullMD stack = pullmd + playwright + trafilatura per env (all from aeternalabshq/*), co-resident with cocoindex in the same Coolify app/compose (not separate apps).

Item	PROD	STAGING	Notes
pullmd image	`aeternalabshq/pullmd:2.0.0`	same	tag-pinned
playwright image	`aeternalabshq/pullmd-playwright@sha256:af1993df…0523`	same	digest-pinned (3.66 GB)
trafilatura image	`aeternalabshq/pullmd-trafilatura@sha256:1e8b2f77…0800`	same	digest-pinned (330 MB)
`CACHE_DB`	`/data/pullmd-cache.db`	`/data/pullmd-cache.db`	persistent (the `{75.18}` fix landed — NOT `/tmp` anymore)
pullmd named volume	`…_pullmd-data-production` → `/data` (232 K)	`…_pullmd-data-staging` → `/data` (296 K)	survives recreates; owned `dhcpcd:messagebus` (image `app` uid/gid)
On-host store files	`pullmd-cache.db` (4 K) + `-shm` (32 K) + `-wal` (193 K)	`pullmd-cache.db` (4 K) + `-shm` (32 K) + `-wal` (259 K)	SQLite WAL mode
`PULLMD_AUTH_MODE`	`single-admin`	`single-admin`	non-secret
`PULLMD_API_TOKEN` (= `PULLMD_AUTH_TOKEN`)	env `udhxtlrn…` AND hardcoded inline in compose (`pmd_…`)	env `g11kadnp…` AND hardcoded inline (`pmd_…`, distinct)	SECRET static bearer; the resolved compose hardcodes the value (drift risk if env + inline diverge)
`PULLMD_ADMIN_EMAIL` / `PULLMD_ADMIN_PASSWORD`	`liam@aisolutionhub.co.uk` / (secret)	same email / (distinct secret)	admin bootstrap
`PULLMD_SERVICE_URL` (cocoindex→pullmd)	`http://pullmd:3000`	`http://pullmd-staging:3000`	inline; the revert valve (inv 24)
`PLAYWRIGHT_URL` / `TRAFILATURA_URL`	`http://playwright:8002/render` / `http://trafilatura:8001/extract`	`…-staging` hostnames	inline

bl-359 relevance: pullmd is wired into BOTH apps’ compose. The decommission removes the 3 pullmd-family services + the pullmd-data-* volumes + the pullmd/admin env keys + repoints PULLMD_SERVICE_URL. Store loss = share-link 404s only (markdown is canonical in Supabase source_documents). Inventory-only — nothing changed.

9. Other carry-across observations

DNS: kh-pipeline.aisolutionhub.co.uk AND kh-pipeline-staging.aisolutionhub.co.uk both have A records → 77.68.122.71 (resolve live). On split, these must be repointed to Server A’s IP (or re-named per client per the S321 “temporary domain” decision).
Traefik ACME certs: acme.json shows NO issued certs for the pipeline hosts (grep for "main" returned empty). Cert issuance for /walk+/health+/extract HTTPS ingress is still pending (matches runbook “live cert issuance = operator”). Nothing to carry — Let’s-Encrypt re-issues on Server A once DNS points there and the labels apply.
Running containers: all 8 pipeline containers (4 prod + 4 staging) Up + healthy; prod cocoindex Up 11 hours (healthy) (bounced by last night’s backup cron); staging cocoindex Up 3 days (healthy). 6 Coolify containers Up 3 weeks (healthy).
Workspace manifests (.kh-workspace-map.json, non-secret, in the corpus volume — these are operator-owned routing config, on-box not git-tracked):
- prod: 1 mapping (test/ → workspace b0000000-…0001).
- staging: 4 mappings (charnwood/, path-a/, s316-smoke/forms/ [route:forms], s316-smoke/content/ [route:content]) → same workspace id. Carry these manifests if the new boxes are meant to resume the same corpus layout (else they regenerate per the new corpus 127.4).
staging corpus extras: _held_forms/ dir + populated corpus/ (476 K) — staging is the active test/smoke env.
SENTRY_AUTH_TOKEN is shared (identical value on prod + staging apps) — note for rotation hygiene.
No .kh-live-verify.env on the host (the {62.9} live-verify secrets file was never placed) — nothing to carry there.

DRIFT vs runbook (`onprem-b1-deploy.md`)

Hostname ubuntu, not the runbook’s narrative. The Host table lists IP/spec but the live hostname is ubuntu. (Brief already noted this; confirmed.)
LMDB backup is BROKEN, not ”✅ ARMED + verified”. Runbook {66.14} claims armed + verified S312, but kh-lmdb-backup-tools:latest is gone (pruned by Coolify nightly force_docker_cleanup), so every nightly run fails at tar+upload. The script + Dockerfile are intact; the IMAGE needs rebuilding + protecting from cleanup. Highest-signal drift.
Backup script location. Runbook {66.14} text implies /root/kh-secrets/; the scripts actually live in /root/kh-backup/ (only the .env is in /root/kh-secrets/). The cron path confirms /root/kh-backup/lmdb-backup-cold.sh.
prod watch_paths = null, not a sentinel. Runbook B1.b describes a watch_paths sentinel as the API-achievable auto-deploy disable. Prod has no sentinel (null) — relies on the UI toggle (not API-visible). Only staging has the sentinel (deploy/coolify/.never-auto-deploy-sentinel). Sentinel string also differs from the runbook’s example .coolify-never-auto-deploy.
Traefik router path scope includes /extract. Runbook Inv-13 says the router covers “exactly /walk + /health”. Live rule on both apps is PathPrefix(/walk) ‖ /health ‖ /extract — /extract was added (the EXTRACT_API_TOKEN endpoint). Runbook §“Pipeline HTTPS ingress” is stale on the path scope.
Duplicate EXTRACT_API_TOKEN env row on BOTH apps. Two env entries with identical value + distinct uuids (prod qwm1wqif…/oyl8brln…; staging vx9xhzk5…/z12z8zb6…) — exactly the duplicate-key footgun the runbook §6 warns about (likely a POST-not-PATCH). On carry-across, create one row only.
prod git branch is release/v1.0.0, staging is staging. Runbook narrative (“every push to main or staging”) — prod tracks a release branch, not main. (Consistent with ID-45 release-pin model, but worth noting the Coolify source-of-truth branch.)
Apps point at CLIENT DBs (correct per AMENDMENT, drift only vs github-environments confusion). prod → rovrymhhffssilaftdwd, staging → turayklvaunphgbgscat (Phew). This is correct for the client pipeline (Server A). The AMENDMENT §6 doc-drift note (that GH Production/Staging were repointed to platform zjqbr/rbwqew) concerns the Vercel/GH-env app DB layer — a DIFFERENT layer from these Coolify pipeline app envs. No conflict; flagged to prevent re-confusion.
No managed databases / no Coolify S3 DB-backup target — matches the runbook’s S311 premise correction (list_databases → []). Confirmed, not drift.

GAPS / could-not-capture

IONOS Cloud-Panel cloud firewall (the provider-level firewall in front of ufw, referenced in runbook {66.6} for opening port 8000) is not inspectable via SSH — it lives in the IONOS Cloud Panel UI. Operator must read it there and reproduce the open ports on the new boxes’ provider firewalls.
Coolify per-app auto-deploy toggle state is not exposed by the Coolify API/MCP (known limitation). Prod’s auto-deploy on/off state could not be read programmatically; the operator must check it in the UI before cancellation (staging uses the watch_paths sentinel instead, which IS visible = sentinel set).
Secret VALUES are intentionally not captured here (key-name + location + disposition only, per the secret-handling rule). Live values: Coolify env (reveal=true), the host files named above, the GH Production/Staging environments, and the password manager.
S3 backup bucket contents (whether any prior lmdb-*.tar.zst object exists) were not listed — the host has no aws CLI and the credentials were not used (read-only mandate + secret-handling). Given the tools image has been absent, assume no recent object landed.
Coolify control-plane internal DB dump not taken (read-only; out of scope) — the carry-across for the control plane is the greenfield reinstall on Server B + the /data/coolify/source/.env backup, not a DB migration.