pullmd Cloud Run deploy — Liam-side gcloud operations

RETIRED (S298): Cloud Run is fully decommissioned. The ingestion pipeline now runs on-prem (IONOS VPS + Coolify) — see docs/runbooks/onprem-b1-deploy.md. This runbook and its referenced cloudrun/ manifests are removed from the active deploy path; retained for historical context only.

Spec: docs/specs/id-42-pullmd-deploy/TECH.md §WP-C (deploy mechanics); docs/specs/id-42-pullmd-deploy/RESEARCH.md §2.1 (pullmd v2.x HTTP contract), §2.4 (no /health), §4.3 (sidecar topology). Manifests: cloudrun/services/{staging,prod}-pullmd.yaml. Owner: Liam (gcloud-authenticated operator).

This runbook documents the operational steps the worktree subagent cannot execute — there are no gcloud credentials in the sandbox, and the Executor that authored the manifests holds no GCP or Secret Manager access. Each section gives the exact commands, expected outputs, and pass/fail criteria.

The pullmd Service is phew-only (the kpf tenant is decommissioned). It is a single Cloud Run Service per environment running three containers in one pod (topology α — pullmd + Playwright + Trafilatura sidecars), ingress: internal, minScale=maxScale=1, Bearer-token-gated (PULLMD_AUTH_MODE=single-admin).

Prerequisites

gcloud auth login with the operator account that owns the aisolutionhub.co.uk organisation (organizations/195144871477).
Confirm the active project before each block — these commands target two projects (kh-staging-494815 and kh-prod-494815). Always pass --project explicitly; never rely on the gcloud default.
The runtime service account phew-pipeline-sa@<project>.iam.gserviceaccount.com already holds roles/secretmanager.secretAccessor at project scope (see docs/runbooks/cloud-run-phase-1-handover.md §3) — no new grant is needed.

§1. Secret mint (out-of-band — Liam runs)

Three secrets are minted in both kh-staging-494815 and kh-prod-494815 before the first deploy. The deploy workflow only references them by :latest; it never creates them.

Secret	Purpose	Value shape
`PULLMD_API_TOKEN`	Bearer token the cocoindex Service presents to pullmd’s `/api`	`pmd_<32-char-base62>` (SHA-256 hashed server-side per RESEARCH §2.1)
`PULLMD_ADMIN_EMAIL`	pullmd single-admin bootstrap identity	an email address you control
`PULLMD_ADMIN_PASSWORD`	pullmd single-admin bootstrap password	a strong random password

§1.1 Generate the values

1
# Bearer token — pmd_ prefix + 32 base62 chars.
2
printf 'pmd_%s\n' "$(LC_ALL=C tr -dc 'A-Za-z0-9' </dev/urandom | head -c 32)"
3

4
# Admin password — 32 base62 chars.
5
LC_ALL=C tr -dc 'A-Za-z0-9' </dev/urandom | head -c 32; echo

Record both in your password manager — they are not recoverable from Secret Manager once minted (you can only add new versions, not read prior plaintext via the UI).

§1.2 Create the secrets (run for EACH project)

Repeat the whole block once with PROJECT=kh-staging-494815 and once with PROJECT=kh-prod-494815.

1
PROJECT=kh-staging-494815   # then re-run with kh-prod-494815
2

3
# PULLMD_API_TOKEN
4
printf 'pmd_<your-32-char-token>' | gcloud secrets create PULLMD_API_TOKEN \
5
  --project="$PROJECT" \
6
  --replication-policy=automatic \
7
  --data-file=-
8

9
# PULLMD_ADMIN_EMAIL
10
printf '<your-admin-email>' | gcloud secrets create PULLMD_ADMIN_EMAIL \
11
  --project="$PROJECT" \
12
  --replication-policy=automatic \
13
  --data-file=-
14

15
# PULLMD_ADMIN_PASSWORD
16
printf '<your-32-char-password>' | gcloud secrets create PULLMD_ADMIN_PASSWORD \
17
  --project="$PROJECT" \
18
  --replication-policy=automatic \
19
  --data-file=-

Note — gcloud secrets create fails if the secret already exists. If it does, add a new version instead:

1
printf '<new-value>' | gcloud secrets versions add PULLMD_API_TOKEN \
2
  --project="$PROJECT" --data-file=-

Expected: Created secret [PULLMD_API_TOKEN]. (or Created version [1] of the secret [PULLMD_API_TOKEN].). Failure: a PERMISSION_DENIED → confirm you are authenticated as an account with roles/secretmanager.admin on the project.

§1.3 Verify the mint

1
for S in PULLMD_API_TOKEN PULLMD_ADMIN_EMAIL PULLMD_ADMIN_PASSWORD; do
2
  gcloud secrets describe "$S" --project="$PROJECT" --format='value(name)'
3
done

Expected: three lines, each ending …/secrets/PULLMD_<NAME>. Confirm the PULLMD_API_TOKEN value you minted here matches the value you will later mount onto the cocoindex Service (see §2.2) — the cocoindex adapter and pullmd must agree on the same token.

§2. Deploy trigger

Two paths. The WIF-CI path is the norm; workflow_dispatch is the first-deploy / recovery fallback. Both run the ID-42.7 deploy step (the workflow wiring that gcloud run services replaces the pullmd manifest, mounts the admin secrets, and runs the GET / smoke-verify). This runbook does NOT re-implement that step — it documents how Liam triggers it.

§2.1 WIF-CI path (the norm)

Push to production-readiness → deploys to staging (kh-staging-494815).
Push to main → deploys to prod (kh-prod-494815).

The workflow (.github/workflows/cloud-run-deploy.yml) path-triggers on cloudrun/services/**, which already matches the new *-pullmd.yaml manifests. Pushing the manifest change to the relevant branch is sufficient to trigger a deploy.

§2.2 Manual fallback (`workflow_dispatch` — first deploy)

Use this for the very first deploy (before the workflow’s pullmd step has ever run against a live Service) or for recovery. Dispatch the workflow against the target environment:

The environment dispatch input is a type: choice whose allowed values are lowercase production / staging (see cloud-run-deploy.yml workflow_dispatch.inputs). Passing the capital-first Production / Staging returns HTTP 422: Provided value 'Production' for input 'environment' not in the list of allowed values. The capital-first names are the GitHub Environment names the resolve-environment job maps the input to internally — they are NOT the dispatch value.

1
gh workflow run cloud-run-deploy.yml --ref main -f environment=production
2
# or, for staging:
3
gh workflow run cloud-run-deploy.yml --ref production-readiness -f environment=staging

If you need to apply the manifest by hand (outside the workflow), the direct declarative apply is:

1
# Staging
2
gcloud run services replace cloudrun/services/staging-pullmd.yaml \
3
  --project=kh-staging-494815 --region=europe-west2
4

5
# Production
6
gcloud run services replace cloudrun/services/prod-pullmd.yaml \
7
  --project=kh-prod-494815 --region=europe-west2

No separate admin-secret mount is needed. PULLMD_ADMIN_EMAIL, PULLMD_ADMIN_PASSWORD and PULLMD_API_TOKEN are referenced via secretKeyRef in the manifest itself, so the single replace above already mounts them. This is deliberate: PULLMD_AUTH_MODE=single-admin makes pullmd exit(1) at boot if the admin creds are absent (ERR_BOOTSTRAP_MISSING_CREDENTIALS), so a credential-less replace revision can never pass its startup probe — the secrets must be present in the same revision, not bolted on by a follow-up gcloud run services update --set-secrets. (The workflow’s pullmd --set-secrets step re-asserts the same three at :latest and is now a harmless idempotent no-op.)

The cocoindex Service (the pullmd consumer) gets PULLMD_API_TOKEN appended to its existing COCOINDEX_SECRETS mount in the workflow (ID-42.7) — it is NOT mounted on the pullmd Service. --set-secrets is declarative, so the token must be appended to the existing comma-separated set, never issued as a separate update that would drop the rest.

§2.3 Smoke-verify after deploy

pullmd has no /health endpoint (RESEARCH §2.4) — verify with GET /:

1
SERVICE_URL=$(gcloud run services describe kh-pullmd-phew \
2
  --project="$PROJECT" --region=europe-west2 \
3
  --format='value(status.url)')
4

5
# Authoritative readiness check from an operator laptop. `ingress: internal` is a
6
# NETWORK-layer block — an identity token does NOT bypass it, so a direct external
7
# `curl ${SERVICE_URL}/` returns 403/000 even when the Service is perfectly
8
# healthy. Instead rely on the revision Ready state + the in-pod startup-probe log
9
# (Cloud Run runs the `GET /` probe inside the pod, so a Ready revision IS the 200
10
# proof). A from-network curl is only meaningful when run from the cocoindex
11
# Service or another in-project context.
12
gcloud run revisions list --service=kh-pullmd-phew \
13
  --project="$PROJECT" --region=europe-west2 \
14
  --format='table(metadata.name, status.conditions[0].status)'
15
gcloud logging read 'resource.type="cloud_run_revision"
16
  resource.labels.service_name="kh-pullmd-phew"' \
17
  --project="$PROJECT" --region=europe-west2 --limit=20 --freshness=15m \
18
  --order=desc --format='value(textPayload, jsonPayload.message)'
19

20
# Confirm the deployed image is PINNED, not :latest.
21
gcloud run services describe kh-pullmd-phew \
22
  --project="$PROJECT" --region=europe-west2 \
23
  --format='value(spec.template.spec.containers[0].image)'

Expected: the latest revision shows status=True; the logs show STARTUP HTTP probe succeeded … on port 3000 path "/" and PullMD running on http://localhost:3000 (auth: single-admin); the image line ends with a concrete :2.x.y tag and NOT :latest. Failure modes:

Cannot open database because the directory does not exist + exit(1) → CACHE_DB is not pointing at a writable, already-existing dir (better-sqlite3 does not mkdir). The manifest bakes CACHE_DB=/tmp/pullmd-cache.db; if you removed it the upstream default ./data does not exist on Cloud Run.
ERR_BOOTSTRAP_MISSING_CREDENTIALS + exit(1) → the admin secrets did not resolve. Confirm PULLMD_ADMIN_EMAIL / PULLMD_ADMIN_PASSWORD exist (§1.3) and the runtime SA has secretmanager.secretAccessor.
revision status=False with a startup-probe message, or an image line ending :latest → stop and re-check the manifest before wiring PULLMD_SERVICE_URL (§3).

§3. `PULLMD_SERVICE_URL` real-URL swap (both envs)

PULLMD_SERVICE_URL currently holds the S258 placeholder (https://pullmd-not-yet-deployed-staging.example.com) so the cocoindex Service deploy could be unblocked before pullmd existed. After the first pullmd deploy, replace that placeholder with the real Service URL, then redeploy the consuming cocoindex revision so the running revision picks up the new value.

Run for EACH project.

1
PROJECT=kh-staging-494815   # then re-run with kh-prod-494815
2

3
# 1. Read the real Service URL.
4
SERVICE_URL=$(gcloud run services describe kh-pullmd-phew \
5
  --project="$PROJECT" --region=europe-west2 \
6
  --format='value(status.url)')
7
echo "$SERVICE_URL"   # sanity-check: an https://…run.app URL, NOT example.com
8

9
# 2. Replace the placeholder secret value with the real URL.
10
printf '%s' "$SERVICE_URL" | gcloud secrets versions add PULLMD_SERVICE_URL \
11
  --project="$PROJECT" --data-file=-
12

13
# 3. Force a new cocoindex revision so it re-reads PULLMD_SERVICE_URL:latest.
14
#    (phew-only — the kpf tenant is decommissioned.)
15
#
16
#    DO NOT use `gcloud run services replace …cocoindex.yaml` here. The cocoindex
17
#    manifest declares NO secrets — its 14 secret env vars (ANTHROPIC_API_KEY …
18
#    PULLMD_SERVICE_URL) and the injected IMAGE_SHA are applied at deploy time by
19
#    the workflow's --set-secrets / --update-env-vars layer. A declarative
20
#    `replace` would strip all of them and the cocoindex container would crash on
21
#    boot (lib/env-server.ts Zod fails fast). Use --update-secrets, which forces a
22
#    new revision (re-resolving :latest to the version added in step 2) while
23
#    preserving every other secret and env var.
24
gcloud run services update kh-cocoindex-pipeline-phew \
25
  --project="$PROJECT" --region=europe-west2 \
26
  --update-secrets=PULLMD_SERVICE_URL=PULLMD_SERVICE_URL:latest

Verify the swap:

1
gcloud secrets versions access latest --secret=PULLMD_SERVICE_URL \
2
  --project="$PROJECT"

Expected: the live https://…run.app URL, with no not-yet-deployed or example.com substring. Failure: still the placeholder → the versions add did not land, or the cocoindex revision was not redeployed; repeat steps 2-3.

§4. Image-tag confirmation (do this BEFORE the first deploy)

The three image references were confirmed against the upstream pullmd repo (docker-compose.yml + README, github.com/AeternaLabsHQ/pullmd, fetched during ID-42.6 authoring):

Container	Manifest image	Listens on	Pin status
pullmd (primary)	`aeternalabshq/pullmd:2.0.0`	`:3000` (`${PORT:-3000}`)	Concrete. `2.0.0` is the v2.x version the bake-off exercised.
Playwright sidecar	`aeternalabshq/pullmd-playwright:2`	`:8002` (`/render`)	Moving major tag — see note below.
Trafilatura sidecar	`aeternalabshq/pullmd-trafilatura:2`	`:8001` (`/extract`)	Moving major tag — see note below.

Upstream-confirmed facts (so they need no re-confirmation):

The sidecar image names are aeternalabshq/pullmd-playwright and aeternalabshq/pullmd-trafilatura (NOT a generic browserless/chrome — the pre-confirmation placeholder was wrong).
The in-pod sidecar ports + paths are PLAYWRIGHT_URL=…:8002/render and TRAFILATURA_URL=…:8001/extract, and pullmd itself listens on :3000. These are wired in the manifests; no confirmation needed.
Upstream’s :latest tag is the v1.x line (kept for backward-compat); the v2.x family is published under the :2 major tag. Never deploy :latest — it is the wrong major version.

Still requires operator action before first deploy — sidecar digest pin. Upstream publishes no concrete version tag for the two sidecars (only the moving :2 major tag and :latest). :2 tracks the correct major line but is not immutable, which is weaker than Inv-4 (reproducible-from-source) wants. Resolve :2 to an immutable digest for each sidecar and pin it in both manifests before the first deploy:

1
# Resolve the current :2 digest for each sidecar, then pin image@sha256:… .
2
for IMG in aeternalabshq/pullmd-playwright aeternalabshq/pullmd-trafilatura; do
3
  docker pull "$IMG:2"
4
  docker inspect --format='{{index .RepoDigests 0}}' "$IMG:2"
5
done
6
# Replace the :2 tag in both manifests with the resolved <image>@sha256:<digest>.

The primary aeternalabshq/pullmd:2.0.0 is already a concrete version — leave it (or resolve it to a digest too if you want full immutability across all three).

Cross-registry pull / GHCR mirror. Cloud Run can usually pull public Docker Hub images directly. Upstream also publishes to GHCR (ghcr.io/aeternalabshq/{pullmd,pullmd-trafilatura,pullmd-playwright}) — use that if Docker Hub rate-limits. If a deploy fails to pull cross-registry, mirror into the project’s Artifact Registry once and re-point the manifest:

1
# One-time mirror into the project's Artifact Registry.
2
docker pull aeternalabshq/pullmd:2.0.0
3
docker tag aeternalabshq/pullmd:2.0.0 \
4
  europe-west2-docker.pkg.dev/$PROJECT/pipeline/pullmd:2.0.0
5
docker push europe-west2-docker.pkg.dev/$PROJECT/pipeline/pullmd:2.0.0
6
# then set image: europe-west2-docker.pkg.dev/$PROJECT/pipeline/pullmd:2.0.0
7
# (repeat for the two sidecars at their resolved digests.)

Do not block the first deploy on the mirror — try the direct public reference first. The sidecar digest pin above IS a pre-deploy requirement.

§5. Rollback

gcloud run services replace creates a new revision; the prior revision is retained. To roll back to the last-good revision:

1
gcloud run revisions list --service=kh-pullmd-phew \
2
  --project="$PROJECT" --region=europe-west2 \
3
  --format='table(metadata.name,status.conditions[0].status,metadata.creationTimestamp)'
4

5
gcloud run services update-traffic kh-pullmd-phew \
6
  --project="$PROJECT" --region=europe-west2 \
7
  --to-revisions=<last-good-revision>=100

If a rollback changes the resolved Service URL (it should not, the URL is stable per Service), re-run §3 to re-confirm PULLMD_SERVICE_URL.

pullmd Cloud Run deploy — Liam-side gcloud operations

pullmd Cloud Run deploy — Liam-side gcloud operations

Prerequisites

§1. Secret mint (out-of-band — Liam runs)

§1.1 Generate the values

§1.2 Create the secrets (run for EACH project)

§1.3 Verify the mint

§2. Deploy trigger

§2.1 WIF-CI path (the norm)

§2.2 Manual fallback (workflow_dispatch — first deploy)

§2.3 Smoke-verify after deploy

§3. PULLMD_SERVICE_URL real-URL swap (both envs)

§4. Image-tag confirmation (do this BEFORE the first deploy)

§5. Rollback

§2.2 Manual fallback (`workflow_dispatch` — first deploy)

§3. `PULLMD_SERVICE_URL` real-URL swap (both envs)