diff --git a/.github/workflows/docker-image.yml b/.github/workflows/docker-image.yml index 475e22254..d881a31af 100644 --- a/.github/workflows/docker-image.yml +++ b/.github/workflows/docker-image.yml @@ -2,9 +2,10 @@ name: Docker image build # Validates that the production app and worker images build cleanly from the # tree — the deploy contract in docs/deployment-architecture.md ("Images are -# built from main"). Runs the exact `npm ci` + `next build` the Fly.io / Cloud -# Run deploy runs, on a runner with enough memory for the build's 8 GiB heap -# (local Docker Desktop OOMs on this build; GitHub runners have ~16 GiB). +# built from main"). Runs the exact `npm ci` + `next build` the Railway deploy +# runs, on a runner with enough memory for the build's 8 GiB heap (local Docker +# Desktop OOMs on this build; GitHub runners and Railway's remote builder have +# enough headroom). # # Non-blocking: this is a standalone workflow, not part of the required `verify` # check. Nothing is pushed to a registry — the build either succeeds or fails. diff --git a/docs/deployment-architecture.md b/docs/deployment-architecture.md index ed39808b8..5ef219879 100644 --- a/docs/deployment-architecture.md +++ b/docs/deployment-architecture.md @@ -1,28 +1,35 @@ # Deployment Architecture -Decision record for the production topology of Clinical KB. Written 2026-07-06. -Companion documents: `docs/observability-slos.md` (SLOs + eval canary) and -`docs/capacity-review.md` (load model, first bottleneck, soak test). - -Status of this document: **decided and partially implemented**. The app-tier and -worker container images ship in this repo (`Dockerfile`, `Dockerfile.worker`). -Host provisioning, staging setup, and secret placement are operator actions and -are specified here but not executed by this change. - -## 1. Current state (what exists today) - -- **No production deployment target.** There is no hosting config; before this - change there was no Dockerfile. `npm run check:deployment-readiness` boots - `next start` locally and verifies project identity — it proves the build can - serve, not that anything is deployed. +Decision record for the production topology of Clinical KB. Written 2026-07-06, +revised 2026-07-12 when the app went live on Railway. Companion documents: +`docs/observability-slos.md` (SLOs + eval canary) and `docs/capacity-review.md` +(load model, first bottleneck, soak test). + +Status of this document: **decided and live in production.** The app tier and +ingestion worker run on Railway (Singapore) from the committed `Dockerfile` and +`Dockerfile.worker`. The core platform is **Railway** (see §2, "Why Railway"). +Host provisioning, staging setup, and secret placement remain operator actions +and are specified here. + +## 1. Current state (what runs today) + +- **Live on Railway.** Project `clinical-kb`, environment `production`, region + **Southeast Asia (`asia-southeast1-eqsg3a`, Singapore)** — the closest Railway + region to the Supabase project. Two services from this one repo: + - **`app`** — the Next.js app tier (`Dockerfile`), reachable at its Railway + service domain (e.g. `https://app-production-68ebf.up.railway.app`), one warm + replica, `/api/health` healthcheck, restart-on-failure. + - **`worker`** — the ingestion worker (`Dockerfile.worker`), one always-on + replica, long-polling the ingestion queue. - **Database/auth/storage:** live Supabase project `Clinical KB Database` (`sjrfecxgysukkwxsowpy`), region **ap-southeast-2 (Sydney)**, Postgres 17, ~2,000 indexed documents / ~69k chunks. RLS is service-role-only; the app - layer is the ownership boundary. -- **Ingestion:** a local worker (`npm run worker`) that needs a Python OCR - stack (PyMuPDF, Pillow, pytesseract + the Tesseract binary), plus the - `indexing-v3-agent` Supabase Edge Function acting as a cron-triggered - completion/repair gate — not a full extraction pipeline. + layer is the ownership boundary. Supabase is a managed external service — it is + **not** on Railway's private network, so the app↔DB path is public internet + fronted by Supabase's CDN (see §2.1). +- **Ingestion:** the containerized `worker` plus the `indexing-v3-agent` Supabase + Edge Function acting as a cron-triggered completion/repair gate — not a full + extraction pipeline. - **Known failure mode:** silent degradation. Hybrid retrieval RPCs once died quietly while the app kept serving from fallbacks. Every topology decision below biases toward _loud_ failure and standing guards. @@ -32,12 +39,33 @@ are specified here but not executed by this change. ### Decision Run the Next.js app as a **single long-lived container** (Node 24, image built -from `Dockerfile`) on a managed container host **in Sydney, co-located with -the Supabase project's ap-southeast-2 region**. - -Recommended host: any OCI-image host. Production runs on **Railway**; Google -Cloud Run (`australia-southeast2`) is a Sydney-region alternative if lower -answer latency matters. The image is host-agnostic. +from `Dockerfile`) on **Railway**, pinned to the **Southeast Asia (Singapore)** +region — the closest Railway region to the Supabase project's ap-southeast-2 +(Sydney) home. Keep one warm replica (no scale-to-zero). + +### Why Railway + +Railway runs plain OCI images with per-service secrets, health checks, rolling +deploys with rollback to a previous deployment, private networking, and a managed +multi-service project model that fits the app-plus-worker topology directly. Two +properties made it the pragmatic core platform: + +- **Remote builds.** Railway builds the image on its own infrastructure, so the + 8 GiB-heap `next build` (see the image contract below) never has to run on a + local Docker daemon — the local build reliably OOMs on that heap, which had been + the deploy blocker. In production the app image compiled in ~21 s with no OOM on + a 1-CPU builder. +- **Already provisioned.** The account, workspace, and billing were in place, so + there was no cold-start account/credentials blocker. + +**The trade Railway forces — and why it was accepted:** Railway has **no +Australian region** (its regions are US West, US East, Amsterdam, Singapore), +while Supabase is in Sydney. The app therefore pays a cross-region hop to the +database (quantified in §2.1). The penalty is real but bounded, falls almost +entirely on _novel_ (cache-miss) answers, is dwarfed by OpenAI generation time, +and is fully addressable within Railway (a Singapore read replica — §2.1). Given +that shipping a real environment was the priority, a live-in-Singapore deployment +beat an indefinitely blocked "optimal" one. ### Why a long-lived container and not serverless (Vercel et al.) @@ -57,13 +85,70 @@ answer latency matters. The image is host-agnostic. PostgREST/auth traffic against a database whose auth server is capped at 10 absolute connections (see `docs/capacity-review.md`). -Scale-out plan: stay at 1 instance (vertical scaling first) until sustained -load demands more; replicas are safe but dilute in-memory coalescing, so add -them only after the shared `rag_response_cache` hit rate is confirmed healthy. +Scale-out plan: stay at 1 replica (vertical scaling first) until sustained load +demands more; replicas are safe but dilute in-memory coalescing, so add them only +after the shared `rag_response_cache` hit rate is confirmed healthy. On Railway, +prefer a single-region replica bump (`railway scale southeast-asia=N`) over +spreading replicas across regions, which would multiply the cross-region DB hop. **Before the first vertical scale-up**, clear the auth 10-connection cap so the auth pool scales with compute instead of staying pinned — operator runbook: `docs/auth-connection-cap-runbook.md` (`docs/capacity-review.md` §2–§3). +### 2.1 The Railway↔Supabase connection (Singapore → Sydney) + +This is the one place the topology is not co-located, so it is characterized here +rather than left as a footnote. + +**Path.** The app uses `@supabase/supabase-js` (PostgREST over HTTPS) against +`https://.supabase.co`. That hostname is anycast/CDN-fronted, so the TCP+TLS +connection terminates at the nearest edge PoP (~2 ms from Railway Singapore) and +the CDN forwards the request over its backbone to the Postgres/PostgREST origin in +Sydney. `@supabase/supabase-js` runs on undici with keep-alive, so warm requests +reuse the pooled connection and skip the client→edge handshake — but every request +that reads data still has to reach the Sydney origin, so the edge→origin hop is +inherent per RPC. + +**Measured (Railway Singapore → Supabase Sydney), 2026-07-12:** + +| Path | What it is | Result | +| ---------------------------------- | ------------------------------------------------------- | ----------------------------------------------------- | +| Raw TCP → Sydney Postgres pooler | Physical Singapore↔Sydney RTT floor | **~94 ms** | +| Authenticated PostgREST round-trip | Real per-RPC cost the app pays | **~145 ms best, ~340 ms typical** | +| Production novel answer | `supabase_rpc_latency_ms` (retrieval, 3 query variants) | **~4.4 s**; total ~25 s incl. ~19 s OpenAI generation | + +**What multiplies, what doesn't.** Retrieval fans out _wide_ but the RPCs within +a stage run in parallel (`Promise.all`), so fan-out width costs ~1×RTT, not N×. +What multiplies RTT is the sequential **depth** — ~5–8 serial DB round-trips on a +cache-miss answer (index-version check [5 s TTL], shared-cache lookups, retrieval +stages, chunk + document hydration). Net penalty vs. full co-location with the +database region: roughly **+0.6 s to +2 s per novel answer**. Cached/repeat answers are served from +the in-memory LRU (or coalesced) and are largely spared; answers that fall to the +shared Postgres cache tier still pay ~one origin round-trip. + +**Optimisations in place:** + +- Region pinned to Singapore (closest region) rather than a Railway default. +- One warm replica, no scale-to-zero — keeps the caches and coalescing hot and + avoids cold-start connection amplification against the 10-connection auth cap. +- Single-region scaling policy (above) so replicas never spread the DB hop. +- keep-alive connection reuse (undici default) removes repeated client→edge + handshakes on the hot path. + +**Mitigations if the penalty starts to bite (not yet needed):** + +1. **Supabase read replica in Singapore** (biggest win). Co-locating a read + replica with Railway serves the read-heavy retrieval RPCs locally (~5 ms), + collapsing the penalty to near-parity with full co-location. Writes (telemetry, + cache) stay on the Sydney primary and are off the answer critical path. Cost: + the paid replica add-on **plus** app work to route reads to the replica + endpoint. This is the recommended first lever. +2. **Reduce sequential DB depth** in the answer path (batch the cache-version + + shared-cache probes, collapse hydration round-trips). App change; measure + `latencyTimings.supabase_rpc_latency_ms` before/after. + +The OpenAI leg is region-agnostic: OpenAI is US-hosted, so app→OpenAI RTT is +comparable (~200 ms) from Singapore or Sydney and does not favour either host. + ### Image contract (`Dockerfile`) - `node:24-bookworm-slim` in all stages — respects `engines`/`engine-strict` @@ -74,36 +159,42 @@ auth pool scales with compute instead of staying pinned — operator runbook: `--webpack` flag is deliberate: `next.config.ts` carries a webpack-specific WasmHash workaround and the CSP-nonce work was validated against webpack prod chunks, so switching bundlers needs its own verified change. The build - allocates an 8 GiB heap; give the Docker builder ≥ 10 GiB memory. + allocates an 8 GiB heap; Railway's remote builder handles it (the ceiling is + headroom, not a reservation — the real build compiled in ~21 s). - `NEXT_PUBLIC_SUPABASE_URL` and `NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY` are - build args (they inline into the client bundle). The publishable key is - public by design; the placeholder default exists so CI can build without + build args (they inline into the client bundle). On Railway, service variables + are exposed to the Dockerfile build via the matching `ARG` declarations, so + setting them as service variables inlines the real values. The publishable key + is public by design; the placeholder default exists so CI can build without secrets. **Production images must be built with the real publishable key.** - Runtime is a non-root `node` user, prod-only `node_modules`, direct - `next start -H 0.0.0.0 -p $PORT` (the local port-picker script is - deliberately bypassed), and a `HEALTHCHECK` against `/api/health`. + `next start -H 0.0.0.0 -p $PORT` (Railway injects `$PORT`; the local + port-picker script is deliberately bypassed), and a `HEALTHCHECK` against + `/api/health`. - No secret is ever baked into a layer. `SUPABASE_SERVICE_ROLE_KEY`, - `OPENAI_API_KEY`, etc. are injected at run time by the host's secret store. - -Minimal Fly config (create at deploy time; not committed until an app is -provisioned): - -```toml -# fly.toml (sketch — values fixed at provisioning time) -app = "clinical-kb" -primary_region = "syd" -[build] -[env] - PORT = "3000" -[http_service] - internal_port = 3000 - force_https = true - min_machines_running = 1 # keep the warm instance: caches + coalescing - auto_stop_machines = false # no scale-to-zero: cold starts defeat the SLOs -[[http_service.checks]] - path = "/api/health" - interval = "30s" - timeout = "5s" + `OPENAI_API_KEY`, etc. are injected at run time by Railway's variable store. + +### Config as code (`railway.app.json`) + +The app service's build/deploy config is captured in `railway.app.json` at the +repo root (Railway schema). It is intentionally **not** named `railway.json` +because a default-named file is auto-loaded by _every_ service in the project and +would clash with the worker (which needs a different Dockerfile and no +healthcheck). To activate it, set the app service's config-as-code path to +`railway.app.json` (dashboard → service → Settings → Config-as-code, or the +service-settings API). Until wired, the live service settings are the source of +truth and the file mirrors them: + +```jsonc +// railway.app.json (mirrors the live app service) +{ + "build": { "builder": "DOCKERFILE", "dockerfilePath": "Dockerfile" }, + "deploy": { + "healthcheckPath": "/api/health", + "restartPolicyType": "ON_FAILURE", + "multiRegionConfig": { "asia-southeast1-eqsg3a": { "numReplicas": 1 } }, + }, +} ``` ## 3. Ingestion tier @@ -112,9 +203,12 @@ primary_region = "syd" Ship the existing worker as a container (`Dockerfile.worker`: Node 24 + tsx + Tesseract + a Python venv with `worker/python/requirements.txt`) and run **one -always-on worker instance** co-located in Sydney. The `indexing-v3-agent` Edge -Function **stays** in its current role as the cron-triggered completion/repair -gate — the two are complementary, not alternatives. +always-on worker instance** co-located in Railway Singapore (`worker` service). +The `indexing-v3-agent` Edge Function **stays** in its current role as the +cron-triggered completion/repair gate — the two are complementary, not +alternatives. The worker service selects its Dockerfile via the +`RAILWAY_DOCKERFILE_PATH=Dockerfile.worker` variable (captured in +`railway.worker.json`). > **Operator run recipe:** the copy-pasteable build/run/verify steps, the > required env + secrets, and the pre-deploy migration gate live in @@ -139,8 +233,11 @@ Reasoning: only new artifact is the image. The edge path would fork the pipeline into two implementations that drift — this repo's defining failure mode. -Scaling: raise `WORKER_BATCH_SIZE` / `WORKER_CONCURRENCY` on the single -instance first; add replicas only for sustained backlog (safe by construction). +Scaling: raise `WORKER_BATCH_SIZE` / `WORKER_CONCURRENCY` on the single instance +first; add replicas (`railway scale --service worker southeast-asia=N`) only for +sustained backlog (safe by construction). The worker also pays the Singapore→ +Sydney hop on each write, but ingestion is batch/background and off the answer +critical path, so its latency budget is generous. ### Queue durability when a worker dies mid-job @@ -175,7 +272,8 @@ Operational rules that follow: - **Worker death costs at most one stale window of latency** for the in-flight job and zero data loss: all artifact writes are idempotent per generation/chunk-key, and completion is gated by the strict completion RPCs - plus the edge agent. + plus the edge agent. Railway's restart-on-failure brings the worker back and + it reclaims stale jobs automatically. - **Backlog improvement (not in this change):** a heartbeat that refreshes `locked_at` could ride the existing throttled progress updates (`WORKER_PROGRESS_UPDATE_MIN_INTERVAL_MS`, 60 s), which would let the stale @@ -184,26 +282,31 @@ Operational rules that follow: ## 4. Secrets management -| Variable | Sensitivity | Build-time or runtime | Where it lives | -| ------------------------------------------------ | ---------------- | --------------------- | ------------------------------------------------------------------- | -| `NEXT_PUBLIC_SUPABASE_URL` | public | build (inlined) | Dockerfile ARG / repo | -| `NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY` | public-by-design | build (inlined) | Dockerfile ARG; real value passed by the release pipeline | -| `SUPABASE_SERVICE_ROLE_KEY` | **critical** | runtime | host secret store; GitHub repo secret (CI boot smoke + eval canary) | -| `OPENAI_API_KEY` | **critical** | runtime | host secret store; GitHub repo secret | -| `SUPABASE_PROJECT_REF` / `SUPABASE_PROJECT_NAME` | low | runtime | plain env (pins `check:supabase-project`) | -| `INDEXING_V3_AGENT_SECRET` | high | runtime | Supabase Edge Function secrets | -| `RAG_QUERY_HASH_SECRET` | high | runtime | host secret store; GitHub repo secret (CI boot smoke) | -| `E2E_USER_EMAIL` / `E2E_USER_PASSWORD` | medium | CI only | GitHub repo secrets | +| Variable | Sensitivity | Build-time or runtime | Where it lives | +| ------------------------------------------------ | ---------------- | ------------------------- | ----------------------------------------------------------------- | +| `NEXT_PUBLIC_SUPABASE_URL` | public | build (inlined) + runtime | Railway service variable (also the Dockerfile ARG default) | +| `NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY` | public-by-design | build (inlined) | Railway service variable (exposed to the build via `ARG`) | +| `SUPABASE_SERVICE_ROLE_KEY` | **critical** | runtime | Railway variable store; GitHub repo secret (CI boot smoke + eval) | +| `OPENAI_API_KEY` | **critical** | runtime | Railway variable store; GitHub repo secret | +| `SUPABASE_PROJECT_REF` / `SUPABASE_PROJECT_NAME` | low | runtime | plain Railway variable (pins the `check:supabase-project` guard) | +| `INDEXING_V3_AGENT_SECRET` | high | runtime | Supabase Edge Function secrets | +| `RAG_QUERY_HASH_SECRET` | high | runtime | Railway variable store; GitHub repo secret (CI boot smoke) | +| `E2E_USER_EMAIL` / `E2E_USER_PASSWORD` | medium | CI only | GitHub repo secrets | Rules: - Secrets never enter images, the repo, or `NEXT_PUBLIC_*` names. `.env.local` - is a local-dev convenience only. + is a local-dev convenience only. Set runtime secrets as Railway service + variables (e.g. `railway variable set KEY --stdin` so the value is piped, never + echoed); Railway injects them at run time. +- `RAG_QUERY_HASH_SECRET` is **required in production** (`src/lib/env.ts` + `requireQueryHashSecret()` throws without it) and is generated fresh per + environment (`openssl rand -hex 32`); it is not copied from local dev. - Each environment (production, staging, CI) gets **separate** service-role and OpenAI keys so rotation and blast radius stay per-environment. - Rotation: publishable-key rotation is already an operator runbook item (`docs/archive/operator-decisions-2026-07-04.md`); service-role rotation is a - Supabase dashboard action + host secret update + redeploy. + Supabase dashboard action + Railway variable update + redeploy. - `npm run check:supabase-project` runs after any Supabase env change (repo rule), and the eval canary runs it before every scheduled eval. @@ -219,12 +322,15 @@ Rules: synthetic/public corpus. `public/demo-documents/` plus generated samples (`npm run samples`) are sufficient for load-shape realism; do not copy clinical production documents into staging. -- One staging app container + one staging worker container from the _same_ - images, different env. `RAG_PROVIDER_MODE=auto` with staging OpenAI key. +- One staging `app` container + one staging `worker` container from the _same_ + images, different env. On Railway this is naturally a **second environment in + the `clinical-kb` project** (e.g. a `staging` environment) or a separate + project, with `RAG_PROVIDER_MODE=auto` and the staging OpenAI key. See + `docs/staging-setup.md` for the turnkey runbook. - `src/lib/supabase/project.ts` is staging-aware only when both `SUPABASE_STAGING_PROJECT_REF` and `SUPABASE_STAGING_PROJECT_NAME` are set. The declared staging ref must differ from production and every stale project; - otherwise `check:supabase-project` fails closed. See `docs/staging-setup.md`. + otherwise `check:supabase-project` fails closed. - The soak test (`scripts/soak-test.ts`) targets staging **only** — see `docs/capacity-review.md`. @@ -232,12 +338,20 @@ Rules: - `.github/workflows/docker-image.yml` validates both container builds on `main`, release branches, a weekly schedule, and container-affecting pull - requests. It deliberately does not push to a registry; registry publication - and deployment remain host-specific release steps after the standard gates + requests. It deliberately does not push to a registry; Railway builds the + deployable image itself from the tree on deploy, after the standard gates (`verify` + `ui-smoke` + the clinical governance preflight where relevant). -- Rollback = redeploy the previous image tag. Database migrations follow the +- **Deploy:** `railway up --service app` / `--service worker` (or a connected + GitHub source) builds and releases. Railway does a rolling deploy and marks the + release `SUCCESS` only after the `/api/health` check passes. +- **Rollback = redeploy the previous Railway deployment** (`railway redeploy`, or + the dashboard's per-deployment rollback). Database migrations follow the existing rule: committed migrations + `schema.sql` reconciliation only, never raw SQL against live. - The nightly eval canary (`.github/workflows/eval-canary.yml`) is the standing guard that retrieval/answer quality did not silently regress after any deploy — see `docs/observability-slos.md`. +- **Observability:** Railway per-service metrics + logs (`railway logs`, + `railway metrics`) cover CPU/memory/HTTP; the app additionally emits + `Server-Timing` and `latencyTimings` (including `supabase_rpc_latency_ms`, + the cross-region signal from §2.1) on the answer path. diff --git a/docs/site-map.md b/docs/site-map.md index c4cdc4da7..91c2f58bc 100644 --- a/docs/site-map.md +++ b/docs/site-map.md @@ -559,6 +559,7 @@ This file is generated by `npm run sitemap:update`. Run `npm run sitemap:check` - `/api/documents/bulk/reindex` - Bulk reindex operation. Source: `src/app/api/documents/bulk/reindex/route.ts`. - `/api/eval-cases` - Evaluation case data. Source: `src/app/api/eval-cases/route.ts`. - `/api/health` - Health check. Source: `src/app/api/health/route.ts`. +- `/api/health/ready` - Route discovered from app directory Source: `src/app/api/health/ready/route.ts`. - `/api/images/[id]/signed-url` - Private image signed URL. Source: `src/app/api/images/[id]/signed-url/route.ts`. - `/api/ingestion/batches` - Ingestion batch state. Source: `src/app/api/ingestion/batches/route.ts`. - `/api/ingestion/jobs` - Ingestion job collection. Source: `src/app/api/ingestion/jobs/route.ts`. diff --git a/docs/staging-setup.md b/docs/staging-setup.md index 717801463..27be48374 100644 --- a/docs/staging-setup.md +++ b/docs/staging-setup.md @@ -25,14 +25,16 @@ those vars are unset. 2. **Apply the schema.** From a checkout linked to the staging project: - ```bash + ````bash supabase link --project-ref supabase db push # applies supabase/migrations/* → matches schema.sql - ``` + ```dotenv Then confirm health: `npm run check:indexing` (runs `search_schema_health()` over the hybrid RPCs) should report ok. + ```` + 3. **Seed a small corpus (~50 docs).** Use synthetic/public content only — do **not** copy clinical production documents into staging. @@ -51,44 +53,47 @@ those vars are unset. ## B. Staging app host (compute tier) -Recommended host: any OCI-image host. Production runs on **Railway**; Google -Cloud Run `australia-southeast2` is a Sydney-region alternative if lower -answer latency matters. No MCP is available here, so this is an operator step. +Host: **Railway**, same as production (see `docs/deployment-architecture.md` §2). +Stand staging up as a **second environment** in the `clinical-kb` Railway project +(e.g. a `staging` environment) or a separate project, with its own `app` (+ +optional `worker`) service pinned to **Southeast Asia (`asia-southeast1-eqsg3a`, +Singapore)** — the closest region to the staging Supabase project in Sydney. +Reuse the same images; only the environment variables differ. -1. **Build the image** from the committed `Dockerfile`, passing the _staging_ - publishable key (it inlines into the client bundle): +1. **Build the image** — Railway builds it remotely from the committed + `Dockerfile` on deploy, so there is no local `docker build` and no local + 8 GiB-heap OOM. The two `NEXT_PUBLIC_*` build args inline into the client + bundle; set them as **staging** service variables and Railway exposes them to + the build via the Dockerfile `ARG`s: - ```bash - docker build \ - --build-arg NEXT_PUBLIC_SUPABASE_URL=https://.supabase.co \ - --build-arg NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY= \ - -t clinical-kb-app:staging . ``` - - Note: local Docker Desktop can OOM on the 8 GiB Next build heap; prefer the - CI image-build workflow (builds on GitHub runners) if the local build wedges. + NEXT_PUBLIC_SUPABASE_URL = https://.supabase.co + NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY = + ``` 2. **Runtime secrets** (injected at deploy, never baked into the image) — all with **staging** values, distinct from production: - | Var | Value | - | ------------------------------- | --------------------------------- | - | `SUPABASE_SERVICE_ROLE_KEY` | staging `sb_secret_…` | - | `OPENAI_API_KEY` | an OpenAI key (staging or shared) | - | `SUPABASE_PROJECT_REF` | `` | - | `SUPABASE_PROJECT_NAME` | `Clinical KB Staging` | - | `SUPABASE_STAGING_PROJECT_REF` | `` | - | `SUPABASE_STAGING_PROJECT_NAME` | `Clinical KB Staging` | - | `RAG_QUERY_HASH_SECRET` | a staging secret | - | `RAG_PROVIDER_MODE` | `auto` | + | Var | Value | + | ------------------------------- | --------------------------------------------------------- | + | `SUPABASE_SERVICE_ROLE_KEY` | staging `sb_secret_…` | + | `OPENAI_API_KEY` | a staging-only OpenAI key; never reuse the production key | + | `SUPABASE_PROJECT_REF` | `` | + | `SUPABASE_PROJECT_NAME` | `Clinical KB Staging` | + | `SUPABASE_STAGING_PROJECT_REF` | `` | + | `SUPABASE_STAGING_PROJECT_NAME` | `Clinical KB Staging` | + | `RAG_QUERY_HASH_SECRET` | a staging secret | + | `RAG_PROVIDER_MODE` | `auto` | The two `SUPABASE_STAGING_PROJECT_*` vars are what make the identity guard accept the staging project. Setting `NEXT_PUBLIC_SUPABASE_URL` + `SUPABASE_PROJECT_REF` to the staging ref **without** them will (correctly) fail `check:supabase-project` — that's the deliberate speed bump. -3. **Host config:** bind `0.0.0.0:$PORT` (the Dockerfile already does), - health check `/api/health`, `min_machines_running=1`, no scale-to-zero. +3. **Service config:** the Dockerfile already binds `0.0.0.0:$PORT` (Railway + injects `$PORT`). Set the app service's healthcheck to `/api/health`, restart + policy to `ON_FAILURE`, and one replica pinned to `southeast-asia` with no + scale-to-zero (mirror `railway.app.json`). 4. **Worker (optional, for ingestion in staging):** build `Dockerfile.worker` and run one instance with the same staging secrets. Required to process the @@ -110,6 +115,8 @@ answer latency matters. No MCP is available here, so this is an operator step. ## What is operator-only (cannot be scripted here) - Creating the Supabase project (billable) and its DB password. -- Opening the host account (e.g. Railway) and setting the runtime secrets. +- Setting the staging runtime secrets in the Railway environment (service + variables). Writing admin credentials to Railway is an operator/authorized + action. - Any change to the **production** project's settings (e.g. auth percentage-based connection allocation — see `docs/capacity-review.md` §3). diff --git a/railway.app.json b/railway.app.json new file mode 100644 index 000000000..1d0253be4 --- /dev/null +++ b/railway.app.json @@ -0,0 +1,16 @@ +{ + "$schema": "https://railway.com/railway.schema.json", + "build": { + "builder": "DOCKERFILE", + "dockerfilePath": "Dockerfile" + }, + "deploy": { + "healthcheckPath": "/api/health/ready", + "restartPolicyType": "ON_FAILURE", + "multiRegionConfig": { + "asia-southeast1-eqsg3a": { + "numReplicas": 1 + } + } + } +} diff --git a/railway.worker.json b/railway.worker.json new file mode 100644 index 000000000..41edf20a5 --- /dev/null +++ b/railway.worker.json @@ -0,0 +1,15 @@ +{ + "$schema": "https://railway.com/railway.schema.json", + "build": { + "builder": "DOCKERFILE", + "dockerfilePath": "Dockerfile.worker" + }, + "deploy": { + "restartPolicyType": "ON_FAILURE", + "multiRegionConfig": { + "asia-southeast1-eqsg3a": { + "numReplicas": 1 + } + } + } +} diff --git a/src/app/api/health/ready/route.ts b/src/app/api/health/ready/route.ts new file mode 100644 index 000000000..8cc8cd7bd --- /dev/null +++ b/src/app/api/health/ready/route.ts @@ -0,0 +1,10 @@ +import { healthResponse } from "@/lib/health-response"; + +export const runtime = "nodejs"; +export const dynamic = "force-dynamic"; + +// Railway cannot attach the authenticated deep-probe header. This endpoint is +// intentionally limited to readiness state and exposes no diagnostic details. +export async function GET(request: Request) { + return healthResponse(request, { forceDeep: true, allowUnauthenticatedDeep: true, includeSlo: false }); +} diff --git a/src/app/api/health/route.ts b/src/app/api/health/route.ts index b53de6313..df3d9eee4 100644 --- a/src/app/api/health/route.ts +++ b/src/app/api/health/route.ts @@ -1,64 +1,8 @@ -import { NextResponse } from "next/server"; -import { env, isDemoMode } from "@/lib/env"; -import type { AnswerSloSnapshot } from "@/lib/observability/answer-slo"; -import { allowDeepHealthProbe } from "@/lib/deep-probe-auth"; +import { healthResponse } from "@/lib/health-response"; export const runtime = "nodejs"; export const dynamic = "force-dynamic"; export async function GET(request: Request) { - const deep = new URL(request.url).searchParams.get("deep") === "1"; - const supabaseConfigured = Boolean(env.NEXT_PUBLIC_SUPABASE_URL && env.SUPABASE_SERVICE_ROLE_KEY); - - const checks: Record = { - supabaseConfig: supabaseConfigured ? "ok" : "missing", - openaiConfig: env.OPENAI_API_KEY ? "ok" : "missing", - }; - - let slo: AnswerSloSnapshot | null = null; - if (deep) { - if (!allowDeepHealthProbe(request)) { - checks.supabase = "unauthorized"; - } else if (supabaseConfigured && !isDemoMode()) { - try { - const [{ createAdminClient }, { probeSupabaseHealth }, { answerSloSnapshot }] = await Promise.all([ - import("@/lib/supabase/admin"), - import("@/lib/supabase/health"), - import("@/lib/observability/answer-slo"), - ]); - const admin = createAdminClient(); - const health = await probeSupabaseHealth(admin); - checks.supabase = health.ok ? "ok" : "error"; - if (health.ok) { - // Reliability telemetry only — a failure here must NOT flip liveness to - // 503, so it stays out of `checks` (which gates `ready`). - try { - slo = await answerSloSnapshot(admin); - } catch { - slo = null; - } - } - } catch { - checks.supabase = "error"; - } - } else { - checks.supabase = "skipped"; - } - } - - const ready = !Object.values(checks).some( - (value) => value === "missing" || value === "error" || value === "unauthorized", - ); - - return NextResponse.json( - { - status: ready ? "ok" : "degraded", - demoMode: isDemoMode(), - timestamp: new Date().toISOString(), - uptimeSeconds: Math.round(process.uptime()), - checks, - ...(slo ? { slo } : {}), - }, - { status: ready ? 200 : 503, headers: { "Cache-Control": "no-store" } }, - ); + return healthResponse(request); } diff --git a/src/app/globals.css b/src/app/globals.css index f673e0e84..4123314b3 100644 --- a/src/app/globals.css +++ b/src/app/globals.css @@ -1407,6 +1407,17 @@ summary::-webkit-details-marker { } } +@media (min-width: 1024px) { + /* The document viewer is a two-column layout (main + 480px right rail). Keep + the floating composer within the main column so it never overlaps the + right-rail evidence/profile/tables text. Scoped to lg+, where the rail sits + beside the main column. */ + .document-viewer-composer.floating-composer-edge { + left: max(2rem, var(--safe-area-left)); + right: calc(max(0px, (100vw - 1440px) / 2) + 480px + 3.5rem + var(--safe-area-right)); + } +} + @media (max-width: 639px) { .edge-glass-header { padding-left: max(0px, var(--safe-area-left)); diff --git a/src/components/AccessibleTable.tsx b/src/components/AccessibleTable.tsx index 3afe28f49..5c7da24ab 100644 --- a/src/components/AccessibleTable.tsx +++ b/src/components/AccessibleTable.tsx @@ -115,7 +115,13 @@ function AccessibleTableMarkup({ @@ -135,7 +141,7 @@ function AccessibleTableMarkup({ "nums border-b border-[color:var(--border)] align-top font-semibold leading-5 text-[color:var(--text)]", renderDensePreview ? "overflow-hidden text-ellipsis whitespace-nowrap" - : "whitespace-normal break-words [overflow-wrap:anywhere]", + : "whitespace-normal break-words", index > 0 && "border-l border-[color:var(--border)]/70", renderDensePreview ? "px-2 py-1.5 text-3xs uppercase tracking-[0.06em]" @@ -152,7 +158,9 @@ function AccessibleTableMarkup({ scope="col" className={cn( "nums border-b border-l border-[color:var(--border)]/70 align-top font-semibold leading-5 text-[color:var(--text)]", - "whitespace-normal break-words [overflow-wrap:anywhere]", + renderDensePreview + ? "overflow-hidden text-ellipsis whitespace-nowrap" + : "whitespace-normal break-words", renderDensePreview ? "px-2 py-1.5 text-3xs uppercase tracking-[0.06em]" : expanded @@ -190,9 +198,7 @@ function AccessibleTableMarkup({ className={cn( "nums align-top text-[color:var(--text)]", renderDensePreview ? "table-cell" : "block md:table-cell", - renderDensePreview - ? "overflow-hidden whitespace-nowrap" - : "whitespace-pre-wrap break-words [overflow-wrap:anywhere]", + renderDensePreview ? "overflow-hidden whitespace-nowrap" : "whitespace-pre-wrap break-words", renderDensePreview ? "border-t border-[color:var(--border)]/70 px-2 py-1.5 leading-4" : "border-b border-[color:var(--border)]/60 pb-2 last:border-b-0 md:border-b-0 md:border-t md:border-[color:var(--border)]/70 md:last:border-b-0", diff --git a/src/components/DocumentViewer.tsx b/src/components/DocumentViewer.tsx index aa9d1ccfb..6bbba325a 100644 --- a/src/components/DocumentViewer.tsx +++ b/src/components/DocumentViewer.tsx @@ -238,11 +238,8 @@ function ClinicalSummaryProfile({ profile }: { profile: ClinicalDocumentSummaryP return (
- {profile.overview ? ( -

- -

- ) : null} + {/* The overview sentence leads the document landing card; the profile here + shows only the structured detail so the same text is not printed twice. */} {sections.map((section) => (

@@ -278,9 +275,18 @@ function ClinicalSummaryProfile({ profile }: { profile: ClinicalDocumentSummaryP const collapsedSummarySectionCap = 4; const collapsedSummaryItemCap = 5; -function FormattedHighYieldSummary({ formatted }: { formatted: FormattedDocumentSummaryModel }) { +function FormattedHighYieldSummary({ + formatted, + showLead = true, +}: { + formatted: FormattedDocumentSummaryModel; + showLead?: boolean; +}) { const [expanded, setExpanded] = useState(false); if (formatted.isEmpty) return null; + // The lead sentence already leads the document landing "Overview" card, so the + // right rail can suppress it to avoid printing the same sentence twice. + const leadVisible = showLead && Boolean(formatted.lead); const visibleSections = expanded ? formatted.sections @@ -293,15 +299,15 @@ function FormattedHighYieldSummary({ formatted }: { formatted: FormattedDocument return (
- {formatted.lead ? ( + {leadVisible ? (

- +

) : null} {visibleSections.map((section, index) => (
0) && "border-t border-[color:var(--border)] pt-3")} + className={cn((leadVisible || index > 0) && "border-t border-[color:var(--border)] pt-3")} >

{section.heading ?? "Key points"} @@ -455,14 +461,10 @@ function DocumentImage({ image }: { image: ImageRow }) { .join(" "), ); - const clinicalUseReasonLine = - image.clinicalUseClass && image.clinicalUseClass !== "clinical_evidence" && image.clinicalUseReason ? ( -

{image.clinicalUseReason}

- ) : null; - - // The image preview and its extracted table are the same content; when a - // structured table is available it leads and the raw image collapses behind a - // disclosure so the card is not doubled in height. + // When a structured, accessible table exists the extracted table leads and the + // raw 4:3 table image collapses behind a "Show original" toggle, so the same + // content is not shown twice at full height. Without a structured table the + // image is the only representation, so it stays inline and prominent. const imageBlock = (
{failed ? ( @@ -514,6 +516,27 @@ function DocumentImage({ image }: { image: ImageRow }) {
); + const figcaptionBlock = ( +
+ {tableHeading ?

{tableHeading}

: null} + {showImageCaptionLine ?

{cleanCaption}

: null} + + {!hasStructuredTable && image.tableTextSnippet ? ( +

{image.tableTextSnippet}

+ ) : null} + {image.clinicalUseClass && image.clinicalUseClass !== "clinical_evidence" && image.clinicalUseReason ? ( +

{image.clinicalUseReason}

+ ) : null} +
+ ); return (

@@ -526,25 +549,12 @@ function DocumentImage({ image }: { image: ImageRow }) {

{hasStructuredTable ? ( <> -
- {tableHeading ?

{tableHeading}

: null} - {showImageCaptionLine ?

{cleanCaption}

: null} - - {clinicalUseReasonLine} -
+ {figcaptionBlock}
- + +
{imageBlock}
@@ -552,14 +562,7 @@ function DocumentImage({ image }: { image: ImageRow }) { ) : ( <>
{imageBlock}
-
- {tableHeading ?

{tableHeading}

: null} - {showImageCaptionLine ?

{cleanCaption}

: null} - {image.tableTextSnippet ? ( -

{image.tableTextSnippet}

- ) : null} - {clinicalUseReasonLine} -
+ {figcaptionBlock} )} {displayLabels.length ? ( @@ -708,15 +711,17 @@ function PinnedSourceEvidence({ : ""; if (!loading && !chunk) { - // Direct visits (not arrived via a cited answer passage) have nothing to - // pin, so keep this a slim, muted note rather than a prominent empty card. + // Nothing is pinned (e.g. a direct visit, not arrived-at via a citation), so + // this stays a quiet one-line hint rather than a full card taking prime space. return ( -
-

-

-
+

+

); } @@ -1341,51 +1346,51 @@ function PdfCanvasViewer({ url, title, initialPage }: { url: string; title: stri >
-
- - {pagesReady ? ( - - ) : ( -
-
- )} - -
-
+ + {pagesReady ? ( + + ) : ( +
+
+ )} + +
@@ -2869,7 +2910,7 @@ export function DocumentViewer({
-