feat(server): Managed Tunnels (Cloudflare + ngrok) for webhook ingress behind NAT - #45
Merged
Conversation
…s behind NAT Adds a built-in, dashboard-managed outbound tunnel so a NAT'd server gets a public URL for GitHub webhook delivery — no inbound ports, no env feature flag. Backend (internal/tunnels): - Provider abstraction + Manager with runtime Apply (start/stop/reconfigure), mirroring the llama-server sidecar supervisor (exec, exit-watcher, bounded crash-restart, graceful SIGTERM->SIGKILL). - Cloudflare provider (named/quick) and ngrok provider (reserved/ephemeral; authtoken always required; URL parsed from the agent JSON log). - Webhook reconciler: re-points webhook_mode=auto repos at the live tunnel URL on boot and on URL change; new githubapi.UpdateWebhook (PATCH). - buildWebhookURL prefers the live tunnel URL over CIX_PUBLIC_URL. Config in DB, managed from the dashboard (no env flag): - tunnel_config single-row table (migration #8); token encrypted via secrets. - tunnelcfg service + GET/PUT /api/v1/tunnels/config (admin); provider-aware validation. Only deployment infra stays in env (binary paths, metrics addr). Binary management: - GET /api/v1/tunnels/binaries reports installed/path/version/managed. - When missing locally, the dashboard shows manual install instructions (brew/linux) noting Docker bundles them automatically. - CIX_TUNNEL_BIN_MANAGED (true in the Docker images) enables Install/Update: the server downloads agents (raw or .tgz, no shell) into a writable /data/tunnel-bin via POST /api/v1/tunnels/binaries/{provider}/{install,update}. Dashboard: - New admin-only "Managed Tunnels" section (provider select, mode, hostname, token, status, Test/Restart, binary install/update). - "GitHub Tokens" -> "GitHub Integration" with Tokens + Webhook Integrations tabs. Docker: both images bundle cloudflared + ngrok and enable managed updates. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…, races - High: require admin on TestTunnel/RestartTunnel/ReconcileWebhooks — these decrypt PATs, register GitHub webhooks, restart the subprocess, or probe outbound, and were only behind requireAuth (any authenticated user). Hide the dashboard "Re-register webhooks" button from non-admins. - Medium: pass the Cloudflare/ngrok tokens via env (TUNNEL_TOKEN / NGROK_AUTHTOKEN) instead of argv so they don't show in ps/proc cmdline. - Medium: installer no longer tracks "latest" — pin cloudflared version (synced with the Dockerfile arg); compute the download SHA-256, verify it against a pinned map when present and warn (with the sum) when not; cap the download/extraction size to guard against a decompression bomb. - Low: serialize Reconcile (mutex) so the boot double-reconcile can't create duplicate GitHub hooks; serialize Manager.Apply so concurrent applies can't orphan a provider; RestartTunnel uses a background-derived ctx so a client disconnect doesn't abort the spawn (matching UpdateTunnelConfig); capture readySignal under the lock before close to avoid a restart-time race. - Nits: drop stale CIX_TUNNEL_ENABLED wording from handler messages/comments. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Built-in, dashboard-managed outbound tunnel so a NAT'd server (the RTX 3090 box) gets a public URL for GitHub webhook delivery — no inbound ports, no env feature flag. We rejected
gh webhook forward(dev/test only) in favour of production-grade agents. Provider-pluggable: Cloudflare Tunnel and ngrok both implemented.What's included
Tunnel engine (
server/internal/tunnels)Managerwith runtimeApply(start/stop/reconfigure), mirroring the llama-server sidecar supervisor (exec, exit-watcher, bounded crash-restart, graceful SIGTERM→SIGKILL).webhook_mode=autorepos at the live tunnel URL on boot and whenever it changes; newgithubapi.UpdateWebhook(PATCH).buildWebhookURLnow prefers the live tunnel URL overCIX_PUBLIC_URL.Config in DB, managed from the dashboard (no env flag)
tunnel_configsingle-row table (migration Bump actions/download-artifact from 4 to 8 #8); connector token encrypted viasecrets.tunnelcfgservice +GET/PUT /api/v1/tunnels/config(admin); provider-aware validation. Only deployment infra stays in env (binary paths, metrics addr, startup timeout).Binary management
GET /api/v1/tunnels/binariesreports installed/path/version/managed.CIX_TUNNEL_BIN_MANAGED(true in the Docker images) enables Install/Update — the Go server downloads agents (raw or.tgz, no shell, distroless-safe) into a writable/data/tunnel-bin, atomic rename.POST /api/v1/tunnels/binaries/{provider}/{install,update}.Dashboard
Docker — both images bundle
cloudflared+ngrokand enable managed updates.Security notes (for reviewers)
localhost:<port>exposes the whole server, not just the webhook path. Webhook is HMAC-protected; the rest is not. Recommend Cloudflare named mode with an ingress rule restricting to/api/v1/webhooks/github/*; quick/ephemeral modes are dev-only.cloudflared/ngrokadd binary + CVE surface → runmake scout-cpu/make scout-cuda(0 HIGH/CRITICAL gate) before any prod push.Test plan
go build/vet/test ./...green; unit tests for UpdateWebhook, manager Apply, reconciler, ngrok URL parse, installer (tgz/raw/asset URLs)tsc -b && vite buildgreen;dist/.gitkeeppreservedfailed(not 500), managed vs unmanaged binaries (409 when disabled)/healthround-trip succeeds (needs the agents installed + network)cloudflaredversion pin + ngrok channel) andmake scout-*clean🤖 Generated with Claude Code