Skip to content

fix(server): clear image CVEs — bump bundled tunnel agents + kin-openapi - #194

Merged
dvcdsys merged 1 commit into
developfrom
fix/tunnel-agent-cves
Jul 24, 2026
Merged

fix(server): clear image CVEs — bump bundled tunnel agents + kin-openapi#194
dvcdsys merged 1 commit into
developfrom
fix/tunnel-agent-cves

Conversation

@dvcdsys

@dvcdsys dvcdsys commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Why

Docker Hub showed every published tag carrying a heavy vulnerability load. A trivy scan of the released images confirmed it — and pinned it to a single file.

:latest and :v0.12.5-cu128, identical results, 3 CRITICAL + 47 HIGH:

artefact CRITICAL+HIGH
/cloudflared 46
/ngrok 3
/cix-server 1
distroless base (OS packages) 0
CUDA libs + llama.cpp .so 0

The bases were clean the whole time. Both images show the same numbers because the findings live entirely in the two bundled tunnel agents, which are byte-identical across CPU and CUDA.

cloudflared was pinned at 2025.2.1 when Managed Tunnels landed (2300997, 2026-05-20) and was never bumped — upstream is now at 2026.7.3. Cloudflare built 2025.2.1 with Go 1.22.10, which is where all 46 come from: stdlib (15), x/crypto 0.31.0 (11), coredns 1.11.3 (9), x/net 0.26.0 (5), grpc 1.63.2 (2, incl. CRITICAL CVE-2026-33186).

Neither automated gate could ever have caught this. Dependabot's docker ecosystem only follows FROM images, so an ARG + curl is invisible to it. security.yml runs govulncheck and Trivy in filesystem mode — both scan the source tree, and this binary only exists inside the built image. There is no image-level scan in CI at all; make scout-* is manual.

What changed

  • cloudflared 2025.2.1 → 2026.7.3 in both Dockerfiles and in internal/tunnels/installer.go. That const is the second half of the pin — it drives the dashboard's Install/Update button, so bumping only the Dockerfiles would have left operators re-installing the old binary over the fresh one.
  • Checksum verification is now mandatory. Previously CLOUDFLARED_SHA256 defaulted to "" and the check was skipped; pinnedSHA256 in the installer was an empty map, so every managed download logged "downloaded without checksum verification". Both Dockerfiles now pin per-arch SHA-256 (an unknown arch fails the build), and the installer map covers all four cloudflared platforms.
    ngrok is deliberately left out of that map: its v3-stable channel has no per-version URL, so a pinned sum would start rejecting legitimate downloads the moment ngrok publishes.
  • kin-openapi v0.142.0 → v0.144.0 — the one finding in cix-server itself (GHSA-r277-6w6q-xmqw, fail-open auth bypass in ValidationHandler). Not reachable here: we only import openapi3 from openapi.gen.go and never construct a ValidationHandler, and govulncheck reported 0 reachable both before and after. Bumped anyway — it is free and it clears the counter.
  • NGROK_NOCACHE on the local docker build targets. A floating channel still freezes once its RUN layer is in the buildx cache; CI runners have no cache so they were always fetching fresh, but local/dev-builder rebuilds were not.

Verification

Built and scanned on the native x86 host (both images, --load, no push):

image before after
CPU 3 CRITICAL / 47 HIGH 0 CRITICAL / 5 HIGH
CUDA 3 CRITICAL / 47 HIGH 0 CRITICAL / 5 HIGH

cix-server itself: 0 findings. The 5 that remain are upstream vendor lag we do not control — cloudflared and ngrok are both built on Go 1.26.4 (needs 1.26.5) and ship grpc slightly behind, plus dd-trace-go in ngrok.

Smoke tests on the same host:

  • cloudflared --version2026.7.3 (built 2026-07-23) in both images; ngrok version3.39.10
  • CPU container healthy in ~10s, /health{"status":"ok"}
  • GET /api/v1/tunnels/binaries reports the new versions for both providers
  • POST /api/v1/tunnels/binaries/cloudflare/install → log line changed from the old warn to tunnel binary checksum verified, confirming both halves of the pin agree
  • CUDA container healthy in ~10s on --gpus all; GPU attribution verified — its llama-server pid appears in nvidia-smi --query-compute-apps at 498 MiB, so no silent CPU fallback
  • go test ./... 42 packages pass; go vet clean; govulncheck 0 reachable (module-level findings 3 → 1)

Throwaway containers, volumes and images were removed afterwards; the running prod stack was not touched.

Not in this PR

Two follow-ups that stop this class of drift from recurring, kept separate so this one stays a clean dependency bump:

  1. A cloudflared-pin-check.yml modelled on the existing llama-pin-check.yml — weekly comparison against upstream, opens a tracking issue when the pin goes stale.
  2. An image-level scan in CI with the same gate already applied by hand: no NEW HIGH/CRITICAL relative to the previous release tag.

🤖 Generated with Claude Code

…CVEs

The published images carried 3 CRITICAL + 47 HIGH findings (trivy, identical
on both :latest and :v0.12.5-cu128). 46 of the 50 came from a single file:
the bundled `cloudflared`, pinned at 2025.2.1 since Managed Tunnels landed on
2026-05-20 and never bumped. Cloudflare built that release with Go 1.22.10, so
it dragged in stdlib (15), x/crypto 0.31.0 (11), coredns 1.11.3 (9),
x/net 0.26.0 (5) and grpc 1.63.2 (2, incl. CRITICAL CVE-2026-33186).

Neither automated gate could see it: Dependabot's docker ecosystem only tracks
`FROM` images, and the Security workflow runs govulncheck plus Trivy in
FILESYSTEM mode — a binary curl'd during the image build appears in neither.
The distroless bases and the CUDA/llama.cpp layers were clean throughout.

Changes:
  - cloudflared 2025.2.1 → 2026.7.3 in both Dockerfiles AND in
    internal/tunnels/installer.go. The const there is the managed-install
    path used by the dashboard's Update button; leaving it stale would have
    re-installed the old binary over the freshly bundled one.
  - Checksum verification is now mandatory, not opt-in. Both Dockerfiles pin
    per-arch SHA-256 (unknown arch fails the build), and pinnedSHA256 in the
    installer is populated for all four cloudflared platforms. ngrok stays
    absent from that map on purpose: its v3-stable channel has no per-version
    URL, so a pinned sum would reject legitimate downloads.
  - kin-openapi v0.142.0 → v0.144.0, clearing the one finding in cix-server
    itself (GHSA-r277-6w6q-xmqw, fail-open auth bypass in ValidationHandler).
    Not reachable here — we only import openapi3 in openapi.gen.go and never
    construct a ValidationHandler — but it is a free bump.
  - NGROK_NOCACHE on the local docker build targets. ngrok's floating channel
    would otherwise stay frozen in the buildx layer cache; CI runners have no
    cache so they were always fetching fresh.

Follow-up (separate PR): a cloudflared pin-freshness workflow modelled on
llama-pin-check.yml, and an image-level scan in CI so this class of drift
cannot go unnoticed again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dvcdsys
dvcdsys merged commit 003fb30 into develop Jul 24, 2026
1 check passed
@dvcdsys
dvcdsys deleted the fix/tunnel-agent-cves branch July 24, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant