[investigate] Baseline tests for #14198 (remote-bindings Access auth)

[petebacondarwin]

Discussion / investigation PR — companion to #14198. Not intended to merge as-is. The goal is to share three baseline-characterisation tests that run on main (without #14198 applied), and use their empirical results to discuss whether #14198's miniflare-side credential plumbing is necessary, and what the genuine outstanding gap is.

cc @krys-cf @petebacondarwin

TL;DR

• All three tests pass on main without [miniflare] Authenticate remote bindings with Cloudflare Access service tokens #14198 applied.
• The decisive result (Test 2) shows that the ProxyWorker→edge hop already carries arbitrary headers, including the proxyData-merged ones, on a WebSocket upgrade. Combined with Test 1's confirmation that RemoteRuntimeController always injects getAccessHeaders(token.host) into proxyData.headers, this means the edge hop already authenticates remote-binding traffic — HTTP and WS alike — with the existing service-token env vars.
• Test 3 demonstrates a real, separate gap: when the proxy returns a Cloudflare Access block on a WebSocket upgrade, the RPC call fails opaquely with no warning — in contrast to [miniflare] Warn when remote-bindings requests are blocked by Cloudflare Access #14011's HTTP-path warning. This gap is unrelated to authentication and worth addressing on its own.

If these results hold up under closer scrutiny / a debug repro from @krys-cf, then #14198's two mechanisms (HTTP creds in makeFetch and the fetch()-based WS upgrade in makeRemoteProxyStub) appear to be redundant for the auth they target, and the AI InferenceUpstreamError: invalid_token reported in #14198 may be a separate AI-upstream concern rather than a Cloudflare Access failure.

Why this matters / topology recap

remoteProxyConnectionString is not the workers.dev URL — it resolves to the local ProxyController Miniflare (http://127.0.0.1:<port>, start-remote-proxy-session.ts:160-161 → DevEnv.ts:215-217 → ProxyController.ts:188 with no host override, defaulting to 127.0.0.1). The Access-protected host is token.host (*.workers.dev), one hop further in.

User worker (Miniflare A)
  └─ remote-proxy-client.worker   ─ws/http─▶   http://127.0.0.1:<port>     ← LOCAL loopback
       Local ProxyWorker (Miniflare B)         [ProxyWorker.ts]
         processQueue:
           headers = new Headers(request.headers);
           for (...) headers.set(key, value);       ← merges proxyData.headers
           fetch(userWorkerUrl, new Request(request, { headers }));
                                          ─▶  https://<token.host>.workers.dev   ← Access-protected
            ProxyServerWorker (edge)
              ├─ newWorkersRpcResponse  (capnweb / WS-RPC)
              └─ fetcher.fetch          (raw HTTP bindings)

The only Access-gated hop is B→edge. RemoteRuntimeController.ts:325 already calls getAccessHeaders(token.host) and spreads it into proxyData.headers (:333-337). ProxyWorker.processQueue unconditionally merges those into every forwarded request (ProxyWorker.ts:146-158), including WS upgrades.

The remaining question — and the entire reason I wrote Test 2 — is whether workerd actually carries those merged headers on a WS upgrade, or whether something subtle (e.g. new Response(res.body, res) at ProxyWorker.ts:160 dropping the webSocket) causes the WS path to silently lose them. The workerd C++ source (Response::constructor in src/workerd/api/http.c++) preserves both webSocket and statusCode when constructed from another Response, but a runtime check is more decisive than reading C++.

The three tests + their empirical results

Test 1 — RemoteRuntimeController puts Access headers in proxyData

File: packages/wrangler/src/__tests__/api/startDevWorker/RemoteRuntimeController.test.ts (+91 lines)
Run: pnpm test:ci -F wrangler -- RemoteRuntimeController
Result: ✅ passes on main. The controller calls getAccessHeaders(token.host) on every reload (including remote: "minimal") and spreads the result into proxyData.headers alongside the preview token. Verified for both auth modes (service-token pair and Cookie: CF_Authorization=... from cloudflared).

Test 2 — workerd forwards headers on a WS upgrade (the linchpin)

File: packages/miniflare/test/plugins/shared/proxy-websocket-header-forwarding.spec.ts (new)
Run: pnpm --filter miniflare exec vitest run test/plugins/shared/proxy-websocket-header-forwarding.spec.ts
Result: ✅ passes on main — "both sentinels" outcome.

The test replicates ProxyWorker.processQueue's exact forward line:

const headers = new Headers(request.headers);
headers.set("x-proxydata-sentinel", "from-proxydata");
return fetch(env.EDGE_URL, new Request(request, { headers }));

…sends an Upgrade: websocket request with x-incoming-sentinel: from-client, and a fake WS server (via useServer(_, wsListener)) captures the upgrade req.headers. Both sentinels arrive at the edge:

Edge receives	Verdict
both ← actual result	Existing path already auths WS → #14198's WS fetch-upgrade is redundant for reaching the edge.
only incoming	proxyData headers dropped on WS → #14198's mechanism would be justified.
only proxyData	Client-side header wouldn't propagate.
neither	WS forwarding broken → AI invalid_token is a separate issue.

Test 3 — WS/RPC path has no Access-block detection

File: packages/miniflare/test/plugins/shared/remote-bindings-access-warning.spec.ts (+87 lines)
Run: pnpm --filter miniflare exec vitest run test/plugins/shared/remote-bindings-access-warning.spec.ts
Result: ✅ passes on main — confirms the gap.

A fake proxy server returns a 403 Cloudflare Access block (the same HTML body the existing HTTP-path tests use) for every request, including WS upgrades. A worker script invokes a non-.fetch RPC method on the service binding (so the proxy stub takes the capnweb / WebSocket path, not makeFetch). The RPC call fails (response.status === 500, body err:…) and no Access warning is logged — in contrast to the HTTP path which produces a single, actionable warning + readable error body courtesy of #14011's maybeReportCloudflareAccessBlock. The capnweb path in makeRemoteProxyStub has no equivalent.

What this implies for #14198

If the empirical results above hold up under @krys-cf's repro:

1. Authentication isn't actually missing on either remote-binding path (HTTP or WS) — proxyData.headers already carries CF-Access-Client-Id/Secret (or Cookie) to the edge. The PR's HTTP-side credential injection in makeFetch and its WS-side fetch()-based upgrade in makeRemoteProxyStub would, in this reading, be attaching Access headers on the localhost loopback hop where Access is not enforced.
2. The AI InferenceUpstreamError: invalid_token reported in [miniflare] Authenticate remote bindings with Cloudflare Access service tokens #14198 may not be a Cloudflare Access failure — it's an AI-upstream / Inference Gateway error code. Worth confirming separately.
3. The real gap is detection/UX on the WS/RPC path: Access blocks there are opaque. The cleanest follow-up — independent of the auth question — is to wire maybeReportCloudflareAccessBlock (or equivalent) into the capnweb path so RPC bindings surface the same single, actionable warning the HTTP path already does.

If, on the other hand, @krys-cf can reproduce the failure with the PR reverted, env creds set, and WRANGLER_LOG=debug confirming Using Access Service Token headers for domain: <token.host> is logged for the proxy session — then there's a real bug somewhere that my static + isolated tests have missed, and we should hunt for it before plumbing more credentials.

Specific things I'd love a debug capture of

If @krys-cf has time, the most useful data point would be: with the PR fully reverted, env creds set, against the failing repro, WRANGLER_LOG=debug output capturing:

1. Whether getAccessHeaders logs Using Access Service Token headers for domain: <token.host> for the proxy session (access.ts:111). If yes, proxyData should carry them, and the question is what happens after the merge.
2. The exact failing response — a 403 Cloudflare Access HTML body, or the AI InferenceUpstreamError: invalid_token text? These are different failure modes.
3. Which binding actually fails first — a wrapped-fetcher binding (AI/Vectorize/Images, HTTP path) or a JSRPC binding (Artifacts/service, WS path)?

That single capture would distinguish "real bug in B→edge for some bindings" from "AI-upstream auth issue conflated with Access" from "Access is the culprit for some other reason I haven't traced".

---

Fixes #N/A — discussion PR; not intended to merge as-is.

• Tests
  • Tests included/updated
• Public documentation
  • Documentation not necessary because: this PR is investigation-only and adds tests that characterise existing behaviour — no shipped behaviour changes.

[changeset-bot]

⚠️ No Changeset found

Latest commit: dafc251

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[ask-bonk]

This confirms Test 3's premise: someRpcMethod() (non-.fetch) goes through newWebSocketRpcSession(url.href) (capnweb/WS), which bypasses makeFetch's maybeReportCloudflareAccessBlock. So the test accurately exercises the gap. The test is correct.

I've completed a thorough static review. Let me summarize my findings:

The tests are well-constructed and accurately mirror the production code:

• Test 1 correctly reflects RemoteRuntimeController.ts:325-335
• Test 2 faithfully mirrors ProxyWorker.ts:121,158
• Test 3 correctly exercises the capnweb/WS path via makeRemoteProxyStub (remote-bindings-utils.ts:195)

No logic bugs, security issues, backward-compat violations, or incorrect API behavior. The only observations are:

1. These are intentional characterisation tests for an investigation PR explicitly not meant to merge as-is.
2. Test 3 asserts the absence of a warning (toHaveLength(0)), which would break once the gap it documents is fixed — but the author explicitly frames this as documenting current behavior on main.

These are characteristics of an investigation PR, not actionable defects. The author clearly states this PR is not intended to merge as-is. There are no logic bugs, security issues, or incorrect API behaviors to flag, and I was instructed not to flag formatting or subjective preferences.

LGTM

github run

[pkg-pr-new]

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@14234

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/@cloudflare/deploy-helpers@14234

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@14234

miniflare

npm i https://pkg.pr.new/miniflare@14234

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@14234

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@14234

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@14234

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@14234

@cloudflare/workers-auth

npm i https://pkg.pr.new/@cloudflare/workers-auth@14234

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@14234

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@14234

wrangler

npm i https://pkg.pr.new/wrangler@14234

@cloudflare/wrangler-bundler

npm i https://pkg.pr.new/@cloudflare/wrangler-bundler@14234

commit: dafc251

[petebacondarwin]

Just added a minimal manual E2E repro at investigation/remote-access-repro/ (dafc251c) so the central question can be settled against a real Access-protected account without the moving parts of de-watchtower.

It's deliberately tiny: one deployed target worker (Api.add(a,b) WorkerEntrypoint + default fetch) and one wrangler dev worker that exercises all three proxy paths in a single GET and reports each independently:

{
  "ai":           { "path": "http/makeFetch (AI)",                  "ok": …, "result|error": … },
  "serviceFetch": { "path": "http/makeFetch (service.fetch)",       "ok": …, "result|error": … },
  "rpc":          { "path": "ws/makeRemoteProxyStub (service RPC)", "ok": …, "result|error": … }
}

So one curl tells us which proxy path fails and how — HTTP wrapped-fetcher, HTTP service .fetch, and WebSocket/capnweb RPC — without having to disentangle them from a real app.

Also commits a throwaway debug log in ProxyWorker.processQueue (grep mf-access-debug). For every request the local ProxyWorker forwards to the edge — HTTP and WebSocket upgrades — it logs the target URL plus whether the merged headers carry CF-Access-Client-Id / cookie / cf-workers-preview-token. That's the direct, unambiguous answer to "do the Access headers reach the edge on the WS hop?". To be reverted before any real fix lands; left in for the duration of this investigation PR.

The decisive run is scenario B (main + creds set) — full protocol + interpretation table in the README. It runs against the locally built wrangler so switching main ↔ #14198 is just git checkout … && pnpm build.

The repro lives outside the pnpm workspace globs (CI, lint, type-check, fixture-validation, turbo all skip it).

[krys-cf]

Cross-linking from #14198 (closing that one). End-to-end verification confirms your baseline conclusion: on current released versions (wrangler@4.99.0 / miniflare@4.20260609.0 / @cloudflare/vite-plugin@1.40.1) remote bindings work behind Cloudflare Access without extra auth-plumbing.

I reproduced it in a minimal vite-plugin app (passes with a service token in .env; fails fast with a clear non-interactive "no service token" error without one) and in the real app that originally hit this, after removing its local hand-rolled patch and upgrading to bare latest. So #14198's miniflare connStr-carrier is redundant; the one gap worth keeping in mind is UX — a clear warning/error when a remote-binding proxy is Access-gated and no creds are present (which released wrangler now does on startup).

[petebacondarwin]

Thanks for folllowing up @krys-cf - I'll close this out. Glad you are unblocked!