feat(diagnostics): add independent runtime MCP verification with detailed failure attribution - #3199
Merged
reachjalil merged 1 commit intoJul 27, 2026
Conversation
…iled failure attribution Extend the Agent Context cloud catalog probe into a full independent runtime MCP verification of the managed openwork-cloud endpoint: - Name the runtime transport explicitly (runtimeDiagnosticFetch + runtimeDiagnosticTransportInfo in server-fetch.ts) so reports state whether Node/undici (Electron main or standalone Node), Bun's fetch, or a test seam performed the probe, and harden the bare-fetch guard so reference fallbacks such as `?? fetch` can no longer bypass it. - Decouple probe eligibility from cached OpenCode registration state and from agent tool policy; both remain reported dimensions and become comparison inputs for a new differential verdict instead of gates. - Run the complete bounded MCP handshake: initialize (with protocol version validation), notifications/initialized, tools/list, and a bounded best-effort DELETE session cleanup that never converts a successful availability check into a failure. No tool is ever called. - Attribute failures precisely: closed stage enum (dns/connect/tls/ proxy/initialize_*/initialized_notification/tools_list_*/ catalog_validation/session_cleanup), closed sanitized network-cause codes, split HTTP classes (401/403/404/429/502-504/redirects), split protocol classes (utf8/json/envelope/request-id/protocol-version/ session-header/catalog/pagination), and deterministic retryability derived only from closed code sets. - Accept forward-compatible catalogs: extra provider tools no longer fail validation, but only the allowlisted expected tool IDs and aggregate counts are exported; both required tools must be present. - Add a cloud-endpoint-differential check distinguishing endpoint availability from engine availability (runtime_and_engine_connected, runtime_connected_engine_failed, runtime_failed_engine_connected, runtime_and_engine_failed, runtime_probe_not_performed, engine_evidence_stale_or_unavailable) with engine evidence source and age surfaced. - Keep the trust boundary: only built-in Cloud origins, exact administrator-configured origins, and loopback may receive the credentialed probe; untrusted_endpoint is reported as probe-not- performed trust configuration, never as a network or MCP failure. Boolean-only proxy/extra-CA posture and a strictly validated x-request-id reference are the only environment-derived exports. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
@reachjalil is attempting to deploy a commit to the Different AI Team on Vercel. A member of the Team first needs to authorize it. |
reachjalil
marked this pull request as ready for review
July 27, 2026 18:45
reachjalil
added a commit
that referenced
this pull request
Jul 27, 2026
…ine-evidence staleness (#3200) * fix(diagnostics): probe activated on-prem Den origins and correct engine-evidence staleness Follow-up to #3199, from a real enterprise diagnostics report where the runtime probe could not run at all. Trust the activated enterprise control plane. An on-prem deployment's managed openwork-cloud entry points at the customer's own Den origin, so the probe reported untrusted_endpoint and sent nothing — leaving exactly the deployments that most need differential diagnosis with no evidence beyond OpenCode's opaque "failed". The desktop writes enterpriseActivation only after a signed activation claim verifies, so that record is administrator-provisioned configuration rather than a renderer- or request-supplied URL. The server now reads it (bounded, read-only, local workspaces only) and admits its exact origin to the diagnostics allowlist. Arbitrary URLs, non-https origins, and credential-, query-, or fragment-bearing values are still rejected, and a missing or malformed record fails closed. Report trust provenance instead of the origin. trustSource (builtin-cloud | loopback | administrator-env | enterprise-activation | untrusted | not-evaluated) and enterpriseActivationPresent let an administrator tell "this install is not activated" apart from "it is activated, but against a different origin than the configured Cloud MCP" — a real misconfiguration that previously looked identical — without the report ever carrying a hostname. Correct the differential staleness rule. Treating any engine record older than 60s as stale made every healthy steady-state run report engine_evidence_stale_or_unavailable, because a connected registration is the engine's standing state and ages naturally between syncs. Only a failure record the reachable engine could already have refreshed is downgraded, mirroring mcp_registration_stale_failure; an aged failure on an unreachable engine keeps implicating the engine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(diagnostics): assert the probe only ever reaches the configured Den endpoint An on-prem operator's probe must reach their own Den deployment, never OpenWork-hosted Cloud. The built-in origins are a membership allowlist consulted with .has() and are never a destination or fallback, but that invariant was only enforced by inspection. Pin every request URL of the handshake and the session cleanup to the exact configured endpoint, and assert no OpenWork-hosted origin is contacted, at both the probe and analyzer levels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 27, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Extends the Agent Context diagnostic so it can verify the configured
openwork-cloudMCP endpoint independently of OpenCode, from the OpenWork runtime's own network stack, and then answer the differential question support actually needs:The probe runs the complete bounded MCP handshake —
initialize(with protocol-version validation),notifications/initialized,tools/list, and a bounded best-effortDELETEsession cleanup. It never calls an MCP tool, never mutates configuration, never reconnects OpenCode, and never remints credentials.Runtime transport is now explicit
apps/server/src/server-fetch.tsgainsruntimeDiagnosticFetchandruntimeDiagnosticTransportInfo(). The probe reportsruntimeFamily(electron-node|node|bun|unknown) andtransport(node-undici|bun-fetch|test-seam|unknown), so a Bun standalone result is never claimed to have come from Node, and the deliberate Node/undici path inside Electron main is a named seam instead of an accidental?? fetchfallback.The bare-fetch guard test now flags bare
fetchreferences, not just call sites, soinput.fetchImpl ?? fetchcan no longer bypass it. The sweep surfaced and fixed three more accidental fallbacks:opencode-models-url.ts(→loopbackFetch),worker-activity-heartbeat.ts(→externalFetch), and the engine-client default inserver.ts(→ explicitglobalThis.fetch).Eligibility is decoupled from OpenCode state
Cached registration states (
failed,needs-auth,needs-client-registration,not-recorded) and agent tool policy no longer gate the probe — the probe exists precisely to diagnose those situations. They are now comparison inputs and separate reported dimensions. A denied or unavailable tool policy is never reported as endpoint unavailability.The probe still skips (as
not-performed, stageeligibility) for: remote-workspace privacy, disabled MCP config, missing/malformed/duplicate credentials, invalid endpoints, untrusted origins, expired overall deadline, and probe-concurrency exhaustion.Trust boundary preserved; on-prem audit result
Only built-in OpenWork Cloud origins, exact administrator-configured origins (
OPENWORK_AGENT_DIAGNOSTICS_TRUSTED_ORIGINS), and loopback may receive the credentialed request. Renderer/user-supplied URLs cannot widen this.Audit: the enterprise/on-prem Den base origin (
denBaseUrl) is provisioned desktop-side (desktop bootstrap / enterprise activation / connect-link claims inapps/desktop/electron/workspace-store.mjs) and is not visible to the server process, so the diagnostic cannot derive a trusted origin from it today. Per that result,untrusted_endpointis reported as probe not performed (warning, stageeligibility) with an actionable admin-configuration explanation — explicitly not as a network/TLS/MCP failure. Plumbing the activationdenBaseUrlinto the server's trust list is left as a follow-up because the desktop runtime env wiring is owned by #3118's area (see overlap below).Detailed failure attribution
Failures no longer collapse into
network_error/invalid_response. The typed result includesperformed,stage,code,networkCode,retryable,runtimeFamily,transport,httpStatus,durationMs,toolsListPerformed,sessionEstablished,cleanupAttempted,cleanupSucceeded,engineRegistrationStatus,engineEvidenceSource,engineEvidenceAgeMs,proxyConfigured/extraCaConfigured(booleans only), a strictly validatedreferenceId(x-request-id), aggregate catalog counts, and a bounded list of sanitized step summaries.eligibility,dns,connect,tls,proxy,initialize_request|_http|_protocol,initialized_notification,tools_list_request|_http|_protocol,catalog_validation,session_cleanup,complete.ENOTFOUND,EAI_AGAIN,ECONNREFUSED,ECONNRESET,ETIMEDOUT,UND_ERR_CONNECT_TIMEOUT, four TLS-validation codes +ERR_TLS_CERT_ALTNAME_INVALID,PROXY_ERROR,UNKNOWN_NETWORK_ERROR.Raw exception messages, cause objects, errno numbers, certificate subjects, paths, proxy values, response bodies, headers, and stack traces are never exported. Catalog validation is forward-compatible: extra provider tools no longer fail the check, but only the allowlisted expected IDs and aggregate counts are exported — provider-controlled names are never reflected.
Differential verdict
A new
cloud-endpoint-differentialcheck compares the runtime observation with the engine's cached registration evidence for the same entry:runtime_and_engine_connected,runtime_connected_engine_failed(implicates the engine-side path/lifecycle — endpoint is reachable),runtime_failed_engine_connected(implicates the OpenWork runtime network path — proxy/trust-store/DNS as seen by Node or Bun),runtime_and_engine_failed(shared outage),runtime_probe_not_performed, andengine_evidence_stale_or_unavailable(record missing, or a failure record older than 60s while the engine is reachable and could have refreshed it — mirroring the existingmcp_registration_stale_failuresemantics; aged connected records are the engine's standing state and stay trusted so healthy steady-state runs do not warn).Schema changes (server + shared types kept in exact sync)
cloud-endpoint-differential(canonical position aftercloud-tool-catalog); one new label key in the report page +en.ts.cloudSessionCleanupRequesteddisclosing the DELETE.cloudCatalogToolsListPerformedcross-invariant no longer requiressyncStatus === "connected"or a passed tool-policy check (those were the schema-level encoding of the removed gates); it still requires the retained runtimeopenwork-cloudentry with the/mcp/agentterminal path, proving which managed entry was probed.The fail-closed redaction principles from #2997 are preserved: all new detail values pass the existing safe-text schema (no URLs, paths, or credential-shaped text), and eligibility/trust failures stay "no request occurred" states.
Overlap with open PRs (checked 2026-07-27)
cloud-mcp-health.ts(pollConnected tracing),server.ts(reload-event stores + dispose avoidance),runtime.mjs/main.mjs(startup tracing + updater-manifestelectronNet.fetch). This PR does not touch any of those regions —cloud-mcp-health.tsand all Electron files are untouched here, and the two-lineserver.tschange (engine-client fetch default) is far from its hunks. Based directly ondev; no stacking needed; none of its lifecycle/networking work is rewritten or absorbed. The on-prem trust plumbing follow-up should coordinate with its owner since it owns the runtime env wiring area.readDirectCloudToolsincloud-mcp-health.ts— untouched here, no conflict.Tests run
From
apps/server(Bun 1.3.9;BUN_RUNTIME_TRANSPILER_CACHE_PATH=0to avoid a pre-existing flakynode:sqlitetranspiler-cache resolution on this machine):bun test(full package): 555 pass, 10 skip, 1 fail — the single failure isworkspace-kv-store.node.test.tsfailing to resolvenode:sqliteunder Bun; verified identical on the pristine base commit (git stash→ same failure), unrelated to this change.bun test src/agent-context-cloud-probe.test.ts src/no-bare-fetch.test.ts— 29 pass.bun test src/agent-context-diagnostics.test.ts— 44 pass.bun test src/agent-context-diagnostics.schema.test.ts src/agent-context-diagnostics-local-schema.test.ts src/agent-context-diagnostics.plugin-label.test.ts src/agent-context-engine-inspection.test.ts src/opencode-models-url.test.ts— pass.bun test src/worker-activity-heartbeat.test.ts src/cloud-mcp-health.test.ts— 26 pass.npx tsc --noEmitinapps/server,packages/types, andapps/app— clean.From
apps/app:bun test tests/agent-context-diagnostics-report.test.tsx— 13 pass.Validation limitations
No video/fraimz run is attached: this is a server-side diagnostic-evidence change whose only UI delta is one additional check row rendered by the existing generic check renderer (label added to
en.ts). To reproduce end-to-end: run the desktop app with a connected OpenWork Cloud workspace, open Settings → Connect → Agent Context diagnostics, run diagnostics, and inspect thecloud-tool-catalogdetails (stage/networkCode/transport/steps) and the newRuntime probe vs engine verdictcheck; to see the differential in action, stop or break the engine registration (e.g. revoke Cloud auth) and rerun — the catalog check now still probes and the differential reportsruntime_connected_engine_failed.🤖 Generated with Claude Code