[aw-failures] P1: MCP Gateway v0.3.26 rejects digest-pinned `safeoutputs.container` (`@sha256`) — 19 runs fail at "Start MCP Gat [Content truncated due to length]

[github-actions]

Problem statement

MCP Gateway v0.3.26 fails configuration validation when the safeoutputs MCP server's container field is pinned by digest (@sha256:...). The gateway's stdioServerConfig.container JSON-schema pattern only accepts a :tag suffix, not an @digest suffix, so the gateway aborts at startup and every affected agentic run fails at the Start MCP Gateway step.

This is introduced by PR #39644 ("Include and pin safe-outputs gh-aw-node image in default container predownload and manifest", branch copilot/add-gh-aw-node-to-default-containers). Production main is not affected — the committed lock files pin by tag (ghcr.io/github/gh-aw-node), which passes validation.

Evidence

1. Runtime value emitted by the PR build (from run 27648258396 gateway config):

   "container": "ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"
   

2. Gateway error (identical across representative run §27653712058 and comparator run §27648258396):

   Configuration validation error (MCP Gateway version: v0.3.26):
   Location: /mcpServers/safeoutputs/container
   Error: does not match pattern '^[a-zA-Z0-9][a-zA-Z0-9./_-]*(:([a-zA-Z0-9._-]+|latest))?$'
   

3. The pattern allows name(:tag)? only. @ is not in the character class, so any @sha256: digest pin is rejected. The same server config also trips the httpServerConfig branch (mounts/args/container/entrypoint/entrypointArgs not allowed there), confirming stdioServerConfig is the intended branch and the container pattern is the sole blocker.
4. audit-diff (failed PR build vs main): the only material delta is the container value — main uses the tag form ghcr.io/github/gh-aw-node (HEAD lock, line 1510), which matches the pattern; the PR appends @sha256:..., which does not.

Affected workflows and run IDs

All on branch copilot/add-gh-aw-node-to-default-containers (PR #39644). 19 runs failed at Start MCP Gateway in the last 6h:

Workflow	Representative run
Smoke Claude	§27653712058, §27648258396
Smoke Codex	§27653712017, §27648258464
Smoke Gemini	§27653712029, §27648258428
Smoke Antigravity	§27653712034, §27648258562
Smoke Copilot (+ AOAI apikey/Entra)	§27653927895, §27653928816, §27653929599, §27648313593, §27648314584, §27648315754
Changeset Generator	§27653712023, §27648258427
Agent Container Smoke Test	§27653712062, §27648258470
Smoke CI	§27651670792

Related downstream failures on the same branch (agent job died, then safe_outputs job failed at Process Safe Outputs): §27655298190, §27655260785.

Probable root cause

The PR changed the default-container manifest so the safeoutputs stdio MCP server's container is emitted in pinned image@sha256:digest form. MCP Gateway v0.3.26's config schema (mcp-gateway-config.schema.json, stdioServerConfig.container) only permits name[:tag], so digest-pinned references are rejected before any server starts.

Proposed remediation (pick one)

1. gh-aw side (preferred, unblocks PR now): for stdio MCP server container fields, emit the tag form (ghcr.io/github/gh-aw-node:<tag>) and carry the digest separately (predownload/manifest only, as is already done for the agent/firewall images via imageTag/digest map). Do not inline @sha256: into the gateway container string.
2. Gateway side: widen the stdioServerConfig.container pattern to also accept the digest form, e.g. append (@sha256:[a-f0-9]{64})? to the existing pattern, and bump the MCP Gateway pin in gh-aw once released.

Option 1 is lower-risk and keeps gh-aw compatible with the already-pinned gateway v0.3.26.

Success criteria / verification

1. Re-run the smoke matrix on copilot/add-gh-aw-node-to-default-containers; Start MCP Gateway passes for all engines (Claude, Codex, Gemini, Antigravity, Copilot) and Agent Container Smoke Test.
2. Gateway log shows no Configuration validation error for /mcpServers/safeoutputs/container.
3. The safeoutputs server still resolves to the digest-pinned image at pull time (digest pinning preserved via predownload/manifest), confirming the security intent of PR Include and pin safe-outputs gh-aw-node image in default container predownload and manifest #39644 is retained.

---

Scope: agentic-workflow failures, last 6h, repo github/gh-aw. Parent: meta-orchestrator #29109. No existing open issue covers this signature.

References: #39644 · §27653712058 · §27648258396 · §27653929599
Related to #29109

Generated by 🔍 [aw] Failure Investigator (6h) · ◷

• expires on Jun 23, 2026, 5:52 PM UTC-08:00

[github-actions]

🍪 Issue Monster selected this for Copilot

I've identified this critical P1 issue as a priority candidate for automated resolution and requested assignment to the Copilot coding agent.

Summary: MCP Gateway v0.3.26 rejects digest-pinned container references in the safeoutputs config. The proposed remediation is to emit the tag form for stdio MCP server container fields while carrying the digest separately via predownload/manifest.

If assignment succeeds, the Copilot coding agent will analyze the issue, implement the fix from the proposed remediation options, and create a pull request to unblock the 19 failing smoke tests on the target branch.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · ◷

[copilot-swe-agent]

The agent encountered an error and was unable to start working on this issue: Please try again later, or contact support if the problem persists. (Request id: 1801:117651:169C4A3:53A2C90:6A320701)