Filter unresolvable model aliases from /reflect and models.json by Copilot · Pull Request #3803 · github/gh-aw-firewall

Copilot · 2026-05-25T19:00:38Z

When a provider retires or disables a model, any configured alias whose patterns only resolve to that retired model would remain visible in /reflect and models.json — misleading agents into using aliases that silently fail or fall back unexpectedly.

Changes

`model-resolver.js` — new `filterResolvableAliases(aliases, availableModels)`

For each alias, attempts resolution (fallback disabled) against every provider that has live model data from the /models endpoint
Drops aliases with zero matches; keeps aliases that resolve for at least one provider
Returns the input unchanged when no provider has model data yet (avoids filtering before startup fetch completes)

`server.js` — apply filter at all external exposure points

buildModelsJson(), writeModelsJson(), and the getModelAliases getter (used by /reflect) all now serve filtered aliases

Behaviour

# Config: { "sonnet": ["copilot/*sonnet*"], "gpt-5-codex": ["copilot/gpt-5*-codex"] }
# Provider copilot /models returns: ["gpt-4o", "gpt-5.2"]   ← no sonnet, no codex

GET /reflect → model_aliases: {}   # both aliases filtered out

# Provider copilot /models returns: ["claude-sonnet-4.6", "gpt-5.2"]

GET /reflect → model_aliases: { "sonnet": [...] }   # gpt-5-codex still filtered

Fallback is intentionally disabled during filtering so that an alias whose target models have all been retired is removed rather than kept alive via a fallback selection.

…output Add filterResolvableAliases() to model-resolver.js which dynamically removes aliases that cannot be resolved to any available model for any provider that has reported models via the /models endpoint. - Only keeps aliases whose patterns match at least one known model - Keeps all aliases when no provider has fetched models yet (unknown state) - Disables middle-power fallback during filtering so only genuine alias→model matches count; fallback selections would keep aliases alive even after all their targets have been retired - Applied in server.js buildModelsJson(), writeModelsJson(), and the getModelAliases getter passed to the management /reflect handler Fixes: "Ignore unknown model aliases" issue

github-actions · 2026-05-25T20:45:36Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	96.54%	96.58%	📈 +0.04%
Statements	96.38%	96.42%	📈 +0.04%
Functions	97.99%	97.99%	➡️ +0.00%
Branches	90.78%	90.82%	📈 +0.04%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/config-writer.ts`	89.3% → 90.9% (+1.65%)	89.3% → 90.9% (+1.65%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-05-25T20:46:29Z

Smoke Test: Claude Engine

✅ GitHub API (recent PRs): PASS
✅ GitHub check (playwright): PASS
✅ File verify: PASS

Result: PASS

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-05-25T20:46:59Z

🔍 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Notes
S1: Module Loading	✅	`otel.js` loads cleanly; exports: `startRequestSpan`, `setTokenAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled` + internal helpers
S2: Test Suite	✅	33/33 tests passed (`otel.test.js`) — spans, token attrs, parent context, OTLP export, file fallback, graceful degradation
S3: Env Var Forwarding	✅	`api-proxy-service.ts` forwards `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`
S4: Token Tracker Integration	✅	`onUsage` callback exists in `token-tracker-http.js` (line 237); invoked after normalized usage extraction
S5: OTEL Diagnostics	✅	`FileSpanExporter` writes to `/var/log/api-proxy/otel.jsonl` when no OTLP endpoint configured; OTLP path available when endpoint set

All 5 scenarios pass. OTEL tracing integration is fully operational.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-05-25T20:46:59Z

🔥 Smoke Test: Copilot BYOK (Offline) Mode

Test	Result
GitHub MCP (list PRs)	✅
GitHub.com connectivity	⚠️ Pre-step data unavailable (template not expanded)
File write/read	⚠️ Pre-step data unavailable (template not expanded)
BYOK inference (this response)	✅

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

Overall: PASS (core BYOK path confirmed; pre-step data missing due to unexpanded template vars)

PR by @Copilot · Assignees: @lpcox, @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-05-25T20:47:15Z

🔬 Smoke Test Results

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	✅
File write/read (`smoke-test-copilot-26415810243.txt`)	✅

PR: Filter unresolvable model aliases from /reflect and models.json
Author: @Copilot · Assignees: @lpcox, @Copilot

Overall: PASS ✅

📰 BREAKING: Report filed by Smoke Copilot

Copilot

Pull request overview

This pull request ensures /reflect and models.json no longer expose model aliases that cannot resolve to any currently-known provider model, preventing agents from selecting aliases that will fail or unexpectedly fall back after models are retired/disabled.

Changes:

Added filterResolvableAliases(aliases, availableModels) to drop aliases that don’t resolve against any provider with cached model data (with fallback intentionally disabled during filtering).
Applied alias filtering to all external exposure points in server.js (/reflect via getModelAliases, plus buildModelsJson() and writeModelsJson()).
Added unit tests covering both the resolver-level filtering behavior and the buildModelsJson() integration behavior.

Show a summary per file

File	Description
containers/api-proxy/model-resolver.js	Adds `filterResolvableAliases` that keeps only aliases resolvable against providers with non-empty cached model lists, with fallback disabled for filtering decisions.
containers/api-proxy/server.js	Ensures `/reflect` and models snapshot generation serve filtered aliases instead of the raw configured alias map.
containers/api-proxy/model-resolver.test.js	Adds dedicated test coverage for `filterResolvableAliases`, including multi-provider, recursion, and extended alias syntax cases.
containers/api-proxy/server.models.test.js	Adds integration tests validating `buildModelsJson()` emits filtered aliases when cached model data is present, and preserves aliases when no model data exists yet.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 4/4 changed files
Comments generated: 0

github-actions · 2026-05-25T20:48:24Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌ NO
Node.js	v24.15.0	v22.22.3	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot

github-actions · 2026-05-25T20:48:29Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #3803 · sonnet46 1M · ◷

github-actions · 2026-05-25T20:48:43Z

Smoke test results failed. Check logs.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-05-25T20:48:55Z

Smoke Test Results

Check	Result
Redis PING	❌ timeout (host.docker.internal unreachable)
PostgreSQL pg_isready	❌ no response
PostgreSQL SELECT 1	❌ skipped (host unreachable)

Overall: FAIL — host.docker.internal is not accessible from this runner environment. Service containers may not be running or the hostname is not resolvable.

🔌 Service connectivity validated by Smoke Services

Initial plan

7dc0fb5

Copilot AI assigned Copilot and lpcox May 25, 2026

Copilot started work on behalf of lpcox May 25, 2026 19:00 View session

Copilot AI linked an issue May 25, 2026 that may be closed by this pull request

Ignore unknow model aliases #3775

Open

Copilot AI changed the title ~~[WIP] Fix filtering of unknown model aliases~~ Filter unresolvable model aliases from /reflect and models.json May 25, 2026

Copilot AI requested a review from lpcox May 25, 2026 19:08

Copilot finished work on behalf of lpcox May 25, 2026 19:08

lpcox marked this pull request as ready for review May 25, 2026 20:44

Copilot AI review requested due to automatic review settings May 25, 2026 20:44

Copilot started reviewing on behalf of lpcox May 25, 2026 20:44 View session

github-actions Bot added the smoke-claude label May 25, 2026

github-actions Bot mentioned this pull request May 25, 2026

[aw] No-Op Runs #2911

Open

github-actions Bot added the smoke-copilot-byok label May 25, 2026

github-actions Bot added the smoke-copilot label May 25, 2026

github-actions Bot mentioned this pull request May 25, 2026

[aw] Smoke Codex failed #3810

Open

Copilot AI reviewed May 25, 2026

View reviewed changes

github-actions Bot added the build-test label May 25, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Filter unresolvable model aliases from /reflect and models.json#3803

Filter unresolvable model aliases from /reflect and models.json#3803
Copilot wants to merge 2 commits into
mainfrom
copilot/ignore-unknown-model-aliases

Copilot AI commented May 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

model-resolver.js — new filterResolvableAliases(aliases, availableModels)

server.js — apply filter at all external exposure points

Behaviour

Uh oh!

github-actions Bot commented May 25, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

github-actions Bot commented May 25, 2026

Smoke Test: Claude Engine

Uh oh!

github-actions Bot commented May 25, 2026

🔍 Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented May 25, 2026

🔥 Smoke Test: Copilot BYOK (Offline) Mode

Uh oh!

github-actions Bot commented May 25, 2026

🔬 Smoke Test Results

Overall: PASS ✅

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

github-actions Bot commented May 25, 2026

Chroot Version Comparison Results

Uh oh!

github-actions Bot commented May 25, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented May 25, 2026

Uh oh!

github-actions Bot commented May 25, 2026

Smoke Test Results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented May 25, 2026 •

edited

Loading

`model-resolver.js` — new `filterResolvableAliases(aliases, availableModels)`

`server.js` — apply filter at all external exposure points