Skip to content

azd init Copilot Preview silently falls back from claude-opus-4.7 to claude-opus-4.6 when using the bundled Copilot CLI #8108

@naoki1213mj

Description

@naoki1213mj

Summary

When using azd init with the Set up with GitHub Copilot (Preview) option, the model configured via azd config set copilot.model claude-opus-4.7 does not appear in the resulting session. The end-of-run Session usage: block reports Model: claude-opus-4.6 and Billing rate: 3x per request, even though azd config show and the azd init startup banner both display claude-opus-4.7.

The behavior is reproducible and depends on which Copilot CLI binary azd invokes:

Layer / source What it reports
azd config show "copilot": { "model": "claude-opus-4.7" }
azd init startup banner Agent: model=claude-opus-4.7, reasoning=
Session usage: block (end of run, default bundled CLI) Model: claude-opus-4.6, Billing rate: 3x per request
Session usage: block (override CLI via AZD_COPILOT_CLI_PATH) Model: claude-opus-4.7, Billing rate: 15x per request

Same azd config, same OAuth token, same Flask + Dockerfile project, same Japan East subscription. The only material difference between the two runs is which copilot binary handles the session.

Environment

OS Windows 11
azd 1.24.3 (commit f5327208c226cc63030391958990ecf6bf971633)
Subscription Pay-as-you-go MCAP, Japan East
Account Personal account with active GitHub Copilot Pro+ subscription

Controls (account-level access to 4.7 is fine)

To rule out account/entitlement issues, I invoked the bundled Copilot CLI binary directly outside the azd SDK harness:

& "$env:USERPROFILE\.azd\bin\copilot-cli-1.0.2.exe" -p "say ok" --model claude-opus-4.7 --allow-all-tools
# → Session usage reports Model=claude-opus-4.7, Billing rate=15x, ~80k input tokens

& "$env:USERPROFILE\.azd\bin\copilot-cli-1.0.2.exe" -p "say ok" --model claude-opus-4.6 --allow-all-tools
# → Session usage reports Model=claude-opus-4.6, Billing rate=3x, ~57k input tokens

The same binary that downgrades to 4.6 when invoked through the azd Copilot SDK can use 4.7 just fine when invoked directly. The fallback is therefore not an account-level entitlement issue — it is something specific to the azd SDK / session-creation path.

Binary comparison

Both binaries report --version 1.0.44-2. They are not byte-identical builds:

Source Path Size (bytes) SHA256 --version
azd-bundled (npm @github/copilot-win32-x64@1.0.2) ~/.azd/bin/copilot-cli-1.0.2.exe 110,972,888 CA3A4174C02F86DA3038B32FB34A3D99027982B62D51126AE00C817908F5FAEC 1.0.44-2
WinGet (GitHub.Copilot) ~/AppData/Local/Microsoft/WinGet/Packages/GitHub.Copilot_Microsoft.Winget.Source_8wekyb3d8bbwe/copilot.exe 121,264,928 514BC491B9D82C3A113156C215CB974C2ADECE6EDCB7CA39D151614D3989132F 1.0.44-2

The npm @github/copilot-win32-x64 package's current dist-tags are latest = 1.0.43 and prerelease = 1.0.44-2. azd pins npm package version 1.0.2. The 1.0.2 package on the npm registry contains the binary above (self-reporting 1.0.44-2, ~110.9 MB). The WinGet package contains a different build (~121.3 MB) that also self-reports 1.0.44-2.

Reproduction

A. Default (bundled CLI), produces 4.6 fallback

# Repro env
azd version
# azd version 1.24.3 (commit f5327208c226cc63030391958990ecf6bf971633) (stable)

azd config set copilot.model claude-opus-4.7
Remove-Item env:AZD_COPILOT_CLI_PATH -ErrorAction SilentlyContinue

# Minimal repro project (any small Flask + Dockerfile works):
#   app.py:           "from flask import Flask\napp = Flask(__name__)\n@app.route('/')\ndef home(): return 'hi'\n"
#   requirements.txt: "Flask>=3.0\ngunicorn>=21\n"
#   Dockerfile:       (any minimal Python container)
git init && git add -A && git commit -m "baseline" -q

azd init
# → choose: Set up with GitHub Copilot (Preview)
# → answer the agent's prompts (subscription, region: japaneast, etc.)
# → wait for completion

End-of-run output (representative, full transcript on request):

Agent: model=claude-opus-4.7, reasoning=

Preparing application for Azure deployment...
[...agent runs successfully, generates azure.yaml + infra/...]

Session usage:
  • Model:            claude-opus-4.6
  • Input tokens:     4.2M
  • Output tokens:    17.5K
  • Total tokens:     4.2M
  • Billing rate:     3x per request
  • Premium requests: 0
  • API duration:     6m 4s

SUCCESS: New project initialized!

B. AZD_COPILOT_CLI_PATH override (WinGet binary), passes through 4.7

$env:AZD_COPILOT_CLI_PATH = "$env:USERPROFILE\AppData\Local\Microsoft\WinGet\Packages\GitHub.Copilot_Microsoft.Winget.Source_8wekyb3d8bbwe\copilot.exe"
azd init
# → choose: Set up with GitHub Copilot (Preview)

End-of-run output:

Agent: model=claude-opus-4.7, reasoning=

[...agent's tool calls all fail with "unexpected user permission response", session never reads any project files...]

Session usage:
  • Model:            claude-opus-4.7
  • Input tokens:     1.2M
  • Output tokens:    4.3K
  • Total tokens:     1.2M
  • Billing rate:     15x per request
  • Premium requests: 0
  • API duration:     1m 26s

ERROR: initializing project extensions: ...azure.yaml: The system cannot find the file specified.

In B, the model passes through correctly, but a separate problem (described below) prevents azd init from completing.

Why this matters

  1. Configuration appears inert: azd config set copilot.model claude-opus-4.7 is accepted, persisted, and echoed by the banner — but does not change which model is actually used.
  2. Visibility: The discrepancy is observable only by reading the last lines of Session usage:. The startup banner (Agent: model=…) is the configured value, not the actually-used value.
  3. No usable workaround in my environment: The only path I have found that actually uses 4.7 is AZD_COPILOT_CLI_PATH override, but that path has its own protocol failure (described below) that prevents azd init from generating IaC. In my Windows 11 / azd 1.24.3 setup, I have not found an end-to-end azd init Copilot Preview run that both uses 4.7 and completes successfully.

Suspected mechanism

I cannot determine the fallback signal from the client side alone. Possibilities I considered include request metadata, advertised capabilities, the SDK protocol version, or some property of the embedded CLI. The salient observable is that the same azd config and the same account produce different session models depending on which copilot binary serves the SDK session — and the only known difference between those binaries is the build (different bytes, same self-reported version).

The pinned CLI version is at:

  • cli/azd/internal/agent/copilot/cli.go#L30-L32:

    // cliVersion is the Copilot CLI version that matches the SDK version in go.mod.
    // SDK v0.1.32 → CLI v1.0.2 (determined by the SDK's package-lock.json).
    const cliVersion = "1.0.2"
  • The AZD_COPILOT_CLI_PATH override path is checked first in ensureInstalled(), otherwise the pinned cliVersion is downloaded from https://registry.npmjs.org/@github/copilot-{platform}-{arch}/-/copilot-{platform}-{arch}-1.0.2.tgz and cached as ~/.azd/bin/copilot-cli-1.0.2.exe.

  • The configured model is passed through in cli/azd/internal/agent/copilot_agent.go (sessionConfig.Model = a.modelOverride). I do not see an azd-side rewrite of the model in this code path; azd appears to pass claude-opus-4.7 into sessionConfig.Model.

Suggested directions

These are framed as observations rather than prescriptions:

  1. Investigate whether the npm 1.0.2 CLI / SDK pairing advertises stale capabilities or request metadata that causes the server to fall back to 4.6, and either bump the pinned SDK/CLI pair or update the compatibility contract accordingly.
  2. Surface the actual session model: when the model reported by the session differs from the one configured/requested, log a warning before any tool call:
    warning: requested model 'claude-opus-4.7' but session reports 'claude-opus-4.6' (cliVersion=1.0.2)
    
    This single diagnostic would have made the issue self-evident.
  3. Document any fallback / capability behavior in the Copilot Preview help text. Note that the Agent: model=… banner echoes the configured model, and that the actually-used model is reported in Session usage: at the end of the run.

Related but separate: AZD_COPILOT_CLI_PATH permission protocol mismatch

In repro B above, the WinGet binary triggers Unhandled error: Error: unexpected user permission response for every agent tool call (view, powershell, etc.), despite azd config show reporting the consent grants azd/*, copilot/*, azure/* all allow. Selecting "I'll allow file reads — please retry" in the agent's prompt does not recover. This makes the override path unable to complete azd init (no azure.yaml is generated). I can file this as a separate issue if it would be more useful tracked apart from the model fallback.

References

Full transcripts (v8-pattern-A.log, v8-pattern-A2.log) and a complete reproduction repository are available on request. Happy to test any proposed fixes.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions