Skip to content

[aw-failures] [aw-fix] P1: Copilot auto model has no AI-credits pricing — api-proxy returns 400 and fails Smoke Copilot / AI-c [Content truncated due to length] #46846

Description

@github-actions

Add a pricing entry (or apiProxy.defaultAiCreditsPricing fallback) for the resolved auto model — the api-proxy 400s on every request, exhausting all retries and keeping Copilot AI-credits runs red.

Parent: #46171

Problem statement

When the Copilot engine runs with model: auto, the AWF api-proxy has no pricing row for the resolved auto model and no default configured, so it rejects every request with a 400 before any tokens are billed. The copilot-harness retries all 4 attempts identically (failureClass=partial_execution, retriesRemaining=0) and the Execute GitHub Copilot CLI step exits 1.

400 400 400 Model "auto" has no AI credits pricing and no default pricing is configured. Set apiProxy.defaultAiCreditsPricing in the AWF config (e.g. {"input": 3.0, "output": 15.0}) to provide a fallback rate, or add the model to the pricing table.

Affected workflows & run IDs

Probable root cause

The api-proxy AI-credits pricing table keys on a concrete model id, but the literal alias auto reaches the proxy unresolved (or resolves to a model missing from the table), and no apiProxy.defaultAiCreditsPricing fallback is set.

Proposed remediation

  1. Set apiProxy.defaultAiCreditsPricing (e.g. {"input": 3.0, "output": 15.0}) so unknown/auto models get a fallback rate instead of a 400.
  2. Resolve the auto alias to a concrete pricing-table model before the credits check, or add an explicit auto row.
  3. Verify the same fix clears the unknown_model_ai_credits / provider-403 path seen in Daily Max Ai Credits Test.

Success criteria / verification

Generated by 🔍 [aw] Failure Investigator (6h) · 136.1 AIC · ⌖ 36.6 AIC · ⊞ 5.2K ·

  • expires on Jul 27, 2026, 5:33 AM UTC-08:00

Scope expansion — same api-proxy AI-credits pricing gate now also hard-fails BYOK Ollama

Extend remediation #1 to BYOK models: a bring-your-own-key model routed to the user's own endpoint must bypass the AI-credits pricing gate entirely — it will otherwise keep 400ing regardless of defaultAiCreditsPricing.

A third workflow hits this exact 400. Add it to the affected list; the fix here also clears it.

Newly affected workflow & run

  • Daily BYOK Ollama Test (.github/workflows/daily-byok-ollama-test.lock.yml) — representative §29784505320 (2026-07-20 22:36 UTC). No successful baseline exists (baseline_found: false); deterministic hard-fail. Raw auto-notification: [aw] Daily BYOK Ollama Test failed #46931 (surfaces only the downstream awf-reflect ... 503 symptom, not this root cause).

Evidence — identical 400, all 4 retries, 0 turns / 0 tokens

Engine copilot v1.0.71, model: qwen2.5:0.5b, COPILOT_PROVIDER_BASE_URL=(host.docker.internal/redacted) (local Ollama). All Ollama setup, Copilot install, and MCP Gateway steps succeeded; only Execute GitHub Copilot CLI failed (exit 1). The copilot-harness retried all 4 attempts identically (partial_execution, retriesRemaining=0`):

400 400 400 Model "qwen2.5:0.5b" has no AI credits pricing and no default pricing is configured. Set apiProxy.defaultAiCreditsPricing in the AWF config (e.g. {"input": 3.0, "output": 15.0}) to provide a fallback rate, or add the model to the pricing table.

Turns=0, TokenUsage=0 — the CLI died before any model turn.

BYOK-specific root cause

Even though the model runs on the user's own Ollama endpoint (not GitHub AI credits), the AWF api-proxy still enforces the AI-credits pricing table and 400s qwen2.5:0.5b. A defaultAiCreditsPricing fallback (remediation #1) would incidentally unblock it, but the correct behavior is that BYOK requests (with COPILOT_PROVIDER_BASE_URL/COPILOT_PROVIDER_API_KEY set) skip the AI-credits pricing check entirely, since AI credits are never consumed.

Added remediation

  1. When a request is BYOK (provider base URL / API key set), bypass the AI-credits pricing gate rather than requiring a pricing row or fallback rate.

Added success criteria

  • Daily BYOK Ollama Test completes green with non-zero turns using qwen2.5:0.5b against the local Ollama endpoint.
  • No has no AI credits pricing 400 in BYOK mode across 3 consecutive scheduled runs.

Generated by 🔍 [aw] Failure Investigator (6h) · 224.4 AIC · ⌖ 34.2 AIC · ⊞ 5.2K ·

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions