[copilot-cli-research] Copilot CLI Deep Research - 2026-08-13

[github-actions]

Analysis Date: 2026-08-13
Repository: github/gh-aw
Scope: 285 total workflows, 138 using the Copilot engine

---

📊 Executive Summary

Research Topic: Copilot CLI Optimization Opportunities (12th consecutive tracked run)
Key Findings:

1. 🆕 A new top-level model field (replacing deprecated engine.model) shipped in PR Route PR code-quality reviews through the Copilot gateway #52377 (merged 2026-08-12) with an automated gh aw fix codemod — already adopted by 32/138 (23%) Copilot workflows, but ~147 workflow files repo-wide still use the legacy nested form.
2. engine.args and the --share flag remain at 0 and 1 usages respectively across 12+ consecutive runs — confirmed stable, low-priority non-issues rather than real gaps.
3. 4 of 11 custom agent files (create-safe-output-type, custom-engine-implementation, grumpy-reviewer, w3c-specification-writer) remain completely unreferenced by any workflow for 12 consecutive research runs.
4. add-dir has zero real workflow adoption; disable-builtin-mcps has only 2 real usages repo-wide.
5. Positive trend: cache-memory (126) and Copilot SDK (copilot-sdk: true, ~71) usage continue steady organic growth without prompting.

Primary Recommendation: Run gh aw fix (codemod engine-model-to-top-level) across the remaining legacy-engine.model workflows to complete the migration to the new top-level model field while the change is fresh, and open a single cleanup PR to resolve or delete the 4 long-orphaned custom agent files.

This run confirms that most previously-identified gaps (--share, engine.args, add-dir) are durable, intentional non-issues at this repo's current usage patterns — no workflow currently needs conversation sharing, custom CLI args, or extra working directories. The one genuinely new, time-sensitive opportunity is the top-level model field migration, since it was merged less than 24 hours before this analysis and adoption is still in progress. The orphaned custom agents represent the longest-standing, easily-actionable cleanup item now overdue for resolution.

---

Critical Findings

🔴 High Priority Issues

• Complete the model field migration: 147 workflow files still use the deprecated nested engine.model: form despite the top-level model: field (and automated codemod) being available. Compile-time deprecation warnings are already firing for all of them. Run gh aw fix to auto-migrate before the deprecated form is removed in a future breaking release.

🟡 Medium Priority Opportunities

• Resolve 4 orphaned custom agent files (create-safe-output-type.agent.md, custom-engine-implementation.agent.md, grumpy-reviewer.agent.md, w3c-specification-writer.agent.md) — zero workflow references across 12 consecutive research runs. Either wire them into a workflow via engine.agent: or remove them to reduce repo clutter.
• disable-builtin-mcps under-used: only auto-triage-issues.md uses it outside this research workflow. Workflows with narrow, well-defined tool needs could benefit from disabling unused built-in MCP servers to reduce tool-call surface and prompt-injection risk.

---

View Full Analysis

1️⃣ Current State Analysis

View Copilot CLI Capabilities Inventory

Copilot CLI Capabilities Inventory

• Engine files reviewed: copilot_engine.go, copilot_engine_execution.go, copilot_engine_tools.go, copilot_mcp.go, copilot_installer.go, copilot_inline_driver.go, copilot_logs.go (all unchanged since last run except general repo activity via unrelated PR Route PR code-quality reviews through the Copilot gateway #52377).
• New feature this run: top-level model field at the workflow-frontmatter level, which takes precedence over engine.model and is documented as a replacement; engine.model is now deprecated with a compile-time warning. A gh aw fix codemod (engine-model-to-top-level) automates migration.
• CLI flags supported: --share, --add-dir, --agent (via engine.agent), --disable-builtin-mcps, --model (via COPILOT_MODEL env var set from the model field).
• Sandbox/network: AWF/SRT sandbox options, network: allowlist config, max-tool-denials, max-continuations (autopilot loop control) all unchanged.

View Usage Statistics

Usage Statistics

• Total workflows: 285
• Copilot workflows (combined engine: copilot + id: copilot): 138 (~48%)
• Top-level model: field usage overall: 98 workflows repo-wide; 32 of these are Copilot-engine workflows
• Legacy engine.model (deprecated) still present: ~147 workflow files
• cache-memory: 126 workflows (steady growth from ~101 several runs ago)
• copilot-sdk: true: ~71 workflows (steady growth)
• max-tool-denials: 65, max-continuations: 11, network config: 159
• engine.args: 0, --share: 1, add-dir: 0, disable-builtin-mcps: 2

---

2️⃣ Feature Usage Matrix

Feature Category	Available Features	Used	Not Used	Usage Rate
CLI Flags	--share, --add-dir, --agent, --disable-builtin-mcps	--agent (11), --disable-builtin-mcps (2)	--share (1, non-issue), --add-dir (0)	Mixed, ~25%
Engine Config	engine.model (deprecated), top-level model, engine.args, engine.env	top-level model (32/138 copilot), engine.env (21)	engine.args (0), legacy engine.model still in 147 files	Migration in progress
MCP Servers	Built-in GitHub MCP, custom MCP servers, disable-builtin-mcps	Custom MCP servers widely used	Explicit disable-builtin-mcps (2)	Low for this specific flag
Network Config	network: allowlist, defaults	159 workflows w/ explicit config	Rest rely on engine defaults	High
Sandbox Options	AWF, SRT	~255 workflows (majority)	Minority using custom/no sandbox	High
Custom Agents	.agent.md files via engine.agent	7 of 11 referenced	4 of 11 orphaned (12 runs)	64%

---

3️⃣ Missed Opportunities

View High Priority Opportunities

🔴 High Priority

Opportunity 1: Complete migration to top-level model field

• What: 147 workflow files still use the deprecated engine.model: nested field instead of the new top-level model: field introduced in PR Route PR code-quality reviews through the Copilot gateway #52377.
• Why It Matters: The deprecated form emits a compile-time warning on every gh aw compile run, adding noise, and risks breaking when the deprecated field is eventually removed in a future major version.
• Where: Run gh aw fix repo-wide; it will apply the engine-model-to-top-level codemod automatically to all affected files.
• How to Implement:

  gh aw fix --all   # or scoped to affected workflow files
  make recompile

• Example (before/after):

  # before (deprecated)
  engine:
    id: copilot
    model: gpt-4.1
  
  # after
  engine:
    id: copilot
  model: gpt-4.1

View Medium Priority Opportunities

🟡 Medium Priority

Opportunity 2: Resolve orphaned custom agent files

• What: create-safe-output-type.agent.md, custom-engine-implementation.agent.md, grumpy-reviewer.agent.md, w3c-specification-writer.agent.md have zero workflow references, stable for 12 consecutive research runs.
• Why It Matters: Dead configuration adds maintenance confusion; either these agents are useful (and should be wired in) or they should be removed.
• Where: .github/agents/
• How to Implement: Reference via engine: {agent: <name>} in a relevant workflow, or delete the unused .agent.md files.

Opportunity 3: Expand disable-builtin-mcps adoption

• What: Only auto-triage-issues.md uses disable-builtin-mcps outside this research workflow.
• Why It Matters: Workflows with narrow tool needs (e.g. single-purpose linters, report generators) reduce their prompt-injection/tool-call attack surface by disabling unused built-in MCP servers.
• Where: Single-purpose workflows like daily-* report generators that don't need the full built-in GitHub MCP toolset.
• How to Implement:

  engine:
    id: copilot
    disable-builtin-mcps: true

View Low Priority Opportunities

🟢 Low Priority

Opportunity 4: --share flag remains effectively unused

• What: Only this research workflow itself uses --share (12+ runs stable).
• Why It Matters: Low — no workflow currently has a clear need for shareable conversation links; treat as accepted, not a gap.
• Where: N/A — informational only.

Opportunity 5: add-dir and engine.args still at zero real usage

• What: No workflow passes custom --add-dir paths or engine.args CLI overrides.
• Why It Matters: Confirmed stable across 5+ direct re-verifications; the repo's workflows don't currently need extra working directories or custom CLI flags beyond what's already exposed via structured frontmatter fields.

---

4️⃣ Specific Workflow Recommendations

View Workflow-Specific Recommendations

Workflows using legacy engine.model (sample)

• Current State: Nested engine.model: field, triggering deprecation warnings at compile time.
• Recommended Changes: Run gh aw fix to migrate to top-level model:.
• Expected Benefits: Clean compiles with no deprecation warnings; future-proofed against removal of the legacy field.

auto-triage-issues.md

• Current State: Already uses disable-builtin-mcps — good example for other narrow-scope workflows to follow.

---

5️⃣ Trends & Insights

View Historical Trends

• Changes since last analysis (2026-08-12, run 31562860555): New top-level model field feature detected and already at 23% adoption among Copilot workflows within 24 hours of merge — fast organic uptake.
• Persistent stable metrics (12 consecutive runs): --share (1), engine.args (0), orphaned agent set (4 of original 5, one resolved).
• Growing adoption: cache-memory (126, up from ~101), Copilot SDK (copilot-sdk: true, ~71, up from 68).

---

6️⃣ Best Practice Guidelines

1. Adopt structured frontmatter fields over legacy nested config: Prefer top-level model: over engine.model: now that it's supported — keeps configs flatter and avoids deprecation warnings.
2. Prune unused custom agent definitions promptly: Files that go 2+ research cycles without a reference are strong candidates for removal to avoid config drift.
3. Scope MCP surface to actual need: Use disable-builtin-mcps for single-purpose workflows to reduce attack surface, rather than leaving all built-in MCP servers enabled by default.

---

---

7️⃣ Action Items

Immediate Actions (this week):

• Run gh aw fix (codemod engine-model-to-top-level) across workflows still using legacy engine.model, then make recompile.

Short-term (this month):

• Triage and resolve the 4 orphaned custom agent files (wire in or delete).
• Evaluate adding disable-builtin-mcps to narrow-scope daily report workflows.

Long-term (this quarter):

• Consider a CI lint check that flags custom agent .agent.md files with zero workflow references, to prevent future orphaning.
• Track engine.model deprecation removal timeline and ensure full migration completes before any breaking release.

---

View Supporting Evidence & Methodology

📚 References

• Copilot Engine source: pkg/workflow/copilot_engine.go, copilot_engine_execution.go, copilot_engine_tools.go, copilot_mcp.go
• Changeset: .changeset/engine-model-to-top-level.md (PR Route PR code-quality reviews through the Copilot gateway #52377, merged 2026-08-12)
• Previous research: repo-memory branch memory/copilot-cli-research, copilot-cli-research/latest.json and notes.md (11 prior runs, 2026-08-03 through 2026-08-12)

Research Methodology

Reviewed Copilot-related Go source files for new capabilities since the last tracked run via git log on key files. Cross-referenced with .changeset/ entries merged since the prior run's timestamp to catch newly-shipped features. Surveyed all 285 workflow markdown files with targeted grep/glob for engine config patterns (engine: copilot, id: copilot, model:, engine.model, engine.args, --share, add-dir, disable-builtin-mcps), distinguishing top-level vs. nested fields via anchored regex. Compared against persisted history in repo-memory to identify genuinely new findings vs. stable/duplicate ones, per prior run's recommendation to avoid issue-tracker noise on unchanged findings.

---

Generated by Copilot CLI Deep Research (Run: 31666761449)

Generated by 🔬 Copilot CLI Deep Research Agent · auto · 32 AIC · ⌖ 4.97 AIC · ⊞ 10.5K · ◷

• expires on Aug 13, 2026, 8:27 PM UTC-08:00