[prompt-analysis] Copilot PR Prompt Analysis - 2026-07-28

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-07-28

Summary

Analysis Period: Last 30 days (2026-07-06 → 2026-07-28)
Total PRs: 1000 | Merged: 746 (74.6%) | Closed: 246 (24.6%) | Open: 8 (0.8%)

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Refactoring	4	4	0	100.0%
Feature Addition	232	181	49	78.7%
Bug Fix	739	549	184	74.9%
Testing	10	5	5	50.0%
Documentation	4	2	2	50.0%
Other	11	5	6	45.5%

Note: categories are keyword-matched from title+body, most bodies mix multiple intents so category totals overlap (e.g. a "bug fix" prompt can also mention "add").

Prompt Analysis

✅ Successful vs ❌ unsuccessful prompt characteristics

Length: Average prompt length is essentially identical for both outcomes (Merged: 205.3 words, Closed: 205.0 words) — length alone is not a discriminator in this dataset.

Code specificity: Merged prompts reference concrete files/functions (backtick-quoted paths or func() calls) slightly more often than closed ones — 86.6% vs 83.7%. A modest but consistent signal that grounding a prompt in specific code artifacts correlates with success.

Conventional-commit style titles (fix:, feat:, refactor:, etc.): 43.0% of merged PRs use this prefix vs only 33.7% of closed PRs — a ~9pt gap, the largest differentiator found.

"daily"/"report" scheduled-workflow PRs: These make up 7.0% of merged PRs but 10.2% of closed PRs — automated recurring-report workflow PRs close at a disproportionately higher rate, likely because they are exploratory/experimental by nature.

Top keywords (merged): workflow, github, actions, runs, copilot, model, engine, agent, comment
Top keywords (closed): workflow, github, copilot, daily, agent, actions, issue, report, workflows — closed prompts skew toward "daily"/"report"/"issue" vocabulary, consistent with the recurring-workflow finding above.

Example merged prompts:

1. PR Add daily agentic spending forecast report #48654: "Adds a daily, manually rerunnable workflow that forecasts agentic workflow spending and reviews sour..." → Merged
2. PR daily-model-inventory: log raw responses in collect steps, drop copilot-api job #48636: "Raw model API responses were not visible in action logs, requiring artifact downloads to inspect the..." → Merged
3. PR Add Copilot SDK billing inventory job #48619: "The daily model inventory needs Copilot billing metadata sourced through the Copilot CLI SDK using `..." → Merged

Example closed prompts:

1. PR Refactor AWF helpers into focused workflow modules #48633: "pkg/workflow/awf_helpers.go had grown into a large mixed-responsibility file, with its coverage co..." → Closed
2. PR fix: use shell env var for sink-visibility to prevent double-quoting in MCP gateway JSON config #48632: "PR fix: move sink-visibility to env var to prevent zizmor template-injection #48589 introduced a double-quoting bug that caused MCP gateway startup to fail with 'Configuratio..." → Closed
3. PR Update daily model inventory Copilot /models discovery across API hosts #48630: "The daily model inventory workflow was querying a single Copilot /models endpoint without the vers..." → Closed

Key Insights

• Pattern 1: Conventional-commit-style titles (fix:, feat(scope):, etc.) correlate with a ~9-point higher merge rate (43.0% vs 33.7% among closed PRs) — the strongest signal identified this cycle.
• Pattern 2: Prompts naming specific files or functions (backtick-quoted) merge slightly more often (86.6% vs 83.7%), reinforcing that grounding requests in concrete code artifacts helps.
• Pattern 3: "Daily"/"report" scheduled-workflow PRs close at nearly 1.5x the base rate (10.2% of closed vs 7.0% of merged) — these exploratory automation PRs may benefit from tighter scoping before submission.
• Prompt length is not a differentiator this cycle (~205 words average for both outcomes), contradicting the "conciseness matters" hypothesis from earlier analyses — sample data suggests structure/specificity matters more than raw length.

Recommendations

1. DO: Prefix PR titles/prompts with a clear conventional-commit-style intent (fix:, feat:, refactor:) — this is the strongest correlate with merge success found so far.
2. DO: Reference specific files, functions, or error messages directly in the prompt rather than describing the problem abstractly.
3. AVOID: Framing recurring/scheduled "daily report" workflow changes loosely — these have the highest closure rate; scope them tightly with a single, testable outcome.

Historical Trends

Date	PRs	Merged	Closed	Open	Success Rate
2026-07-28	1000	746	246	8	74.6%
2026-07-27	1000	764	231	5	76.4%
2026-07-26	1000	764	230	6	76.4%
2026-07-25	1000	764	233	3	76.4%
2026-07-24	1000	766	231	3	76.6%
2026-07-23	1000	764	231	5	76.4%
2026-07-22	1000	760	237	3	76.0%

Trend: Merge success rate dipped slightly today (74.6%) versus the fairly stable ~76% seen over the prior 6 days — within normal day-to-day noise given the rolling 30-day window, but worth watching if the decline continues.

---

Generated by Copilot PR Prompt Analysis (Run: 30398573752)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · sonnet50 · 22 AIC · ⌖ 10.6 AIC · ⊞ 10.3K · ◷

• expires on Jul 29, 2026, 1:03 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-07-29T21:03:35.985Z.

Closed by Workflow