Skip to content

Add deployment_status trigger guidance to create-agentic-workflow prompt - #18274

Merged
pelikhan merged 4 commits into
mainfrom
copilot/add-deployment-status-trigger-guidance
Feb 25, 2026
Merged

Add deployment_status trigger guidance to create-agentic-workflow prompt#18274
pelikhan merged 4 commits into
mainfrom
copilot/add-deployment-status-trigger-guidance

Conversation

Copilot AI commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

The agent was defaulting to workflow_run + if:failure for all deployment monitoring scenarios, missing deployment_status as the correct trigger for external deployment services (Heroku, Vercel, Railway, Fly.io) that post status back to GitHub.

Changes

  • .github/aw/create-agentic-workflow.md — Extended the trigger inference list with:

    • deployment_status entry for external deployment monitoring, with the correct if: condition and a reference to the new example
    • workflow_run entry for GitHub Actions-internal pipeline monitoring
    • An explicit decision rule distinguishing the two: use deployment_status for services integrating with the GitHub Deployments API; use workflow_run for in-repo Actions pipelines
  • .github/aw/deployment-status.md (new) — Reference prompt (not a runnable workflow) that the agent consults when creating a deployment monitoring workflow. Formatted like report.md with only description in the frontmatter. The body includes:

    • Recommended frontmatter snippet with deployment_status trigger, read-only permissions, and safe-outputs
    • Table of available github.event.deployment_status.* and github.event.deployment.* context expressions
    • Agent instructions pattern for verifying the failure, checking duplicates, and filing an incident issue
    • Decision guide for choosing deployment_status vs workflow_run
Original prompt

This section details on the original issue you should resolve

<issue_title>[plan] Add deployment_status trigger guidance to create-agentic-workflow prompt</issue_title>
<issue_description>## Objective

Add deployment_status event documentation and examples to the create-agentic-workflow prompt so the agent suggests it for external deployment monitoring scenarios instead of defaulting to workflow_run.

Context

From Agent Persona Exploration discussion github/gh-aw#18250, the DO-1 (Failed Deployment Incident Creator) scenario scored 4.4/5.0. The agent applies the workflow_run + if:failure pattern correctly but misses deployment_status as a better trigger for external deployment services (e.g., Heroku, Vercel, Railway, Fly.io).

Approach

  1. Locate the create-agentic-workflow prompt/skill
  2. Add a section on deployment_status event:
    • Explain when to use it: external deployment services that post deployment status back to GitHub
    • Show the trigger syntax:
      on:
        deployment_status:
    • Add if: github.event.deployment_status.state == 'failure' condition example
  3. Distinguish from workflow_run: use deployment_status for external deployments, workflow_run for GitHub Actions-internal pipelines
  4. Consider adding a reference workflow for deployment incident creation

Files to Modify

  • Update: the create-agentic-workflow prompt/skill with deployment_status guidance
  • Optionally create: .github/workflows/deployment-incident.md reference workflow

Acceptance Criteria

  • deployment_status trigger is documented in the create prompt
  • Agent suggests deployment_status when user describes external deployment monitoring
  • Clear guidance on when to choose deployment_status vs workflow_run

Generated by Plan Command for issue #discussion #18250

  • expires on Feb 27, 2026, 4:46 AM UTC

Comments on the Issue (you are @copilot in this section)

@pelikhan Use .github/aw/deployment-status.md

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add deployment_status trigger guidance to create-agentic-workflow prompt Add deployment_status trigger guidance to create-agentic-workflow prompt Feb 25, 2026
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot no write permissions

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Feb 25, 2026

Copy link
Copy Markdown
Contributor Author

@copilot no write permissions

Fixed in f3ff02a — changed issues: write to issues: read. The safe-outputs system handles the actual write operation, so the workflow only needs read permissions.

@@ -0,0 +1,60 @@
---

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot format this file as a prompt, not an agentic workflow

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 4913808 — reformatted as a prompt with just description in the frontmatter. The file now follows the same pattern as report.md and github-agentic-workflows.md: reference/instructional content the agent consults when creating a deployment_status workflow, including the recommended frontmatter snippet, available event expressions, an agent instructions template, and the deployment_status vs workflow_run decision guide.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review February 25, 2026 06:22
Copilot AI review requested due to automatic review settings February 25, 2026 06:22
@pelikhan
pelikhan merged commit 4570f89 into main Feb 25, 2026
@pelikhan
pelikhan deleted the copilot/add-deployment-status-trigger-guidance branch February 25, 2026 06:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the agentic workflow creation guidance so external deployment failure monitoring uses the deployment_status trigger (instead of defaulting to workflow_run) and adds a reference prompt for deployment-status-based incident workflows.

Changes:

  • Extend trigger inference guidance in create-agentic-workflow.md to distinguish deployment_status (external deploy services) vs workflow_run (in-repo pipelines).
  • Add .github/aw/deployment-status.md as a reference pattern for deployment failure monitoring and incident issue creation.
  • Update the dispatcher agent doc to include routing for the test-coverage prompt.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
.github/aw/deployment-status.md Adds a reference pattern (frontmatter + context expressions + instruction template) for deployment_status-driven incident creation.
.github/aw/create-agentic-workflow.md Adds explicit trigger-selection guidance for external deployments vs internal workflow monitoring.
.github/agents/agentic-workflows.agent.md Documents a new dispatcher route/use-case section for test coverage analysis workflows.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +36 to +40
| Expression | Description |
|---|---|
| `${{ github.event.deployment.environment }}` | Target environment (e.g. `production`) |
| `${{ github.event.deployment_status.state }}` | Status (`failure`, `success`, `error`, etc.) |
| `${{ github.event.deployment_status.target_url }}` | URL to the external service deployment logs |

Copilot AI Feb 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The markdown table is using || at the start of rows, which won’t render as a table in standard Markdown. Use single | delimiters (like other docs in this repo) so the expressions/context list renders correctly.

Copilot uses AI. Check for mistakes.
- Issue automation → `on: issues: types: [opened, edited]` (add `workflow_dispatch:` manually if manual runs needed)
- PR automation → `on: pull_request: types: [opened, synchronize]` (add `workflow_dispatch:` manually if manual runs needed)
- Scheduled tasks → `on: schedule: daily on weekdays` (prefer weekdays to avoid Monday backlog - workflow_dispatch auto-added for fuzzy schedules only)
- **External deployment monitoring** (Heroku, Vercel, Railway, Fly.io, etc.) → `on: deployment_status:` with `if: ${{ github.event.deployment_status.state == 'failure' }}` — use this when third-party services post deployment status back to GitHub. See reference: @.github/aw/deployment-status.md

Copilot AI Feb 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guidance filters only on github.event.deployment_status.state == 'failure', but deployment statuses may also report unsuccessful deployments as error. To avoid missing failures from external providers, consider updating the recommended condition to include both failure and error (or a non-success check).

Suggested change
- **External deployment monitoring** (Heroku, Vercel, Railway, Fly.io, etc.) → `on: deployment_status:` with `if: ${{ github.event.deployment_status.state == 'failure' }}` — use this when third-party services post deployment status back to GitHub. See reference: @.github/aw/deployment-status.md
- **External deployment monitoring** (Heroku, Vercel, Railway, Fly.io, etc.) → `on: deployment_status:` with `if: ${{ github.event.deployment_status.state == 'failure' || github.event.deployment_status.state == 'error' }}` — use this when third-party services post deployment status back to GitHub. See reference: @.github/aw/deployment-status.md

Copilot uses AI. Check for mistakes.
Comment on lines +13 to +20
```yaml
on:
deployment_status:
if: ${{ github.event.deployment_status.state == 'failure' }}
permissions:
contents: read
issues: read
deployments: read

Copilot AI Feb 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The recommended if: filter is indented with a leading space, which makes the YAML snippet invalid if copied verbatim. Also, deployment_status.state can be error as well as failure for unsuccessful deployments, so filtering only on 'failure' can miss real deploy failures—consider matching both (failure or error) or using a broader non-success check.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[plan] Add deployment_status trigger guidance to create-agentic-workflow prompt

3 participants