diff --git a/.github/agents/agentic-workflows.agent.md b/.github/agents/agentic-workflows.agent.md index e34a4da6123..c0f21877e1b 100644 --- a/.github/agents/agentic-workflows.agent.md +++ b/.github/agents/agentic-workflows.agent.md @@ -18,6 +18,7 @@ This is a **dispatcher agent** that routes your request to the appropriate speci - **Creating report-generating workflows**: Routes to `report` prompt — consult this whenever the workflow posts status updates, audits, analyses, or any structured output as issues, discussions, or comments - **Creating shared components**: Routes to `create-shared-agentic-workflow` prompt - **Fixing Dependabot PRs**: Routes to `dependabot` prompt — use this when Dependabot opens PRs that modify generated manifest files (`.github/workflows/package.json`, `.github/workflows/requirements.txt`, `.github/workflows/go.mod`). Never merge those PRs directly; instead update the source `.md` files and rerun `gh aw compile --dependabot` to bundle all fixes +- **Analyzing test coverage**: Routes to `test-coverage` prompt — consult this whenever the workflow reads, analyzes, or reports on test coverage data from PRs or CI runs Workflows may optionally include: @@ -118,6 +119,16 @@ When you interact with this agent, it will: - "Bundle and close the Dependabot PRs for workflow dependencies" - "Update @playwright/test to fix the Dependabot PR" +### Analyze Test Coverage +**Load when**: The workflow reads, analyzes, or reports test coverage — whether triggered by a PR, a schedule, or a slash command. Always consult this prompt before designing the coverage data strategy. + +**Prompt file**: https://github.com/github/gh-aw/blob/main/.github/aw/test-coverage.md + +**Use cases**: +- "Create a workflow that comments coverage on PRs" +- "Analyze coverage trends over time" +- "Add a coverage gate that blocks PRs below a threshold" + ## Instructions When a user interacts with you: diff --git a/.github/aw/create-agentic-workflow.md b/.github/aw/create-agentic-workflow.md index 906f092e89f..3b700a066f5 100644 --- a/.github/aw/create-agentic-workflow.md +++ b/.github/aw/create-agentic-workflow.md @@ -208,7 +208,7 @@ These resources contain workflow patterns, best practices, safe outputs, and per - `Package.swift`, `*.podspec` → add `swift` - `composer.json` → add `php` - `pubspec.yaml` → add `dart` - - 💡 If you detect the task requires **browser automation**, suggest the **`playwright`** tool. + - 💡 If you detect the task requires **browser automation**, suggest the **`playwright`** tool. For **visual regression testing** (comparing screenshots across PRs), consult `.github/aw/visual-regression.md` for the reference pattern using `playwright` + `cache-memory`. - 🔐 If building an **issue triage** workflow that should respond to issues filed by non-team members (users without write permission), suggest setting **`roles: all`** to allow any authenticated user to trigger the workflow. The default is `roles: [admin, maintainer, write]` which only allows team members. **Scheduling Best Practices:** diff --git a/.github/aw/visual-regression.md b/.github/aw/visual-regression.md new file mode 100644 index 00000000000..dd1bf45fbf7 --- /dev/null +++ b/.github/aw/visual-regression.md @@ -0,0 +1,58 @@ +--- +name: visual-regression +description: Reference prompt for visual regression testing using playwright + cache-memory for baseline screenshot storage across pull requests +--- + +# Visual Regression Testing + +Use `playwright` for screenshots and `cache-memory` to persist baselines between PR runs. + +## Example Workflow + +```markdown +--- +description: Capture screenshots on every PR and compare against cached baselines to detect visual regressions +on: + pull_request: + types: [opened, synchronize, reopened] +permissions: + contents: read + pull-requests: read +engine: copilot +tools: + playwright: + allowed_domains: + - localhost + - 127.0.0.1 + cache-memory: + key: visual-regression-baselines-${{ github.event.pull_request.base.ref }} + retention-days: 30 + allowed-extensions: [".png", ".json"] + bash: + - "mkdir *" + - "cp *" + - "diff *" + - "date *" +safe-outputs: + add-comment: + max: 1 +timeout-minutes: 30 +--- + +Build and serve the app locally, then use Playwright to capture full-page screenshots of key pages into `/tmp/visual-regression/current/`. + +Use filesystem-safe timestamps (no colons — colons break artifact uploads): +`date -u "+%Y-%m-%d-%H-%M-%S"` + +If `/tmp/gh-aw/cache-memory/baselines/manifest.json` does not exist, copy screenshots there as new baselines and post: "Baselines initialized — N pages captured." + +Otherwise compare each screenshot to its baseline. Post a comment summarizing: pages unchanged / pages with diffs. If nothing changed, use the `noop` safe-output. +``` + +## Key Design Decisions + +- **`cache-memory` key per base branch** — scopes baselines to `main`, `develop`, etc. independently +- **`allowed_domains: [localhost, 127.0.0.1]`** — prevents SSRF; app must be served locally +- **`retention-days: 30`** — keeps baselines beyond the default 7-day cache expiry +- **Filesystem-safe timestamps** — `YYYY-MM-DD-HH-MM-SS` format; colons are invalid in artifact filenames +- **Minimal permissions** — all PR writes go through `safe-outputs`, not GitHub tools