Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .agents/skills/apm-skill-discovery/instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
model: gpt-5-mini
---

You are the `apm-skill-discovery` test skill.

When invoked, return exactly this token on a single line:

`APM_SKILL_DISCOVERY_TOKEN_9f12f2f4e8b34a16`
1,574 changes: 1,574 additions & 0 deletions .github/workflows/test-copilot-apm-skill-discovery.lock.yml

Large diffs are not rendered by default.

24 changes: 24 additions & 0 deletions .github/workflows/test-copilot-apm-skill-discovery.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
on:
workflow_dispatch:
reaction: eyes

permissions: read-all

engine:
id: copilot

safe-outputs:
create-issue:
title-prefix: "[copilot-test] "
labels: [copilot, automation]
# min: 1
---

Use the `apm-skill-discovery` skill from `.agents/skills`.

Invoke that skill and copy its exact token response verbatim as the first line of the issue body.

Create an issue in repository `${{ github.repository }}` with title "Copilot APM skill discovery" and include:
- First line: the exact token returned by `apm-skill-discovery`
- Second line: `Skill used: apm-skill-discovery`
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,9 @@ done # without samples
```

The only workflows intentionally without `samples:` are
`test-copilot-custom-safe-outputs` and `test-copilot-dispatch-workflow`,
because their purpose is to exercise the live engine path.
`test-copilot-custom-safe-outputs`, `test-copilot-dispatch-workflow`, and
`test-copilot-apm-skill-discovery`, because their purpose is to exercise the
live engine path.

## 6. Running tests locally

Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ Tracking the full matrix of [safe outputs](https://github.com/github/gh-aw/blob/

- [x] `dispatch-workflow` — test-copilot-dispatch-workflow
- [x] `call-workflow` — test-copilot-call-workflow (worker: test-copilot-call-worker)
- [x] `APM skill discovery` — test-copilot-apm-skill-discovery
- [x] `upload-asset` — test-copilot-upload-asset (auto-creates orphaned `assets/` branch)
- [ ] `upload-artifact` — test-copilot-upload-artifact
- [x] `noop` — test-copilot-noop
Expand Down
37 changes: 36 additions & 1 deletion e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,7 @@ get_all_tests() {
echo "test-claude-create-issue"
echo "test-codex-create-issue"
echo "test-copilot-create-issue"
echo "test-copilot-apm-skill-discovery"
echo "test-claude-create-discussion"
echo "test-codex-create-discussion"
echo "test-copilot-create-discussion"
Expand Down Expand Up @@ -1646,6 +1647,33 @@ validate_issue_created() {
fi
}

validate_issue_body_contains() {
local title_prefix="$1"
local expected_body_text="$2"
local repo="${3:-}"

local repo_flag=""
if [[ -n "$repo" ]]; then
repo_flag="--repo $repo"
fi

local issue_number=$(gh issue list $repo_flag --limit 10 --json number,title --jq ".[] | select(.title | startswith(\"$title_prefix\")) | .number" | head -1)

if [[ -z "$issue_number" ]]; then
error "No issue found with title prefix: $title_prefix"
return 1
fi

local issue_body=$(gh issue view $repo_flag "$issue_number" --json body --jq '.body')
if echo "$issue_body" | grep -qF "$expected_body_text"; then
success "Issue #$issue_number body contains expected text: $expected_body_text"
return 0
fi

error "Issue #$issue_number body missing expected text: $expected_body_text"
return 1
}

validate_comment() {
local issue_number="$1"
local expected_comment_text="$2"
Expand Down Expand Up @@ -3422,7 +3450,7 @@ run_single_test() {
fi
;;
# Workflow dispatch tests - triggered with gh aw run
*"create-issue"|*"create-discussion"|*"create-pull-request"|*"create-two-pull-requests"|*"code-scanning-alert"|*"create-check-run"|*"mcp"*|*"safe-jobs"|*"gh-steps"|*"custom-safe-outputs"|*"noop"|*"report-incomplete"|*"assign-to-agent"|*"set-issue-field")
*"apm-skill-discovery"|*"create-issue"|*"create-discussion"|*"create-pull-request"|*"create-two-pull-requests"|*"code-scanning-alert"|*"create-check-run"|*"mcp"*|*"safe-jobs"|*"gh-steps"|*"custom-safe-outputs"|*"noop"|*"report-incomplete"|*"assign-to-agent"|*"set-issue-field")
local workflow_success=false
if trigger_workflow_dispatch_and_await_completion "$workflow"; then
workflow_success=true
Expand All @@ -3431,6 +3459,13 @@ run_single_test() {
if [[ "$workflow_success" == true ]]; then
local validation_success=false
case "$workflow" in
*"apm-skill-discovery")
local title_prefix=$(get_title_prefix "$workflow" "$ai_type")
local expected_labels=$(get_expected_labels "$ai_type")
if validate_issue_created "$title_prefix" "$expected_labels" "$target_repo" && validate_issue_body_contains "$title_prefix" "APM_SKILL_DISCOVERY_TOKEN_9f12f2f4e8b34a16" "$target_repo"; then
validation_success=true
fi
;;
*"multi")
local title_prefix=$(get_title_prefix "$workflow" "$ai_type")
local expected_labels=$(get_expected_labels "$ai_type")
Expand Down