Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions .github/workflows/copilot-agent-analysis.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

117 changes: 4 additions & 113 deletions .github/workflows/copilot-agent-analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,19 +85,6 @@ Daily analysis of pull requests created by copilot-swe-agent in the last 24 hour
- `/tmp/gh-aw/agent/pr-data/copilot-prs.json` - Full PR data in JSON format
- `/tmp/gh-aw/agent/pr-data/copilot-prs-schema.json` - Schema showing the structure

You can use `jq` to process this data directly. For example:
```bash
# Get PRs from the last 24 hours
TODAY="$(date -d '24 hours ago' '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || date -v-24H '+%Y-%m-%dT%H:%M:%SZ')"
jq --arg today "$TODAY" '[.[] | select(.createdAt >= $today)]' /tmp/gh-aw/agent/pr-data/copilot-prs.json

# Count total PRs
jq 'length' /tmp/gh-aw/agent/pr-data/copilot-prs.json

# Get PR numbers for the last 24 hours
jq --arg today "$TODAY" '[.[] | select(.createdAt >= $today) | .number]' /tmp/gh-aw/agent/pr-data/copilot-prs.json
```

**Alternative Approaches** (if you need additional data not in the pre-fetched file): Use `gh pr list --search "head:copilot/"` with `--state all` or `list_pull_requests` filtered by `user.login == "app/github-copilot"` or branch prefix `copilot/`.

### Phase 2: Analyze Each PR
Expand Down Expand Up @@ -175,23 +162,7 @@ The history file should contain daily metrics in this format:

**If Historical Data is Missing or Incomplete:**

If the history file doesn't exist or has gaps in the data, rebuild it by querying historical PRs:

1. **Determine Missing Date Range**: Identify which dates need data (up to last 3 days maximum for concise trends)

2. **Query PRs One Day at a Time**: To avoid context explosion, query PRs for each missing day separately

3. **Process Each Day**: For each day with missing data:
- Query PRs created on that specific date
- Calculate the same metrics as for today (total PRs, merged, closed, success rate, etc.)
- Store in the history file
- Limit to 3 days total to keep reports concise

4. **Simplified Approach**:
- Process one day at a time in chronological order (oldest to newest)
- Save after each day to preserve progress
- **Stop at 3 days** - this is sufficient for concise trend analysis
- Prioritize most recent days first
{{#runtime-import? shared/copilot-agent-analysis-history.md}}

#### 4.2 Store Today's Metrics

Expand All @@ -211,42 +182,6 @@ mkdir -p /tmp/gh-aw/repo-memory/default/copilot-agent-metrics/

Store the data in JSON format with proper structure.

#### 4.2.1 Rebuild Historical Data (if needed)

**When to Rebuild:**
- History file doesn't exist
- History file has gaps (missing dates in the last 3 days)
- Insufficient data for trend analysis (< 3 days)

**Rebuilding Strategy:**
1. **Assess Current State**: Check how many days of data you have
2. **Target Collection**: Aim for 3 days maximum (for concise trends)
3. **One Day at a Time**: Query PRs for each missing date separately to avoid context explosion

**For Each Missing Day:**
```
# Query PRs for specific date using keyword search
repo:${{ github.repository }} is:pr "START COPILOT CODING AGENT" created:YYYY-MM-DD..YYYY-MM-DD
```

Or use `list_pull_requests` with date filtering and filter results by agent criteria (see `agent_prs_total` in scratchpad/metrics-glossary.md for scope).

**Process:**
- Start with the oldest missing date in your target range (maximum 3 days ago)
- For each date:
1. Search for PRs created on that date
2. Analyze each PR (same as Phase 2)
3. Calculate daily metrics (same as Phase 4.2)
4. Add to history.json
5. Save immediately to preserve progress
- Stop at 3 days total

**Important Constraints:**
- Process dates in chronological order (oldest first)
- Save after processing each day
- **Maximum 3 days** of historical data for concise reporting
- Prioritize data quality over quantity

#### 4.4 Analyze Trends

**Concise Trend Analysis** - If historical data exists (at least 3 days), show:
Expand Down Expand Up @@ -302,67 +237,23 @@ _Generated by Copilot Agent Analysis (Run: [run_id])_
<details>
<summary><b>Agent Task Texts</b></summary>

[Show this table for all PRs created in the last 24 hours - extract task text from PR body]

| PR # | Status | Task Text (first 100 chars) |
|------|--------|----------------------------|
| [#number]([url]) | [status] | [First 100 characters of PR body/task description...] |
| [#number]([url]) | [Merged/Closed/Open] | [PR body truncated to 100 chars, or "No description provided"] |

</details>

<details>
<summary><b>Notable PRs</b></summary>

[Only list if there are failures, closures, or issues - otherwise omit this section]

**Issues ⚠️**
- **PR #[number]**: [title] - [brief reason for failure/closure]

**Open PRs ⏳**
[Only list if open for >24 hours]
- **PR #[number]**: [title] - [age]
[Only if failures, closures, or PRs open >24h — otherwise omit]

</details>

**Key Insights**

[1-2 bullet points only, focus on actionable items or notable observations]
[1-2 bullets max — omit if nothing notable]
```

**Agent Task Texts Table Instructions:**

The "Agent Task Texts" section should include a table showing all PRs created in the last 24 hours with their task text:

1. **For each PR created in the last 24 hours:**
- Extract the PR number and URL
- Determine the status (Merged, Closed, or Open)
- Extract the task text from the PR's `body` field (this is the original task description)
- Truncate the task text to the first 100 characters for display in the table
- If the body is empty or null, show "No description provided"

2. **Table Format:**
```markdown
| PR # | Status | Task Text (first 100 chars) |
|------|--------|----------------------------|
| [#123](https://github.com/owner/repo/pull/123) | Merged | Fix the login validation to handle edge cases where users enter special char... |
| [#124](https://github.com/owner/repo/pull/124) | Open | Implement new feature for exporting reports in CSV format with proper heade... |
```

3. **Status Values:**
- "Merged" - PR was successfully merged
- "Closed" - PR was closed without merging
- "Open" - PR is still open

4. **If no PRs in last 24 hours:**
- Omit the "Agent Task Texts" section entirely

**Important Brevity Guidelines:**
- **Skip the "PR Summary Table"** - use simple 3-day metrics table instead
- **Omit "Detailed PR Analysis"** section - only show notable PRs with issues
- **Skip "Weekly Summary"** and **"Monthly Summary"** sections - use 3-day trend only
- **Remove "Instruction File Changes"** section entirely
- **Eliminate "Recommendations"** section - fold into "Key Insights" (1-2 bullets max)
- **Remove verbose methodology** and historical context sections
{{/if}}
{{#if experiments.output_format == 'prose' }}
**Prose Variant Body Template**:
Expand Down
16 changes: 7 additions & 9 deletions .github/workflows/copilot-pr-merged-report.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading