feat: Replace IssueOps with deterministic orchestrator + agentic workers

[AlexDeMichieli]

TL;DR

This PR introduces architectural changes from discussion #14 and issue #17, evolved through E2E testing on volcano-coffee EMU enterprise and review feedback. It replaces the IssueOps batch orchestrator with a two-tier architecture: a deterministic classic Actions orchestrator scans repos and dispatches N agentic migration workers that each clone a target repo, perform the migration, and open a cross-repo PR — all from .github-private.

---

Before and After

Before: IssueOps — orchestrator creates issues, Copilot runs on target repos

.github-private                              Target repo
┌────────────────────┐                       ┌──────────────────┐
│ submit-repos.yml   │                       │                  │
│ + submit-repos.js  │── create issue ──────►│ Issue #1         │
│                    │── assign Copilot ────►│ Copilot runs     │
│ (PAT: full r+w)   │                       │ HERE             │
└────────────────────┘                       │ - reads code     │
                                             │ - writes .yml    │
Problems:                                    │ - opens PR       │
- PAT does reads AND writes                  └──────────────────┘
- No limit on issues created                 Needs per-repo:
- No scan of issue content                   - copilot env + MCP token
- Tag clear fails = duplicates               - hooks pre-deployed
- SSO token bug blocks assignments           - firewall allowlist
- Failed assignments = ghost sessions

After: Deterministic orchestrator dispatches agentic workers from .github-private

.github-private                              Target repo
┌──────────────────────────────┐             ┌──────────────────┐
│                              │             │                  │
│ submit-repos.yml (classic)   │             │  (passive)       │
│ ┌──────────────────────────┐ │             │                  │
│ │ Jobs (deterministic):    │ │             │                  │
│ │ 1. prepare: parse orgs   │ │             │                  │
│ │ 2. scan-and-dispatch:    │ │             │                  │
│ │    per org (matrix):     │ │             │                  │
│ │    - mint App token      │ │             │                  │
│ │    - scan repos (JS)     │ │             │                  │
│ │    - create tracking     │ │             │                  │
│ │      issue               │ │             │                  │
│ │    - dispatch N workers  │ │             │                  │
│ └──────────┬───────────────┘ │             │                  │
│            │                 │             │                  │
│ ┌──────────▼───────────────┐ │             │                  │
│ │ migration-worker.md (×N) │ │             │                  │
│ │ (agentic workflow)       │ │             │                  │
│ │ - reads target via API ────────────────►│                  │
│ │ - reads agents/ + knowledge/ locally   │                  │
│ │ - AI performs migration   │ │             │                  │
│ │ - create-pull-request ──────── safe ───►│ PR #1            │
│ │ - reports to tracking    │ │  output     │                  │
│ │   issue                  │ │             │                  │
│ └──────────────────────────┘ │             │                  │
└──────────────────────────────┘             └──────────────────┘
                                              No Copilot session
AI runs in .github-private:                   No issue created
- read-only sandbox                           No hooks needed
- threat detection on output                  No MCP token needed
- safe output creates PR                      No firewall allowlist
- knowledge/ already local                    Zero per-repo setup

Key change: The AI that performs the migration moves from Copilot coding agent sessions on target repos to gh-aw agentic workers running inside .github-private. Target repos are passive — they only receive a PR. The orchestrator is a classic Actions workflow (no AI tokens consumed for scanning/dispatching).

---

What this PR does

File	Type	Purpose
submit-repos.yml	Classic Actions workflow	Deterministic orchestrator: scans repos per org (matrix), creates tracking issue, dispatches worker per eligible repo
migration-worker.md + .lock.yml	Agentic workflow	Per-repo worker: reads target via API, reads local agents/knowledge, AI performs migration, opens cross-repo PR via safe output
scan-repositories.js	Script	Scans repos for GH_MIGRATION_TYPE custom property, idempotency check, returns eligible list
create-tracking-issue.js	Script	Creates [Migration Batch] tracking issue on .github-private
dispatch-migrations.js	Script	Dispatches migration-worker workflow per eligible repo

How it works

submit-repos.yml triggers (manual or weekly)
│
├─ prepare job: parse ORGANIZATIONS variable into matrix
│
├─ scan-and-dispatch job (per org, parallel):
│  ├─ Mint GitHub App token for this org
│  ├─ scan-repositories.js:
│  │   ├─ List repos, read GH_MIGRATION_TYPE property
│  │   ├─ Idempotency: search for open "[Actions Migration]" issues → skip
│  │   ├─ Retry on 403/429 (up to 3x)
│  │   └─ Return eligible repos (up to BATCH_SIZE)
│  ├─ create-tracking-issue.js:
│  │   └─ Create "[Migration Batch] <date> - N repos" issue
│  └─ dispatch-migrations.js:
│      └─ For each eligible repo: dispatch migration-worker.md
│         inputs: target_repo, migration_type, tracking_issue, target_repo_owner
│
└─ Each worker runs independently:
   ├─ Reads target repo via GitHub API (repos toolset)
   ├─ Reads agent file + knowledge base (local to .github-private)
   ├─ AI performs the migration
   ├─ Opens draft PR on target repo (create-pull-request safe output)
   └─ Reports status to tracking issue (add-comment safe output)

Observability: single pane of glass

Workers are isolated — a failure in repo A doesn't affect repo B. The tracking issue aggregates status:

[Migration Batch] April 13 - 5 repos

✅ org/repo-1: PR #5 opened
✅ org/repo-3: PR #2 opened
❌ org/repo-2: no CI source files found
🔄 org/repo-4: in progress...
🔄 org/repo-5: in progress...

Why the orchestrator is a classic .yml (not agentic)

Scanning and dispatching is purely mechanical — zero AI needed. Using a classic workflow saves ~2K AI tokens per run and avoids the COPILOT_GITHUB_TOKEN dependency for the orchestrator. The trade-off: we lose gh aw replay on the orchestrator, but the idempotency check in scan-repositories.js (search for open migration issues before dispatching) prevents duplicates on re-run. Workers keep full agentic resilience — each failed worker migration can be replayed individually via gh aw replay.

Resolved questions

• MCP token for manual mode: Still needed. Manual mode (Copilot Chat + custom agent on a target repo) requires COPILOT_MCP_GITHUB_PERSONAL_ACCESS_TOKEN. settings.yml continues provisioning. Batch mode doesn't need it.

Known blockers

• Copilot CLI crashes inside gh-aw container on EMU — tracked in https://github.com/github/amer-es-nes/issues/104. Token works locally, crashes in the container. Waiting for gh-aw team investigation.

Testing checklist

• Deterministic scan finds eligible repos and logs per-repo decisions
• Idempotency: skips repos with open [Actions Migration] issues
• Tracking issue created with batch summary
• dispatch-workflow triggers migration-worker.md per eligible repo
• Worker reads target repo via API
• Worker creates Actions workflow, archives originals, writes migration report
• create-pull-request opens draft PR on target repo
• Worker reports status to tracking issue via add-comment
• noop called when no eligible repos found
• Threat detection scans PR content before creation

[Copilot]

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

[Copilot]

Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.

[antgrutta]

@AlexDeMichieli, I gave this a try and want to simplify a few things:

Remove hooks from this PR. Hooks require commits to individual repositories, and there's nuance there that needs more exploration. Let's treat hooks as a separate feature for later.

Rethink PRU usage. Scanning repos for a custom property and creating an issue is a deterministic operation, it shouldn't require AI. When a customer migrates pipelines in batch, we want to give them a single pane of glass to orchestrate and manage everything. Moving to an agentic workflow should let us identify eligible repos (up to the batch size), clone them, run the migration, and open a PR on each target repository — all from one workflow in the orchestration repository (.github-private).

Proposed architecture: A standard (deterministic) workflow identifies eligible repositories up to the batch number, then kicks off N agentic workflows. Each agentic workflow takes a target repo as input, performs the migration, and opens a PR on that target repository.

[AlexDeMichieli]

@AlexDeMichieli, I gave this a try and want to simplify a few things:

Remove hooks from this PR. Hooks require commits to individual repositories, and there's nuance there that needs more exploration. Let's treat hooks as a separate feature for later.

Rethink PRU usage. Scanning repos for a custom property and creating an issue is a deterministic operation, it shouldn't require AI. When a customer migrates pipelines in batch, we want to give them a single pane of glass to orchestrate and manage everything. Moving to an agentic workflow should let us identify eligible repos (up to the batch size), clone them, run the migration, and open a PR on each target repository — all from one workflow in the orchestration repository (.github-private).

Proposed architecture: A standard (deterministic) workflow identifies eligible repositories up to the batch number, then kicks off N agentic workflows. Each agentic workflow takes a target repo as input, performs the migration, and opens a PR on that target repository.

@antgrutta Thanks for the review. This makes a lot of sense and aligns with where I was heading.

On hooks: Agreed, removing from this PR. The hooks work (tested on volcano-coffee) but deploying them per-repo is a separate concern. Will track separately.

On the architecture: I like the orchestrator → N workers pattern. To confirm my understanding:

submit-repos.yml (deterministic, standard Actions)
├─ Scan repos, identify eligible ones
└─ For each: dispatch-workflow → migration-worker.md

migration-worker.md (agentic workflow, one per repo)
├─ Input: target_repo, migration_type
├─ Checks out target repo (cross-repo checkout)
├─ Reads agent prompt + knowledge base (already local in .github-private)
├─ AI performs the migration
└─ Opens PR on target repo (create-pull-request with target-repo)

Since the worker runs in .github-private, the knowledge/ directory is already available locally. No MCP token needed. Does this mean we can drop the copilot environment + COPILOT_MCP_GITHUB_PERSONAL_ACCESS_TOKEN setup from settings.yml entirely? Or is it still needed for manual mode?

I'll rework the PR to this architecture.

[AlexDeMichieli]

Architectural Considerations for the Orchestrator → N Workers Pattern

Based on the proposed architecture — evolved through discussion #14, PR review feedback, and E2E testing on volcano-coffee — moving toward a dispatch-workflow → N agentic workers architecture. Capturing considerations. Most are future-scale concerns (current batch size is 5), but the observability and MCP questions need answers now.

Observability: Single pane of glass with isolated workers

Workers are intentionally isolated. A failure in repo A shouldn't affect repo B. But the orchestrator needs to know the overall status. Since dispatch-workflow is fire-and-forget, workers can self-report to a shared tracking issue:

Orchestrator (submit-repos.yml):
  1. Scan repos (deterministic)
  2. Create parent issue: "[Migration Batch] April 7 - 5 repos"
  3. For each eligible repo:
     dispatch-workflow → migration-worker.md
       inputs:
         target_repo: org/repo-1
         migration_type: Jenkins
         tracking_issue: 42        ← parent issue number

Worker (migration-worker.md):
  safe-outputs:
    create-pull-request:
      target-repo: ${{ inputs.target_repo }}
    add-comment:
      target: ${{ inputs.tracking_issue }}  ← reports back here

  On success: add-comment "✅ org/repo-1: PR #5 opened"
  On failure: add-comment "❌ org/repo-2: no CI source files"

The parent issue becomes the dashboard:

[Migration Batch] April 7 - 5 repos

✅ org/repo-1: PR #5 opened
✅ org/repo-3: PR #2 opened
❌ org/repo-2: no CI source files found
🔄 org/repo-4: in progress...
🔄 org/repo-5: in progress...

Workers don't know about each other. They just know where to post their status. Failed jobs also get gh-aw's built-in report-failure-as-issue for workflow-level crashes.

Future-scale considerations (not blocking at batch size 5)

These become relevant when scaling to 50+ repos per batch:

• Concurrency/cost: 100 workers = 100 concurrent AI sessions consuming PRUs. dispatch-workflow max is 50. Mitigate with staggered batches.
• Redundant clones: Each worker clones its target repo. Use sparse-checkout for large repos.
• PAT rate limiting: All workers share one PAT for create-pull-request. Built-in 10-second delays help; staggered dispatch adds spacing.

[antgrutta]

Scanning for repos based on custom properties is a very deterministic task, I have moved that workflow to a classic Actions workflow.