Replace missing evaluator tools in Daily GitHub Docs SEO Optimizer with a Copilot SDK driver by Copilot · Pull Request #47984 · github/gh-aw

Copilot · 2026-07-25T11:25:38Z

The Daily GitHub Docs SEO Optimizer depended on two unavailable tools (automation-request-generator and baseline-copilot-evaluator), so the workflow could not perform its baseline collection phase in this environment. This change moves that orchestration into a dedicated Copilot SDK TypeScript driver and keeps the reporting phase inside the workflow.

Workflow execution model
- switches the workflow to engine.driver: .github/drivers/daily_github_docs_seo_optimizer_driver.ts
- removes the inline agent blocks for request generation and baseline evaluation
- updates the prompt contract so the workflow consumes a driver-supplied structured dataset instead of invoking missing tools
Custom Copilot SDK driver
- adds a driver that:
  1. generates exactly 10 baseline automation requests
  2. runs 10 isolated Copilot evaluation sessions, one per request
  3. injects the collected dataset into the final reporting prompt
- preserves the original workflow intent: one reporting pass, one issue, same downstream analysis shape
Isolation and permission boundaries
- baseline collection runs with an explicit deny-by-default permission config so those sessions cannot read the repo, use shell, MCP, web, or write tools
- final reporting still uses the workflow’s normal permission model so safe-outputs issue creation continues to work
Driver-side validation
- validates that request generation returns exactly 10 distinct requests
- validates that each evaluator response is strict JSON with 3 ranked options and explicit documentation page metadata

engine:
  id: copilot
  copilot-sdk: true
  driver: .github/drivers/daily_github_docs_seo_optimizer_driver.ts
  bare: true

const dataset = await collectBaselineDataset(runSession);
const finalPrompt = buildFinalReportingPrompt(workflowPrompt, dataset);
return runSession(finalPrompt, permissionConfig);

Fixes [aw] Daily GitHub Docs SEO Optimizer is missing required tool #47982

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-07-25T11:46:58Z

@@ -0,0 +1,11 @@
+const { runDailyGitHubDocsSEOOptimizerDriver } = require("../../actions/setup/js/daily_github_docs_seo_optimizer_driver_helpers.cjs");


@copilot keep all the code for this driver here in a single file

Addressed in 58d28a1. The driver implementation now lives entirely in .github/drivers/daily_github_docs_seo_optimizer_driver.ts, and the separate helper module was removed.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-07-25T12:05:29Z

@@ -0,0 +1,100 @@
+import { describe, expect, it, vi } from "vitest";


@copilot delete this file

Addressed in d366a07.

pelikhan · 2026-07-25T12:47:37Z

@@ -0,0 +1,100 @@
+import { describe, expect, it, vi } from "vitest";
+


@copilot delete this file and any changes in actions/setup/js

Addressed in 2bdffa2.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Replaces unavailable SEO evaluation tools with a custom Copilot SDK driver while retaining safe-output reporting.

Changes:

Adds isolated request generation and baseline evaluation orchestration.
Updates the workflow to consume the driver-supplied dataset.
Regenerates the compiled workflow.

Show a summary per file

File	Description
`.github/workflows/daily-github-docs-seo-optimizer.md`	Configures the driver and updates reporting instructions.
`.github/workflows/daily-github-docs-seo-optimizer.lock.yml`	Reflects the recompiled TypeScript-driver execution.
`.github/drivers/daily_github_docs_seo_optimizer_driver.ts`	Implements collection, validation, isolation, and reporting orchestration.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 3/3 changed files
Comments generated: 3
Review effort level: Medium

+import { applyModelFallback } from "../../actions/setup/js/model_fallback.cjs";
+import { getErrorMessage } from "../../actions/setup/js/error_helpers.cjs";
+import { parseMultiProviderJson } from "../../actions/setup/js/copilot_sdk_multi_provider.cjs";
+import { parsePermissionConfigFromServerArgs } from "../../actions/setup/js/copilot_sdk_permissions.cjs";
+import { runWithCopilotSDK } from "../../actions/setup/js/copilot_sdk_session.cjs";


+  return runSession(finalPrompt, permissionConfig);
+}
+
+const currentFile = fileURLToPath(import.meta.url);


+    if (typeof page.title !== "string" || typeof page.url !== "string" || typeof page.used_for !== "string") {
+      throw new Error(`documentation_pages entries for request ${JSON.stringify(expectedRequest)} must include title, url, and used_for strings`);
+    }


github-actions · 2026-07-27T09:16:09Z

🎉 This pull request is included in a new release.

Release: v0.83.4

Initial plan

e475359

Copilot AI assigned Copilot and pelikhan Jul 25, 2026

Copilot started work on behalf of pelikhan July 25, 2026 11:26 View session

Copilot AI linked an issue Jul 25, 2026 that may be closed by this pull request

[aw] Daily GitHub Docs SEO Optimizer is missing required tool #47982

Closed

fix: add copilot sdk driver for docs seo optimizer

7a701ea

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix missing tools in Daily GitHub Docs SEO Optimizer~~ Replace missing evaluator tools in Daily GitHub Docs SEO Optimizer with a Copilot SDK driver Jul 25, 2026

Copilot finished work on behalf of pelikhan July 25, 2026 11:44

Copilot AI requested a review from pelikhan July 25, 2026 11:44

pelikhan reviewed Jul 25, 2026

View reviewed changes

Copilot started work on behalf of pelikhan July 25, 2026 11:47 View session

Refactor daily docs SEO driver into one file

58d28a1

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan July 25, 2026 11:59

Copilot AI requested a review from pelikhan July 25, 2026 11:59

pelikhan reviewed Jul 25, 2026

View reviewed changes

Copilot started work on behalf of pelikhan July 25, 2026 12:06 View session

Copilot finished work on behalf of pelikhan July 25, 2026 12:17

Copilot AI requested a review from pelikhan July 25, 2026 12:17

pelikhan reviewed Jul 25, 2026

View reviewed changes

Copilot started work on behalf of pelikhan July 25, 2026 12:48 View session

Remove actions setup SEO driver test file

2bdffa2

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan July 25, 2026 13:00

Copilot AI requested a review from pelikhan July 25, 2026 13:00

pelikhan marked this pull request as ready for review July 25, 2026 13:24

Copilot AI review requested due to automatic review settings July 25, 2026 13:24

pelikhan merged commit d8e3d68 into main Jul 25, 2026

pelikhan deleted the copilot/aw-fix-missing-tools-in-workflow branch July 25, 2026 13:24

Copilot started reviewing on behalf of pelikhan July 25, 2026 13:25 View session

Copilot AI reviewed Jul 25, 2026

View reviewed changes

github-actions Bot mentioned this pull request Jul 25, 2026

Smoke Test: MAI-Code-1-Flash - 30162378433 #48022

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Replace missing evaluator tools in Daily GitHub Docs SEO Optimizer with a Copilot SDK driver - #47984

Replace missing evaluator tools in Daily GitHub Docs SEO Optimizer with a Copilot SDK driver#47984
pelikhan merged 4 commits into
mainfrom
copilot/aw-fix-missing-tools-in-workflow

Copilot AI commented Jul 25, 2026 •

edited

Loading

Uh oh!

pelikhan Jul 25, 2026

Uh oh!

Copilot AI Jul 25, 2026

Uh oh!

pelikhan Jul 25, 2026

Uh oh!

Copilot AI Jul 25, 2026

Uh oh!

pelikhan Jul 25, 2026

Uh oh!

Copilot AI Jul 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,11 @@
		const { runDailyGitHubDocsSEOOptimizerDriver } = require("../../actions/setup/js/daily_github_docs_seo_optimizer_driver_helpers.cjs");

		@@ -0,0 +1,100 @@
		import { describe, expect, it, vi } from "vitest";

Uh oh!

Conversation

Copilot AI commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pelikhan Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

pelikhan Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

pelikhan Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

github-actions Bot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jul 25, 2026 •

edited

Loading