Add evals to 20 agentic workflows

.github/workflows/ai-moderator.md

@@ -63,6 +63,13 @@ features:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: action-taken
+    question: Did the agent apply at least one label (spam, ai-generated, link-spam, or ai-inspected) or call noop?
+  - id: rationale-provided
+    question: Does the agent output include a rationale explaining why the label(s) were applied or why noop was called?
+  - id: no-unsupported-action
+    question: Does the agent output show that only allowed safe-output actions (add-labels, hide-comment, noop) were used?
 ---
 
 # AI Moderator

.github/workflows/cloclo.md

@@ -57,6 +57,11 @@ safe-outputs:
 timeout-minutes: 20
 features:
   gh-aw-detection: true
+evals:
+  - id: output-produced
+    question: Did the agent create a pull request or post a comment?
+  - id: summary-comment-added
+    question: Does the agent output confirm that a summary comment was added to the original conversation thread?
 ---
 
 # /cloclo

.github/workflows/constraint-solving-potd.md

@@ -36,6 +36,13 @@ features:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: discussion-created
+    question: Did the agent create a constraint solving problem of the day discussion?
+  - id: problem-stated
+    question: Does the agent output include a clear problem statement with constraints?
+  - id: solution-provided
+    question: Does the agent output include a solution or answer to the problem?
 ---
 
 # Constraint Solving — Problem of the Day

.github/workflows/copilot-cli-deep-research.md

@@ -70,6 +70,13 @@ tools:
     max-file-size: 204800
 features:
   gh-aw-detection: true
+evals:
+  - id: issue-created
+    question: Did the agent create a research issue with Copilot CLI findings?
+  - id: optimization-opportunities
+    question: Does the agent output identify specific missed optimization opportunities for Copilot CLI usage in this repository?
+  - id: evidence-based
+    question: Does the agent output cite specific repository evidence (workflow files, usage patterns) to support its findings?
 ---
 
 # Copilot CLI Deep Research Agent

.github/workflows/daily-assign-issue-to-user.md

@@ -33,6 +33,13 @@ imports:
   - shared/otlp.md
 features:
   gh-aw-detection: true
+evals:
+  - id: assignment-made
+    question: Did the agent assign an issue to a user?
+  - id: contributor-selected
+    question: Does the agent output show that a relevant contributor was selected from recent merged PRs?
+  - id: comment-posted
+    question: Did the agent post a comment explaining the assignment decision?
 ---
 
 {{#runtime-import? .github/shared-instructions.md}}

.github/workflows/daily-skill-optimizer.md

@@ -155,6 +155,13 @@ imports:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: issue-created
+    question: Did the agent create an issue with skill improvement suggestions?
+  - id: three-improvements
+    question: Does the agent output include at least three specific skill improvement recommendations?
+  - id: improvements-actionable
+    question: Does the agent output describe each improvement in enough detail to be actionable rather than only describing the general area?
 ---
 
 # Daily Skill Optimizer Improvements

.github/workflows/delight.md

@@ -87,6 +87,13 @@ tools:
     - default
     - discussions
 tracker-id: delight-daily
+evals:
+  - id: output-produced
+    question: Did the agent create an issue or post a message with UX findings?
+  - id: ux-issues-identified
+    question: Does the agent output identify at least one specific UX improvement opportunity?
+  - id: actionable-findings
+    question: Does the agent output include actionable recommendations rather than only generic observations?
 ---
 
 {{#runtime-import? .github/shared-instructions.md}}

.github/workflows/deployment-incident-monitor.md

@@ -34,6 +34,13 @@ timeout-minutes: 10
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: incident-or-noop
+    question: Did the agent either create an incident issue or call noop?
+  - id: root-cause-included
+    question: Does the agent output include a root cause analysis or a reason for the noop decision?
+  - id: dedup-respected
+    question: Does the agent output show that it checked for an existing open incident issue before creating a new one?
 ---
 
 ### Deployment Incident Monitor

.github/workflows/design-decision-gate.md

@@ -144,7 +144,13 @@ steps:
           file_count: $file_count,
           diff_available: $diff_available
         }' > /tmp/gh-aw/agent/adr-prefetch-summary.json
-
+evals:
+  - id: adr-check-performed
+    question: Does the agent output confirm that it checked for existing ADRs before deciding on an action?
+  - id: action-taken
+    question: Did the agent add a PR comment, push a draft ADR, or call noop?
+  - id: decision-justified
+    question: Does the agent output explain why an ADR is required or why no ADR gate was triggered for this PR?
 ---
 
 # Design Decision Gate 🏗️

.github/workflows/dev-hawk.md

@@ -46,6 +46,11 @@ imports:
       toolsets: [pull_requests, actions, repos]
   - shared/reporting.md
   - shared/otlp.md
+evals:
+  - id: comment-posted
+    question: Did the agent post a comment with PR or CI analysis?
+  - id: insights-provided
+    question: Does the agent output include specific insights about pull request status or CI results?
 ---
 
 # Dev Hawk - Development Workflow Monitor

.github/workflows/discussion-task-miner.md

@@ -65,6 +65,13 @@ tracker-id: discussion-task-miner
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: output-produced
+    question: Did the agent create at least one code quality issue or add a comment?
+  - id: tasks-extracted
+    question: Does the agent output show that actionable tasks were identified from the analyzed discussions?
+  - id: labels-applied
+    question: Does the agent output confirm that the created issues include the expected labels (code-quality, automation, task-mining)?
 ---
 
 # Discussion Task Miner - Code Quality Improvement Agent

.github/workflows/docs-noob-tester.md

@@ -103,6 +103,13 @@ pre-agent-steps:
       echo "Server URL: http://localhost:4321/gh-aw/"
 features:
   gh-aw-detection: true
+evals:
+  - id: findings-reported
+    question: Did the agent report documentation testing findings (via asset upload or report)?
+  - id: confusing-steps-identified
+    question: Does the agent output identify at least one confusing, broken, or unclear step in the documentation?
+  - id: user-perspective-maintained
+    question: Does the agent output reflect the perspective of a new user rather than an expert reviewer?
 ---
 
 # Documentation Noob Testing

.github/workflows/go-logger.md

@@ -124,6 +124,13 @@ tools:
     mode: gh-proxy
     toolsets:
     - default
+evals:
+  - id: pr-created
+    question: Did the agent create a pull request with Go logging improvements?
+  - id: logging-changes-made
+    question: Does the agent output confirm that logging statements were added or modified in Go source files?
+  - id: validation-run
+    question: Does the agent output show that it ran validation commands to verify the logging changes compile correctly?
 ---
 # Go Logger Enhancement
 

.github/workflows/gpclean.md

@@ -88,6 +88,13 @@ steps:
         PACKAGE_COUNT=$(jq '.sbom.packages | length' /tmp/gh-aw/agent/sbom.json 2>/dev/null || echo "unknown")
         echo "📊 SBOM contains ${PACKAGE_COUNT} packages"
       fi
+evals:
+  - id: issue-or-noop
+    question: Did the agent either create a GPL dependency issue or call noop?
+  - id: go-mod-analyzed
+    question: Does the agent output confirm that go.mod was analyzed for GPL-licensed transitive dependencies?
+  - id: decision-explained
+    question: Does the agent output include an explanation of why a GPL issue was created or why noop was called?
 
 ---