Add evals to 24 more agentic workflows

.github/workflows/aw-failure-investigator.md

@@ -318,6 +318,11 @@ steps:
       PY
 features:
   gh-aw-detection: true
+evals:
+  - id: failures_investigated
+    question: Did the agent investigate agentic workflow failures from the last 6 hours and produce findings?
+  - id: issues_created_or_closed
+    question: Were fix sub-issues created for unresolved failures, or were resolved tracking issues closed?
 ---
 
 # [aw] Failure Investigator (6h)

.github/workflows/cli-consistency-checker.md

@@ -76,6 +76,11 @@ features:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: cli_inspected
+    question: Did the agent inspect the gh-aw CLI commands and analyze their output for inconsistencies, typos, or documentation gaps?
+  - id: issue_created_or_noop
+    question: Was an issue created with specific CLI inconsistencies found, or was noop used when no issues were detected?
 ---
 
 # CLI Consistency Checker

.github/workflows/code-scanning-fixer.md

@@ -47,6 +47,11 @@ features:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: alerts_analyzed
+    question: Did the agent analyze code scanning alerts and identify at least one fixable alert, or correctly skip when no fixable alerts were found?
+  - id: pr_created_or_noop
+    question: Was a pull request created with a remediation for a code scanning alert, or was noop used when no fixable alerts existed?
 ---
 
 # Code Scanning Alert Fixer Agent

.github/workflows/commit-changes-analyzer.md

@@ -43,6 +43,11 @@ imports:
   - shared/otlp.md
 features:
   gh-aw-detection: true
+evals:
+  - id: changes_analyzed
+    question: Did the agent analyze all repository changes since the specified commit and produce a comprehensive report?
+  - id: discussion_created
+    question: Was a discussion or report created summarizing the changes since the specified commit?
 ---
 
 # Commit Changes Analyzer

.github/workflows/daily-compiler-quality.md

@@ -98,6 +98,11 @@ features:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: files_analyzed
+    question: Did the agent analyze compiler code files for quality standards such as readability and maintainability?
+  - id: discussion_created_or_noop
+    question: Was a discussion or report created with quality findings, or was noop used when all analyzed files met the quality standards?
 ---
 
 {{#runtime-import? .github/shared-instructions.md}}

.github/workflows/daily-doc-updater.md

@@ -75,6 +75,11 @@ tools:
 tracker-id: daily-doc-updater
 features:
   gh-aw-detection: true
+evals:
+  - id: docs_reviewed
+    question: Did the agent review documentation and identify whether updates are needed based on recent code changes?
+  - id: pr_created_or_noop
+    question: Was a documentation pull request created with updates, or was noop used when no documentation changes were required?
 ---
 {{#runtime-import? .github/shared-instructions.md}}
 

.github/workflows/daily-function-namer.md

@@ -50,6 +50,11 @@ timeout-minutes: 30
 strict: true
 features:
   gh-aw-detection: true
+evals:
+  - id: package_analyzed
+    question: Did the agent analyze at least one Go package for function naming quality and discoverability?
+  - id: issue_created_or_noop
+    question: Was an issue created with function rename suggestions, or was noop used when no naming improvements were identified?
 ---
 
 # Daily Go Function Namer

.github/workflows/daily-malicious-code-scan.md

@@ -36,6 +36,11 @@ features:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: scan_completed
+    question: Did the agent complete a scan of recent code changes and report on whether malicious patterns were found?
+  - id: alert_or_noop
+    question: Was a code scanning alert created when threats were found, or does the agent output confirm the scan found no suspicious patterns?
 ---
 
 {{#runtime-import? .github/shared-instructions.md}}

.github/workflows/daily-semgrep-scan.md

@@ -52,6 +52,11 @@ experiments:
     analysis_type: proportion_test
     tags: [security, output-quality, semgrep]
     issue: 32795
+evals:
+  - id: scan_completed
+    question: Did the agent complete a Semgrep security scan and report on the findings?
+  - id: alert_created_or_noop
+    question: Was a code scanning alert created for real security findings, or does the agent output confirm no vulnerabilities were found?
 
 ---
 

.github/workflows/daily-syntax-error-quality.md

@@ -58,6 +58,11 @@ steps:
 imports:
   - shared/reporting.md
   - shared/otlp.md
+evals:
+  - id: errors_tested
+    question: Did the agent introduce syntax errors into candidate workflows and evaluate the quality of the resulting compiler error messages?
+  - id: issue_created_or_noop
+    question: Was a quality report issue created with improvement suggestions, or was noop used when all error messages already met quality standards?
 ---
 
 {{#runtime-import? .github/shared-instructions.md}}

.github/workflows/daily-yamllint-fixer.md

@@ -103,6 +103,11 @@ imports:
 sandbox:
   agent:
     sudo: false
+evals:
+  - id: lint_issues_identified
+    question: Does the agent output confirm that yamllint issues were identified and analyzed in the generated lock files?
+  - id: pr_created_or_noop
+    question: Was a pull request created with Go generator fixes for yamllint issues, or was noop used when no issues required changes?
 ---
 
 # Daily yamllint Fixer