Skip to content

[docs] Self-healing documentation fixes from issue analysis - 2026-07-15 #45578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 1 commit into from
Jul 15, 2026
+27 −0
Merged

[docs] Self-healing documentation fixes from issue analysis - 2026-07-15 #45578

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/src/content/docs/reference/artifacts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ GitHub Agentic Workflows upload several artifacts during workflow execution. Thi
| `prompt` | — | Single-file | Generated prompt (`prompt.txt`) |
| `experiment` | `constants.ExperimentArtifactName` | Multi-file | A/B experiment state (`state.json`) uploaded by the activation job when experiments are declared in the frontmatter |
| `usage` | `constants.UsageArtifactName` | Multi-file | Compact conclusion-job artifact with workflow-run metadata and token-usage files used by lightweight reporting and forecasting paths |
| `evals` | `constants.EvalsArtifactName` | Single-file | BinEval evaluation results (`evals.jsonl`) uploaded by the evals job when `evals` are declared in the workflow frontmatter |
| `safe-outputs-items` | `constants.SafeOutputItemsArtifactName` | Single-file | Safe output items manifest |
| `code-scanning-sarif` | `constants.SarifArtifactName` | Single-file | SARIF file for code scanning results |

Expand All @@ -38,6 +39,7 @@ The `gh aw logs` and `gh aw audit` commands support `--artifacts` to download on
| `detection` | `detection` | Threat detection output |
| `experiment` | `experiment` | A/B experiment state (only present when experiments are declared) |
| `usage` | `usage` | Compact conclusion-job artifact for lightweight reporting and forecasting |
| `evals` | `evals` | BinEval evaluation results (only present when `evals` are declared) |
| `github-api` | `activation`, `agent` | GitHub API rate limit logs |

```bash
Expand Down Expand Up @@ -206,6 +208,31 @@ gh aw logs <run-id> --artifacts usage
gh run download <run-id> -n usage
```

## `evals`

The `evals` artifact is uploaded by the **evals job** only when the workflow frontmatter declares one or more `evals` entries. It is not present on runs without evals.

### Contents

- `evals.jsonl` — Per-question BinEval evaluation results (YES/NO records) produced by running the declared evaluation questions against the agent output

### Accessing evals data

```bash
# Download only the evals artifact
gh aw logs <run-id> --artifacts evals

# Or with gh run download
gh run download <run-id> -n evals
```

The `gh aw audit` command exposes an `--evals` flag that skips runs without evals results and automatically downloads the evals artifact when `--artifacts` is narrowed:

```bash
# Audit only runs that contain evals results
gh aw audit <run-id> --evals

@github-actions github-actions Bot Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The example gh aw audit <run-id> --evals implies --evals requires a specific run ID, but the flag is primarily a multi-run filter (skip runs without evals results). With a single run-id it still works, but the usage pattern shown here is the less common case.

💡 Suggested revision

Consider showing the multi-run (list) form first, which is the primary use case:

# Audit all runs, skipping those without evals results
gh aw audit --evals

# Or narrow to a specific run
gh aw audit <run-id> --evals

Also worth noting: gh aw logs supports the same --evals flag (gh aw logs --evals) — adding a cross-reference or example here would make the docs consistent.

@copilot please address this.

```

## Naming Compatibility

Artifact names changed between upload-artifact v4 and v5. The `gh aw logs` and `gh aw audit` commands handle both naming schemes transparently:
Expand Down
Loading