Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR-26587: Pre-Agent Cache-Memory Working-Tree Sanitization

**Date**: 2026-04-16
**Status**: Draft
**Status**: Accepted
**Deciders**: pelikhan, Copilot

---
Expand Down Expand Up @@ -77,6 +77,7 @@ The existing integrity and threat-detection pipeline could be trusted to catch m
2. Extensions in `GH_AW_ALLOWED_EXTENSIONS` **MUST** include the leading dot (e.g. `.json`, `.md`); files with no extension are treated as if their extension is an empty string.
3. When `GH_AW_ALLOWED_EXTENSIONS` is absent or empty, implementations **MUST NOT** remove any files based on extension (all extensions are allowed).
4. Implementations **SHOULD** log each removed file path and its extension, and **SHOULD** log a final count of removed files.
5. If `GH_AW_ALLOWED_EXTENSIONS` contains any non-empty entry without a leading dot, implementations **SHOULD** emit a warning identifying the malformed entry before continuing with any fallback or filtering behavior.

### Go-to-Shell Configuration Contract

Expand All @@ -90,4 +91,4 @@ An implementation is considered conformant with this ADR if it satisfies all **M

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/24514620900) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
*Accepted on 2026-07-19 after validating the Go-side env-var contract and the shell-script sanitization tests.*
12 changes: 6 additions & 6 deletions docs/adr/26666-add-pre-agent-steps-support.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR-26666: Introduce `pre-agent-steps` as a Distinct Workflow Extension Point

**Date**: 2026-04-16
**Status**: Draft
**Status**: Accepted
**Deciders**: pelikhan, Copilot

---
Expand All @@ -10,11 +10,11 @@

### Context

The gh-aw workflow compiler translates markdown workflow files into GitHub Actions YAML. It already supports two custom step extension points: `pre-steps` (injected before the checkout step, very early in the agent job) and `post-steps` (injected after AI engine execution). A gap exists for steps that must run after all framework initialization — including checkout, credential setup, and environment preparation — but *immediately before* the AI engine starts. Use cases include final context preparation scripts, last-moment environment variable injection, and validations that depend on checked-out code being available.
The gh-aw workflow compiler translates markdown workflow files into GitHub Actions YAML. It already supports two custom step extension points: `pre-steps` (injected before the checkout step, very early in the agent job) and `post-steps` (injected after AI engine execution). A gap exists for steps that must run after checkout, checkout cleanup, and PR/base restoration, but before MCP startup and later engine-owned runtime setup. Use cases include final context preparation scripts, last-moment environment variable injection, and validations that depend on checked-out code being available.

### Decision

We will add a new `pre-agent-steps` frontmatter field that is injected into the compiled GitHub Actions YAML immediately before the engine execution step, after all setup steps (including the CLI proxy start step). Like `pre-steps`, `pre-agent-steps` participates in the import merge system: imported `pre-agent-steps` are prepended before the main workflow's `pre-agent-steps`, preserving a deterministic, layered execution order. The existing `validateStepsSecrets` infrastructure is extended to cover the new field, applying identical secret expression restrictions.
We will add a new `pre-agent-steps` frontmatter field that is injected into the compiled GitHub Actions YAML after checkout cleanup and trusted base-restore steps, but before MCP startup and the later engine execution sequence. Like `pre-steps`, `pre-agent-steps` participates in the import merge system: imported `pre-agent-steps` are prepended before the main workflow's `pre-agent-steps`, preserving a deterministic, layered execution order. The existing `validateStepsSecrets` infrastructure is extended to cover the new field, applying identical secret expression restrictions.

### Alternatives Considered

Expand Down Expand Up @@ -61,7 +61,7 @@ Users could approximate "run before engine" behaviour by placing steps in `post-
### Compilation and Placement

1. Implementations **MUST** inject all resolved `pre-agent-steps` into the compiled GitHub Actions YAML immediately before the engine execution step.
2. `pre-agent-steps` **MUST** be placed after all framework setup steps (including the CLI proxy start step) and **MUST NOT** be placed before any framework-owned initialization steps.
2. `pre-agent-steps` **MUST** be placed after checkout cleanup and any trusted PR/base restoration steps, and **MUST** be placed before MCP startup and the later engine execution sequence.
3. If no `pre-agent-steps` are defined (neither in the main workflow nor in any imports), the compiler **MUST NOT** emit any placeholder or empty steps block.

### Import Merge Ordering
Expand All @@ -81,8 +81,8 @@ Users could approximate "run before engine" behaviour by placing steps in `post-

### Conformance

An implementation is considered conformant with this ADR if it satisfies all **MUST** and **MUST NOT** requirements above. Specifically: `pre-agent-steps` are injected immediately before the engine execution step (not before setup steps), imports are merged in prepend order, secret validation is applied, and action pinning is applied. Failure to meet any **MUST** or **MUST NOT** requirement constitutes non-conformance.
An implementation is considered conformant with this ADR if it satisfies all **MUST** and **MUST NOT** requirements above. Specifically: `pre-agent-steps` are injected after checkout cleanup / trusted restore steps and before MCP startup and engine execution, imports are merged in prepend order, secret validation is applied, and action pinning is applied. Failure to meet any **MUST** or **MUST NOT** requirement constitutes non-conformance.

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/24519930482) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
*Accepted on 2026-07-19 after validating compiler ordering and adding a conformance test for workflows that define only `pre-agent-steps`.*
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR-26688: Configurable Expiration for Action Failure Issues via aw.json

**Date**: 2026-04-16
**Status**: Draft
**Status**: Accepted
**Deciders**: pelikhan, Copilot

---
Expand Down Expand Up @@ -43,7 +43,7 @@ Setting `GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS` as a GitHub Actions repositor

#### Neutral
- The `DefaultActionFailureIssueExpiresHours` constant (`24 * 7 = 168`) is now an exported symbol, making it accessible to other packages that may need the canonical default.
- Documentation in `docs/src/content/docs/guides/ephemerals.md` is updated to describe the new field alongside the existing `runs_on` maintenance option.
- Documentation in `docs/src/content/docs/reference/ephemerals.md` is updated to describe the new field alongside the existing `runs_on` maintenance option.

---

Expand All @@ -60,7 +60,7 @@ Setting `GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS` as a GitHub Actions repositor
### Compilation and Environment Variable Propagation

1. The workflow compiler **MUST** read `action_failure_issue_expires` from `aw.json` (via `LoadRepoConfig`) during `buildConclusionJob` and inject the resolved value as the `GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS` environment variable into the generated conclusion-job YAML.
2. If `aw.json` cannot be loaded or parsed, the compiler **MUST** log a warning and fall back to the default value of `168`; it **MUST NOT** fail the build.
2. If `.github/workflows/aw.json` exists but cannot be loaded, parsed, or validated, the compiler **MUST** emit a warning that identifies the file path and the fallback value (`168`) before continuing; it **MUST NOT** fail the build.
3. The injected environment variable value **MUST** be a positive integer string representing hours.

### Runtime Handling
Expand All @@ -75,4 +75,4 @@ An implementation is considered conformant with this ADR if it satisfies all **M

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/24522565038) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
*Accepted on 2026-07-19 after validating the `aw.json`-driven expiration path and documenting the fallback warning behavior.*
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR-26693: Reaction Target Scoping and Union Activation Permission Derivation

**Date**: 2026-04-16
**Status**: Draft
**Status**: Accepted
**Deciders**: pelikhan, Copilot

---
Expand Down Expand Up @@ -88,4 +88,4 @@ An implementation is considered conformant with this ADR if it satisfies all **M

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/24526383923) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
*Accepted on 2026-07-19 after validating target-scoped permission derivation and parser defaults for scalar and object reaction forms.*
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR-26827: Cap gh-aw Native Action Updates at Running CLI Version

**Date**: 2026-04-17
**Status**: Draft
**Status**: Accepted
**Deciders**: dsyme, Copilot

---
Expand Down Expand Up @@ -55,6 +55,7 @@ Introduce a checked-in file (e.g., `compatibility.json`) that explicitly maps CL

1. Implementations **MUST** classify an action as a "gh-aw native action" if and only if its base repository (resolved via `gitutil.ExtractBaseRepo`) is `github/gh-aw` or `github/gh-aw-actions`.
2. Implementations **MUST NOT** classify any other action as a gh-aw native action, including actions in forked or similarly named repositories.
3. Future native action repositories **MAY** be added only by an explicit allowlist expansion in a later release; until then, unknown repositories **MUST** remain non-native.

### Version Cap Enforcement

Expand All @@ -75,4 +76,4 @@ An implementation is considered conformant with this ADR if it satisfies all **M

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/24560253328) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
*Accepted on 2026-07-19 after validating the CLI-version cap behavior and documenting the explicit-allowlist extension model for future native action repositories.*
2 changes: 2 additions & 0 deletions docs/src/content/docs/reference/cache-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ tools:

If files with disallowed extensions are found, the workflow will report validation failures.

When a cache is restored for agent execution, gh-aw also strips execute bits from restored working-tree files and removes disallowed file types before the agent can read them. See [ADR-26587](https://github.com/github/gh-aw/blob/main/docs/adr/26587-pre-agent-cache-memory-working-tree-sanitization.md) for the pre-agent sanitization contract behind `allowed-extensions`.

## Multiple Configurations

```aw wrap
Expand Down
2 changes: 2 additions & 0 deletions docs/src/content/docs/reference/ephemerals.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,8 @@ Customize the runner:

`action_failure_issue_expires` sets expiration, in hours, for failure issues opened by the conclusion job, including grouped parent issues when `group-reports: true`. The default is `168` (7 days).

If `.github/workflows/aw.json` is present but cannot be loaded, parsed, or validated, compilation keeps the default `168`-hour expiration and emits a warning that identifies the config path and fallback value.

`disabled_jobs` lets you omit specific maintenance jobs from the generated workflow. Job IDs are case-insensitive, and `_` / `-` are treated equivalently.

Supported job IDs:
Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/docs/reference/frontmatter.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ imports:

See [Imports](/gh-aw/reference/imports/) for complete documentation on syntax, shared components, APM package dependencies, and composition patterns.

### Custom Steps and Jobs (`steps:`, `pre-agent-steps:`, `post-steps:`, `jobs:`)
### Custom Steps and Jobs (`pre-steps:`, `steps:`, `pre-agent-steps:`, `post-steps:`, `jobs:`)

Add deterministic steps before or after agentic execution, or define full custom GitHub Actions jobs that run before the agent. See [Custom Steps and Jobs](/gh-aw/reference/steps-jobs/) for complete documentation.

Expand Down
24 changes: 23 additions & 1 deletion docs/src/content/docs/reference/steps-jobs.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Custom Steps and Jobs
description: "Add deterministic pre-processing steps and custom GitHub Actions jobs to agentic workflows using steps:, pre-agent-steps:, post-steps:, and jobs:"
description: "Add deterministic pre-processing steps and custom GitHub Actions jobs to agentic workflows using pre-steps:, steps:, pre-agent-steps:, post-steps:, and jobs:"
sidebar:
order: 820
---
Expand All @@ -9,6 +9,28 @@ Custom steps and jobs let you mix deterministic computation with agentic executi

See [DeterministicOps](/gh-aw/patterns/deterministic-ops/) for patterns combining computation with AI reasoning.

## Choosing the Right Step Hook

Use these top-level step hooks in this order:

1. `pre-steps:` — runs before checkout and all later built-in agent-job setup; only compiler-injected OTLP masking steps may run earlier. Use this for short-lived token minting or anything that must happen before repository checkout.
2. `steps:` — runs after checkout and the normal runtime/bootstrap steps, but before the final pre-agent phase. Use this for deterministic preprocessing that needs the checked-out repository.
3. `pre-agent-steps:` — runs after framework-owned initialization such as checkout cleanup and base-branch restoration, and before MCP startup and engine execution. Use this for last-moment environment preparation immediately before the agent starts.
4. `post-steps:` — runs after the engine finishes. Use this for cleanup, summaries, uploads, or follow-up automation.

## Custom Pre-Checkout Steps (`pre-steps:`)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] Section order in this file contradicts the lifecycle ordering just stated in the new guide above.

The "Choosing the Right Step Hook" guide (lines 11–17) lists hooks in execution order: pre-stepsstepspre-agent-stepspost-steps. But the sections below start with ## Custom Steps (steps:) at line 21, followed by ## Custom Pre-Checkout Steps (pre-steps:) here — the reverse of the lifecycle order. A reader scanning the headings will see steps: documented before pre-steps:, which directly contradicts the numbered guide.

💡 Suggested fix

Move ## Custom Steps (steps:) after ## Custom Pre-Checkout Steps (pre-steps:) so the section order matches the lifecycle order presented in the new guide. That way the page reads top-to-bottom in execution order: pre-steps → steps → pre-agent-steps → post-steps.

@copilot please address this.


Add custom steps before checkout and the later pre-checkout agent-job setup. Compiler-injected OTLP masking steps may still run earlier.

```yaml wrap
pre-steps:
- name: Mint checkout token
id: checkout_app
uses: actions/create-github-app-token@v2
```

Use pre-steps when later checkout or setup must consume outputs from a step in the same job.

## Custom Steps (`steps:`)

Add custom steps before agentic execution. If unspecified, a default checkout step is added automatically.
Expand Down
4 changes: 4 additions & 0 deletions docs/src/content/docs/setup/creating-workflows.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,10 @@ After merging the pull request, you can run the workflow to see it in action. Ei
- trigger runs manually from the Actions tab in GitHub.com, or
- use [the `gh aw run` command](/gh-aw/setup/cli/#run) to trigger runs from your terminal.

## Choose the Right Step Lifecycle Hook

When you add deterministic setup or cleanup to a workflow, choose the smallest hook that matches the lifecycle point you need. See [Custom Steps and Jobs](/gh-aw/reference/steps-jobs/#choosing-the-right-step-hook) for the canonical ordering guide, including the distinction between pre-checkout `pre-steps:` and the later `pre-agent-steps:` phase.

## Initialize the Repository

Running `gh aw init` is **required** to enable the authoring experience in the GitHub code agent. This step configures your repository so that you can create and modify agentic workflows directly from [github.com](https://github.com) or the GitHub mobile app, using the Copilot coding agent.
Expand Down
34 changes: 31 additions & 3 deletions pkg/workflow/compiler_pre_agent_steps_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,22 @@ import (
"testing"

"github.com/github/gh-aw/pkg/testutil"
"gopkg.in/yaml.v3"
)

type compiledWorkflowJobs struct {
Jobs map[string]map[string]any `yaml:"jobs"`
}

func TestPreAgentStepsGeneration(t *testing.T) {
tmpDir := testutil.TempDir(t, "pre-agent-steps-test")

testContent := `---
on: push
permissions:
contents: read
checkout:
force-clean-git-credentials: true
pre-agent-steps:
- name: Finalize prompt context
run: echo "finalize"
Expand All @@ -39,28 +46,49 @@ Test pre-agent-steps.
}

lockFile := filepath.Join(tmpDir, "test-pre-agent-steps.lock.yml")
content, err := os.ReadFile(lockFile)
lockBytes, err := os.ReadFile(lockFile)
if err != nil {
t.Fatalf("Failed to read generated lock file: %v", err)
}
lockContent := string(content)
lockContent := string(lockBytes)

if !strings.Contains(lockContent, "- name: Finalize prompt context") {
t.Error("Expected pre-agent-step to be in generated workflow")
}

startMCPGatewayIndex := indexInNonCommentLines(lockContent, "- name: Start MCP Gateway")
cleanCheckoutCredentialsIndex := indexInNonCommentLines(lockContent, "- name: Clean git credentials after checkout")
preAgentStepIndex := indexInNonCommentLines(lockContent, "- name: Finalize prompt context")
aiStepIndex := indexInNonCommentLines(lockContent, "- name: Execute Claude Code CLI")
if startMCPGatewayIndex == -1 || preAgentStepIndex == -1 || aiStepIndex == -1 {
if startMCPGatewayIndex == -1 || cleanCheckoutCredentialsIndex == -1 || preAgentStepIndex == -1 || aiStepIndex == -1 {
t.Fatal("Could not find expected steps in generated workflow")
}
if cleanCheckoutCredentialsIndex >= preAgentStepIndex {
t.Errorf("Clean git credentials after checkout (%d) should appear before pre-agent-step (%d)", cleanCheckoutCredentialsIndex, preAgentStepIndex)
}
if preAgentStepIndex >= startMCPGatewayIndex {
t.Errorf("Pre-agent-step (%d) should appear before Start MCP Gateway (%d)", preAgentStepIndex, startMCPGatewayIndex)
}
if preAgentStepIndex >= aiStepIndex {
t.Errorf("Pre-agent-step (%d) should appear before AI execution step (%d)", preAgentStepIndex, aiStepIndex)
}
var compiled compiledWorkflowJobs
if err := yaml.Unmarshal(lockBytes, &compiled); err != nil {
t.Fatalf("Failed to parse generated lock file as YAML: %v", err)
}
for jobName, job := range compiled.Jobs {
rawSteps, hasSteps := job["steps"]
if !hasSteps {
continue
}
steps, ok := rawSteps.([]any)
if !ok {
t.Fatalf("Expected %s job steps to decode as a YAML sequence, got %T", jobName, rawSteps)
}
if len(steps) == 0 {
t.Fatalf("Expected %s job to omit empty steps blocks instead of emitting steps: []", jobName)
}
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] This assertion matches on literal YAML text fragments (pre-agent-steps: [] etc.) that may never actually appear in the generated output — making it a no-op rather than a real guard.

If the compiler never emits such markers, the test always passes regardless of whether empty step blocks are suppressed. A stronger check would assert on the YAML structure directly or use an existing helper that parses the lock file.

💡 Suggested approach

Instead of checking for the absence of a string, assert that there are no empty steps sequences in the parsed YAML, e.g. unmarshal the lock content and verify each top-level steps key is either absent or non-empty. Alternatively, document the exact serialised form the compiler could emit so the string guard has a known failure case.

@copilot please address this.

func TestPreAgentStepsImportsMergeOrder(t *testing.T) {
Expand Down
15 changes: 15 additions & 0 deletions pkg/workflow/compiler_repo_config.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package workflow

import (
"fmt"
"maps"
"os"
"strings"
)

Expand All @@ -17,6 +19,19 @@ func (c *Compiler) loadRepoConfig() (*RepoConfig, error) {
c.repoConfigLoaded = true
if c.repoConfigErr != nil {
repoConfigLog.Printf("loadRepoConfig: failed to load repo config: %v", c.repoConfigErr)
fmt.Fprintln(
os.Stderr,
formatCompilerMessage(
RepoConfigFileName,
"warning",
fmt.Sprintf(
"failed to load aw.json; compilation will continue with defaults, and action_failure_issue_expires will fall back to %d hours where applicable: %v",
DefaultActionFailureIssueExpiresHours,
c.repoConfigErr,
),
),
)
c.IncrementWarningCount()
} else {
repoConfigLog.Print("loadRepoConfig: repo config loaded successfully")
}
Expand Down
Loading