Skip to content

Add step shell script validator: gh CLI without GH_TOKEN heuristic#44661

Merged
pelikhan merged 11 commits into
mainfrom
copilot/update-go-compiler-validator
Jul 10, 2026
Merged

Add step shell script validator: gh CLI without GH_TOKEN heuristic#44661
pelikhan merged 11 commits into
mainfrom
copilot/update-go-compiler-validator

Conversation

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Steps using the gh CLI without GH_TOKEN in env: fail at runtime with opaque auth errors. This adds a compile-time heuristic to catch it early.

New validator: step_shell_validator.go

  • Checks pre-steps, steps, pre-agent-steps, post-steps for run: scripts invoking gh
  • Detection pattern: gh as a command token at line-start or after &&/||/;/|
  • Strict mode → error; non-strict → warning
  • Known limitation: $(gh ...) substitutions are not detected

Example violation caught:

steps:
  - name: List issues
    run: gh issue list  # ← no GH_TOKEN → compile error in strict mode

Fix suggested in the error message:

env:
  GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Wiring

validateStepShellScripts added to the validation chain in compiler_orchestrator_engine.go, alongside validateEnvSecrets and validateStepsSecrets.

Incidental fix

craft.md "Install gh-aw extension" step was genuinely missing GH_TOKEN — caught and fixed by the new validator.


Generated by 👨‍🍳 PR Sous Chef · 12.5 AIC · ⌖ 6.14 AIC · ⊞ 7.1K ·
Comment /souschef to run again


Generated by 👨‍🍳 PR Sous Chef · 4.34 AIC · ⌖ 5.93 AIC · ⊞ 7.1K ·
Comment /souschef to run again


Generated by 👨‍🍳 PR Sous Chef · 11.3 AIC · ⌖ 6.23 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Copilot AI and others added 2 commits July 10, 2026 05:45
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Add step shell script validator: gh CLI missing GH_TOKEN heuristic Add step shell script validator: gh CLI without GH_TOKEN heuristic Jul 10, 2026
Copilot AI requested a review from pelikhan July 10, 2026 05:51
@pelikhan pelikhan marked this pull request as ready for review July 10, 2026 06:39
Copilot AI review requested due to automatic review settings July 10, 2026 06:39
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot run pr-finisher skill

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new compile-time heuristic validator to detect run: scripts invoking the gh CLI without GH_TOKEN set in env: (error in strict mode, warning otherwise), wires it into the compiler’s validation chain, and updates the craft workflow to satisfy the new rule.

Changes:

  • Introduce validateStepShellScripts / step_shell_validator.go to detect gh usage in step run: scripts and require GH_TOKEN.
  • Add unit tests covering strict vs non-strict behavior and step section coverage.
  • Update .github/workflows/craft.md (and regenerated lock) to include GH_TOKEN for the gh extension install step.
Show a summary per file
File Description
pkg/workflow/step_shell_validator.go New heuristic validator for gh usage without GH_TOKEN in step env:.
pkg/workflow/step_shell_validator_test.go Unit tests for strict/non-strict behavior and section scanning.
pkg/workflow/compiler_orchestrator_engine.go Hooks the new validator into the existing validation chain.
.github/workflows/craft.md Adds GH_TOKEN to a step that runs gh.
.github/workflows/craft.lock.yml Regenerated compiled workflow reflecting the craft.md change.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/5 changed files
  • Comments generated: 3
  • Review effort level: Low

Comment thread pkg/workflow/step_shell_validator.go
Comment on lines +80 to +85
var violations []string
for _, step := range steps {
if v := checkStepGHToken(step); v != "" {
violations = append(violations, v)
}
}
Comment thread pkg/workflow/step_shell_validator.go Outdated
Copilot AI and others added 2 commits July 10, 2026 07:03
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot run pr-finisher skill

Addressed the finisher pass in bedc162: fixed the validator review feedback, added the missing regression coverage, and reran local validation (make fmt, make lint, make test-unit, make test, make recompile, make agent-report-progress). CI on this head is still unverified until a maintainer re-triggers it.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Unresolved review threads:

Generated by 👨‍🍳 PR Sous Chef · 12.5 AIC · ⌖ 6.14 AIC · ⊞ 7.1K ·
Comment /souschef to run again

…piler-validator

# Conflicts:
#	.github/workflows/craft.lock.yml

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Field Value
Category feature
Risk 🔴 High
Score 72 / 100
Action ⚡ fast_track

Score breakdown: impact 38 + urgency 22 + quality 12

Rationale: Adds compile-time validator catching gh CLI usage without GH_TOKEN, preventing runtime auth failures. Touches core compiler orchestrator and new validator. High value — catches a real class of silent failures. Recommend expedited human review.

Generated by 🔧 PR Triage Agent · 44.7 AIC · ⌖ 5.13 AIC · ⊞ 5.4K ·

Copilot AI requested a review from gh-aw-bot July 10, 2026 07:36
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot validate container shas

Copilot AI and others added 3 commits July 10, 2026 07:51
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot validate container shas

Addressed in 04b8256: I validated the 0.27.29 firewall/build-tools image digests against docker buildx imagetools inspect, refreshed the incorrect pins in actions-lock.json, regenerated the affected lockfiles/synced pin data, updated the compile regression expectations, and reran make recompile, the firewall pin tests, the container-pin validation script, and make agent-report-progress.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

pr-sous-chef
@copilot please run the pr-finisher skill, ensure the branch is up to date, address any remaining review feedback, and rerun checks.

Run: https://github.com/github/gh-aw/actions/runs/29079112957

Generated by 👨‍🍳 PR Sous Chef · 8.44 AIC · ⌖ 5.58 AIC · ⊞ 7.1K ·
Comment /souschef to run again

…update-go-compiler-validator

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

pr-sous-chef
@copilot please run the pr-finisher skill, ensure the branch is up to date, address any remaining review feedback, and rerun checks.
...

Addressed on 4ab51f9: I unshallowed the repo, merged origin/main into the branch, reran local validation (make fmt, make lint, make test-unit, make test, make recompile, make agent-report-progress), and confirmed the remaining review thread is already outdated/satisfied on the current head. The linked PR Sous Chef run succeeded; CI on this updated head is still unverified until a maintainer re-triggers it.

@github-actions

Copy link
Copy Markdown
Contributor

Hey @app/copilot-swe-agent 👋 — great addition! The new compile-time heuristic for catching gh CLI usage without GH_TOKEN is exactly the kind of early-error-detection that saves contributors from confusing runtime auth failures.

The PR is well-structured: the validator is cleanly wired into the orchestrator alongside validateEnvSecrets and validateStepsSecrets, the test file covers the new logic, golden files are updated consistently, and the incidental craft.md fix is a nice real-world validation that the heuristic works. This looks ready for review! 🚀

Generated by ✅ Contribution Check · 115.9 AIC · ⌖ 13.8 AIC · ⊞ 6.2K ·

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address any remaining review feedback, refresh the branch if needed, and rerun checks.

Generated by 👨‍🍳 PR Sous Chef · 4.34 AIC · ⌖ 5.93 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address any remaining review feedback, refresh the branch if needed, and rerun checks.

...

Checked the branch on fbd3e9f: there’s no remaining actionable review feedback, the outstanding Copilot review thread is outdated/satisfied by the current validator implementation, and the branch already contains the current PR base commit. I reran local validation (make fmt, make lint, make test-unit, make test, make recompile), all of which passed. The latest PR workflows on this head are still action_required/stale after the agent push, so CI still needs a maintainer re-trigger before merge.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address unresolved review comments, refresh the branch if needed, and rerun checks once ready.
Unresolved review threads:

Generated by 👨‍🍳 PR Sous Chef · 11.3 AIC · ⌖ 6.23 AIC · ⊞ 7.1K ·
Comment /souschef to run again

@pelikhan pelikhan merged commit a1efc55 into main Jul 10, 2026
1 of 2 checks passed
@pelikhan pelikhan deleted the copilot/update-go-compiler-validator branch July 10, 2026 14:30
Copilot stopped work on behalf of gh-aw-bot due to an error July 10, 2026 14:30
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.82.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants