Skip to content

fix(shipping-pr): wait for missing required checks - #39

Merged
ether-moon merged 1 commit into
mainfrom
ether_karrot/fix-shipping-pr-required-checks
Aug 5, 2026
Merged

fix(shipping-pr): wait for missing required checks#39
ether-moon merged 1 commit into
mainfrom
ether_karrot/fix-shipping-pr-required-checks

Conversation

@ether-moon

@ether-moon ether-moon commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • collect effective required status-check contexts from active rulesets and legacy branch protection
  • treat configured required contexts that have not registered on the current HEAD as pending
  • keep optional reviewer telemetry and non-gating merge states out of shipping verdicts and blocker fingerprints
  • document the required-only review contract and add regressions for missing review contexts

Testing

  • bash plugins/skill-set/tests/test-pr-runner.sh
  • bash plugins/skill-set/tests/test-shipping-pr-happy-path-eval.sh
  • bash plugins/skill-set/tests/test-skill-contracts.sh
  • bash plugins/skill-set/tests/test-references.sh
  • bash plugins/skill-set/tests/test-portable-runner-layout.sh
  • bash plugins/skill-set/tests/test-packaging.sh
  • bash plugins/skill-set/tests/test-host-neutral-skill-runtime.sh
  • python3 /Users/ether/.codex/skills/.system/skill-creator/scripts/quick_validate.py plugins/skill-set/skills/shipping-pr
  • git diff --check

Summary by CodeRabbit

  • New Features

    • Shipping now detects required checks from repository rules and inherited rulesets.
    • Missing required checks remain pending and are clearly identified at deadlines.
    • Optional observed checks can be included with --required-only false.
    • Pull request base-branch changes are detected to prevent invalid snapshots.
  • Behavior Changes

    • --review-timeout is deprecated and no longer affects completion.
    • Reviewer telemetry is informational unless tied to a required check or actionable thread.
    • Mergeability and branch-state changes are tracked more accurately.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The shipping PR workflow now discovers effective required checks from repository rules, tracks the PR base branch, represents missing checks as pending, and excludes reviewer telemetry from completion gates and fingerprints. Tests cover optional checks, delayed registration, blocked merge state, malformed responses, and fingerprint stability.

Changes

Shipping PR gating

Layer / File(s) Summary
Gating contract and polling semantics
plugins/skill-set/commands/pr/ship.md, plugins/skill-set/skills/shipping-pr/SKILL.md, plugins/skill-set/skills/shipping-pr/reference/polling.md
Review timeout is compatibility-only. Effective required checks gate completion. Observed optional checks are included when requested. Reviewer telemetry remains reporting-only.
Base binding and required-check collection
plugins/skill-set/skills/shipping-pr/scripts/skill-set-pr, plugins/skill-set/evals/shipping-pr/state-machine-happy-path/fixtures/mock-gh, plugins/skill-set/tests/fixtures/mock-gh-pr, plugins/skill-set/tests/test-pr-runner.sh
Initialization and snapshots record baseRefName, discover required contexts from rules and protection APIs, and represent missing contexts as pending checks.
Snapshot evaluation and regression coverage
plugins/skill-set/skills/shipping-pr/scripts/skill-set-pr, plugins/skill-set/tests/fixtures/mock-gh-pr, plugins/skill-set/tests/test-pr-runner.sh
Snapshot binding, merge state, status evaluation, persistence, and fingerprints now include required-check and base-branch data. Tests cover delayed checks, blocked merge state, malformed responses, and telemetry-only changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ShippingPR as skill-set-pr
  participant GitHubAPI
  participant SnapshotState
  ShippingPR->>GitHubAPI: Query PR baseRefName and merge fields
  ShippingPR->>GitHubAPI: Query branch rules and legacy protection
  GitHubAPI-->>ShippingPR: Return configured required contexts
  ShippingPR->>GitHubAPI: Query check runs
  GitHubAPI-->>ShippingPR: Return observed check results
  ShippingPR->>SnapshotState: Store required, observed, and missing contexts
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: shipping-pr now waits for missing required checks.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ether_karrot/fix-shipping-pr-required-checks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/skill-set/skills/shipping-pr/scripts/skill-set-pr (1)

1218-1244: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Reset deadlines when the base branch changes during a snapshot.

Line 1218 resets deadlines for a changed HEAD binding, but it does not compare $start_base_branch and $end_base_branch. Line 1243 discards the snapshot for that base-branch change, but the state retains the check deadline and registration grace from the previous base branch. A retargeted PR can time out before checks required by its new base branch register.

Include the start/end base-branch comparison in the deadline-reset condition.

Proposed fix
-    $start_head_branch != "$end_head_branch" || $start_host != "$end_host" ]]; then
+    $start_head_branch != "$end_head_branch" || \
+    $start_base_branch != "$end_base_branch" || $start_host != "$end_host" ]]; then
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/skill-set/skills/shipping-pr/scripts/skill-set-pr` around lines 1218
- 1244, Update the deadline-reset condition in the snapshot state logic to also
detect changes between start_base_branch and end_base_branch. When that
base-branch comparison changes, reset head_changed-related deadlines,
checks_observed, and resolver_attempt alongside the existing HEAD binding
changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@plugins/skill-set/skills/shipping-pr/scripts/skill-set-pr`:
- Around line 1218-1244: Update the deadline-reset condition in the snapshot
state logic to also detect changes between start_base_branch and
end_base_branch. When that base-branch comparison changes, reset
head_changed-related deadlines, checks_observed, and resolver_attempt alongside
the existing HEAD binding changes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1347bdd5-fa5d-47bd-9b55-da6126f2db7c

📥 Commits

Reviewing files that changed from the base of the PR and between b11847e and 88c0f27.

📒 Files selected for processing (7)
  • plugins/skill-set/commands/pr/ship.md
  • plugins/skill-set/evals/shipping-pr/state-machine-happy-path/fixtures/mock-gh
  • plugins/skill-set/skills/shipping-pr/SKILL.md
  • plugins/skill-set/skills/shipping-pr/reference/polling.md
  • plugins/skill-set/skills/shipping-pr/scripts/skill-set-pr
  • plugins/skill-set/tests/fixtures/mock-gh-pr
  • plugins/skill-set/tests/test-pr-runner.sh

@ether-moon
ether-moon merged commit 6d968c5 into main Aug 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant