Skip to content

[plan] Add end-to-end regression test: validation and push must operate on the same object #48999

Description

@github-actions

Objective

Add an end-to-end/integration-style regression test for create_pull_request that asserts the validated .patch artifact and the actually-pushed .bundle/commit contain the identical file set, across both the normal path and the merge-commit rewrite path.

Context

The root cause shared by both bugs reported in issue #48934 is: validation and push do not operate on the same object, so a passing validation does not reliably predict what lands on the remote. This issue tracks adding a regression test suite that would have caught both bugs, and will guard against future variants (e.g., any new code path that rebuilds/rewrites the commit before push).

Suggested Approach

  1. Add a test harness (in actions/setup/js test suite or equivalent Go integration tests in pkg/workflow) that:
    • Simulates a base branch with drift (extra commits added after the agent's checkout, including files matching excluded-files).
    • Simulates a shallow checkout forcing the merge-commit rewrite path.
    • Runs the full create_pull_request flow (patch validation + bundle/push synthesis).
    • Extracts the file list of the validated .patch and the file list of the final pushed commit (via git diff --cached --name-only on the synthesized commit, or by inspecting the .bundle).
    • Asserts the two file lists are identical (same set of paths).
  2. Include a variant for the non-rewrite path (full-depth checkout, no merge commit) to ensure the same invariant holds there too, catching the excluded-files/bundle bug independently of the rewrite bug.
  3. Wire this test into the existing test suite so it runs in CI (make test-unit / relevant Go or JS test target) and will fail if a future change reintroduces the discrepancy.
  4. This issue depends on the fixes in the other sub-issues from this plan being complete (or can be developed test-first / marked expected-fail until those land).

Files to Modify

  • New test file(s) under actions/setup/js/ and/or pkg/workflow/ test directories
  • Possibly a small test-only helper to extract file lists from a bundle or synthesized commit

Acceptance Criteria

  • New regression test(s) assert patch file set == pushed/bundle file set, for both rewrite and non-rewrite paths
  • Test(s) fail on the pre-fix code (validate by checking against a pre-fix commit, or documenting expected failure) and pass after the companion fixes land
  • Test(s) integrated into the standard test run (make test-unit or equivalent)

Generated by 📋 Plan Command · aut00 · 18.7 AIC · ⌖ 5.78 AIC · ⊞ 6.8K ·
Comment /plan to run again

  • expires on Jul 31, 2026, 2:14 PM UTC-08:00

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions