Skip to content

Fix documented Breeze selective checks command#69861

Merged
jason810496 merged 1 commit into
apache:mainfrom
viiccwen:fix-selective-check-command-docs
Jul 17, 2026
Merged

Fix documented Breeze selective checks command#69861
jason810496 merged 1 commit into
apache:mainfrom
viiccwen:fix-selective-check-command-docs

Conversation

@viiccwen

@viiccwen viiccwen commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

closes #69858

Correct the contributor documentation to use breeze ci selective-check, which is the command registered by Breeze and executed by the GitHub Actions workflows.

This updates the source AGENT-SKILL command block, regenerates the Commands section in AGENTS.md, fixes the local reproduction example in the selective checks troubleshooting guide, and adjusts the generator test to verify the valid command.


Was generative AI tooling used to co-author this PR?
  • Yes - GPT-5.6 Sol

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@viiccwen

viiccwen commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Sorry I don't have Claude 😭
Does anyone can help me to update last-eval-hash.txt?

@Andrushika

Copy link
Copy Markdown
Contributor

Hi! I have regenerated the hash for you. You can simply paste it to dev/skill-evals/last-eval-hash.txt, then CI should pass.
Hash: cdf346752bd43c958355d98e63453bc4a00dd82727dde4243b1c01a2e2afb5fe

Contributor instructions currently point to a nonexistent Breeze command, preventing developers and coding agents from reproducing selective CI decisions locally.

Signed-off-by: viiccwen <vicwen@apache.org>
@viiccwen
viiccwen force-pushed the fix-selective-check-command-docs branch from 8877930 to ff4cbd3 Compare July 16, 2026 12:53
@viiccwen

Copy link
Copy Markdown
Contributor Author

Hi! I have regenerated the hash for you. You can simply paste it to dev/skill-evals/last-eval-hash.txt, then CI should pass. Hash: cdf346752bd43c958355d98e63453bc4a00dd82727dde4243b1c01a2e2afb5fe

Thx for your kindness. 🫡

@nailo2c

nailo2c commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Thanks, really helpful 😄

@RoyLee1224

Copy link
Copy Markdown
Contributor

Thanks for the fix, and sorry the eval-hash gate (from #69308) added friction here. I'm experimenting with additional agent runtimes for the eval (Codex and OpenCode SDKs)and will propose that in a follow-up PR, so the gate won't assume a Claude subscription. 🙌

@jason810496

Copy link
Copy Markdown
Member

Thanks for the fix, and sorry the eval-hash gate (from #69308) added friction here. I'm experimenting with additional agent runtimes for the eval (Codex and OpenCode SDKs)and will propose that in a follow-up PR, so the gate won't assume a Claude subscription. 🙌

Nice!

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the catch.

Additionally, this is exactly what #64012 would like to prevent from happening -- we should introduce an automated mechanism to prevent further drift instead of relying on manual catch.

@jason810496
jason810496 merged commit d9dcc11 into apache:main Jul 17, 2026
65 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test Commit Link

You can attempt to backport this manually by running:

cherry_picker d9dcc11 v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@RoyLee1224

RoyLee1224 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

I was curious what a wrong documented command like this actually costs an agent in tokens, so I ran this fix through the skill-eval harness (#69308) as a before/after A/B — same task ("determine which CI test types run for HEAD"), same tools, only AGENTS.md differs, n=3 per arm:

CleanShot 2026-07-17 at 10 44 03@2x

Every "before" run trusted the doc, failed, and had to rediscover breeze ci selective-check via --help — roughly 2× tokens per affected task, re-paid on every fresh session. And that's the best-case branch: when the agent doesn't recover, the cost is way more than 2×.

@RoyLee1224

Copy link
Copy Markdown
Contributor

@jason810496 Agreed! #68204 only keeps docs and AGENTS.md in sync, the bug was in the docs source, so the sync faithfully copied it over. Nothing checks that the commands are actually real.

Working on that now: validate documented commands against the real Breeze CLI, both when docs are generated and when a command gets renamed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect Breeze selective checks command in contributor documentation

6 participants