Skip to content

Validate documented commands against the Breeze and prek registries#70186

Merged
potiuk merged 1 commit into
apache:mainfrom
RoyLee1224:feat/validate-documented-commands
Jul 21, 2026
Merged

Validate documented commands against the Breeze and prek registries#70186
potiuk merged 1 commit into
apache:mainfrom
RoyLee1224:feat/validate-documented-commands

Conversation

@RoyLee1224

Copy link
Copy Markdown
Contributor

cc @potiuk @bugraoz93 you've got the most context on these changes, mind a quick sanity check before this merges? Want to make sure I'm reading them right.

Why

AGENTS.md used to document breeze selective-checks, which was never a real command (it's breeze ci selective-check), until #69861 fixed it by hand. contributing-docs/ is all hand(agent)-written, so this kind of thing rots quietly.

These are the docs agents read to run commands and increasingly edit themselves, so validating them against the real registry makes agent-assisted doc changes more reliable, wrong commands get caught at commit time instead of propagating.

What

A prek hook pulls every `breeze …` and `prek run …` out of AGENTS.md and contributing-docs/ and checks them against the real registries:

  • breeze — the actual click command tree, introspected at runtime (subcommands and --options)
  • prek — hook ids across the .pre-commit-config.yaml files

It catches both a command that's simply wrong in the docs, and one that got renamed without the docs following — the hook re-runs when commands/** changes, so a rename that touches no docs still fails. Passthrough commands,
<placeholder> args and --help are skipped so it doesn't flag things that are fine, and errors include a "did you mean" suggestion. It only runs when docs, breeze commands or prek configs change, not on every PR.

CleanShot 2026-07-21 at 20 04 08@2x

Evidence

Running it over the current tree turned up 10 stale/mistyped commands in contributing-docs/ — 7 distinct ones, wrong for 6 to 20 months. One (#43979) deleted the integration-tests subcommand and in the same commit added a doc
line pointing at it. All fixed here.

file wrong should be from
03_contributors_quick_start.rst breeze setup config --terminal_multiplexer --terminal-multiplexer #60844
testing/task_sdk_integration_tests.rst breeze prod image build breeze prod-image build #58231
testing/integration_tests.rst breeze providers-integration-tests --help breeze testing providers-integration-tests #48620
testing/integration_tests.rst breeze core-integration-tests --help breeze testing core-integration-tests #48620
testing/unit_tests.rst breeze testing integration-tests breeze testing core-integration-tests / providers-integration-tests #43979
27_cli_implementation_guide.rst breeze testing airflow-ctl-integration-test breeze testing airflow-ctl-integration-tests #66566
testing/airflow_ctl_tests.rst breeze testing airflow-ctl-integration-test breeze testing airflow-ctl-integration-tests #59387

Was generative AI tooling used to co-author this PR?
  • Yes (Claude Fable 5)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, deterministic check. Agents miss thing sometimes so indeed deterministic harness around it is a good idea.

@potiuk potiuk removed the backport-to-v3-3-test Backport to v3-3-test label Jul 21, 2026
@potiuk
potiuk merged commit 834fed9 into apache:main Jul 21, 2026
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants