diff --git a/contributing-docs/27_cli_implementation_guide.rst b/contributing-docs/27_cli_implementation_guide.rst new file mode 100644 index 0000000000000..50b96ec59b273 --- /dev/null +++ b/contributing-docs/27_cli_implementation_guide.rst @@ -0,0 +1,144 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +CLI Implementation Guide +================================== + +This document describes the direction for implementing new CLI functionality in Apache Airflow. + +.. contents:: Table of Contents + :depth: 2 + :local: + + +Overview +-------- + +Airflow ships two CLIs: + +- **airflow** (``airflow-core``) — bundled with the core distribution. Hosts both legacy + remote commands (being rewired internally) and admin/local commands that have no Public + API equivalent. +- **airflowctl** (``airflow-ctl``) — a standalone CLI distributed separately that talks to a + running Airflow instance exclusively through the Public (Core) API. + +Following AIP-94 (tracked via `GitHub Projects #570 and #571 +`_), CLI work follows two rules: + +1. **New commands** that are achievable via the Public API are added to ``airflowctl`` + **only**. Adding the same command to the ``airflow`` CLI as well is discouraged — it + duplicates maintenance surface without user benefit. +2. **Existing** ``airflow`` **CLI remote commands** stay in place (so users keep running + ``airflow dags list``, ``airflow pools get``, …) but are rewired internally to call the + Public API via the ``airflowctl`` HTTP client instead of accessing the metadata + database directly. + +Both rules enforce RBAC, remove direct database exposure for remote operations, and eliminate +duplicate code paths. This builds on AIP-81, which introduced the distinction between *local* +and *remote* commands. + +Decision Table +--------------- + +.. list-table:: + :header-rows: 1 + :widths: 35 35 30 + + * - Scenario + - Where it goes + - Notes + * - New command, achievable via Public API + - ``airflowctl`` only + - Do not add to the ``airflow`` CLI unless strongly needed in core + * - New command, not achievable via Public API + - ``airflow`` CLI (admin/local) + - Admin/local commands only — see below + * - Existing ``airflow`` CLI command, achievable via Public API + - ``airflow`` CLI → delegates to the ``airflowctl`` HTTP client + - Rewire; no direct DB access, no SQLAlchemy/``session`` usage + * - Existing ``airflow`` CLI command, not achievable via Public API + - ``airflow`` CLI, unchanged + - Stays as a pure ``airflow-core`` implementation + +"Not achievable via the Public API" means the operation has no API representation and is +inherently admin/local in nature — database shell, schema migrations, process management, +or deployment configuration that requires direct infrastructure access. + +Adding a New Command (Public API achievable) +--------------------------------------------- + +Add the command to ``airflowctl`` **only**. Do not also add it to the ``airflow`` CLI unless +there is a strong reason it must live in core (e.g., it is tightly coupled to a local process +or a deployment concern with no API representation). + +Source location: ``airflow-ctl/src/airflowctl/ctl/commands/`` + +HTTP client and operations: ``airflow-ctl/src/airflowctl/api/`` (``client.py``, ``operations.py``). + +1. Add the command under the appropriate group module in + ``airflow-ctl/src/airflowctl/ctl/commands/``. +2. Call the Public API through the ``airflowctl`` HTTP client (``airflowctl.api.client``) and + the operations layer in ``airflowctl.api.operations``. Do not import ``airflow-core`` + models or touch the metadata database. +3. If the required API endpoint does not exist yet, add it first + (see `Adding API Endpoints <16_adding_api_endpoints.rst>`__). +4. Add tests under ``airflow-ctl/tests/``. +5. Run integration tests with: + + .. code-block:: bash + + breeze testing airflow-ctl-integration-test + +Rewiring an Existing ``airflow`` CLI Command +--------------------------------------------- + +Use this when an existing ``airflow`` CLI remote command still talks to the database +directly and needs to go through the Public API instead. The user-facing command name and +arguments stay the same. + +Source location: ``airflow-core/src/airflow/cli/`` + +1. Replace direct database access with calls through the ``airflowctl`` HTTP client + (``airflowctl.api.client`` / ``airflowctl.api.operations``). Remove SQLAlchemy model + imports and ``session``\-based helpers from the command. +2. If the required API endpoint does not exist yet, add it first + (see `Adding API Endpoints <16_adding_api_endpoints.rst>`__). +3. Update tests under ``airflow-core/tests/cli/`` to mock or exercise the HTTP client + instead of the database. + +Adding an Admin/Local Command (no Public API equivalent) +--------------------------------------------------------- + +Use this only when the operation cannot reasonably be exposed through the Public API — +typically database shell, schema migrations, process management, or deployment-time +configuration. + +Source location: ``airflow-core/src/airflow/cli/`` + +1. Add the command to an appropriate admin group (e.g., ``db``, ``config``). +2. Add ``(admin only)`` to the ``help`` string so users know the command requires direct + infrastructure access. +3. Add tests under ``airflow-core/tests/cli/``. + +References +----------- + +- `AIP-94 Confluence page `_ +- `Adding API Endpoints <16_adding_api_endpoints.rst>`__ +- `Airflow Ctl Tests `__ +- `GitHub Project #570 `_ +- `GitHub Project #571 `_ diff --git a/contributing-docs/README.rst b/contributing-docs/README.rst index 6313acf1a96b6..505f006442937 100644 --- a/contributing-docs/README.rst +++ b/contributing-docs/README.rst @@ -136,3 +136,7 @@ Maintainer Tools ``pr-triage`` and ``pr-stats`` skills that maintainers invoke from Claude Code to sweep the open Pull Request queue (run deterministic quality checks, propose actions under explicit per-batch confirmation) and to surface backlog statistics by ``area:*`` label. + +* `CLI Implementation Guide <27_cli_implementation_guide.rst>`__ describes where to implement new + CLI features following AIP-94: remote commands go to ``airflowctl``, admin/deployment commands + stay in the ``airflow`` CLI.