Document when to use common.ai vs vendor-specific AI providers

providers/anthropic/docs/index.rst

@@ -19,6 +19,41 @@
 ``apache-airflow-providers-anthropic``
 ======================================
 
+When to use this provider
+--------------------------
+
+Use ``anthropic`` when a Dag needs Anthropic's native API surface — services Anthropic runs
+for you, which no vendor-neutral operator wraps:
+
+* ``AnthropicBatchOperator`` and ``AnthropicBatchSensor`` — submit a Claude
+  `Message Batches <https://docs.claude.com/en/docs/build-with-claude/batch-processing>`__
+  job for asynchronous bulk processing and wait for it to complete.
+* ``AnthropicAgentSessionOperator`` — start a Managed Agents session in which the agent loop
+  runs server-side on Anthropic's infrastructure; the Airflow task only kicks off the session
+  and waits for its outcome.
+
+Use :doc:`apache-airflow-providers-common-ai:index` instead when the AI step should be run by
+Airflow itself and stay vendor-neutral:
+
+* Generation, classification, or structured extraction with ``LLMOperator`` — it works with
+  Claude via a connection, and switching to another model provider later is a connection
+  change, not a Dag rewrite.
+* Agents whose loop runs **in the Airflow worker** with ``AgentOperator`` — Airflow-defined
+  toolsets (SQL, hooks, MCP servers), human-in-the-loop review, and durable step replay.
+
+Both ``AgentOperator`` and ``AnthropicAgentSessionOperator`` run agents with Claude models;
+the difference is where the loop executes and who controls the tools. ``AgentOperator``
+orchestrates the loop in the worker and calls tools you define in the Dag.
+``AnthropicAgentSessionOperator`` hands the whole session to Anthropic's managed service —
+the agent runs autonomously against a pre-created agent and environment, without
+Airflow-side tool execution.
+
+For example, running the Message Batches API directly:
+
+.. exampleinclude:: /../tests/system/anthropic/example_anthropic_batch.py
+    :language: python
+    :start-after: [START howto_operator_anthropic_batch]
+    :end-before: [END howto_operator_anthropic_batch]
 
 .. toctree::
     :hidden:

providers/common/ai/docs/index.rst

@@ -19,6 +19,46 @@
 ``apache-airflow-providers-common-ai``
 ##################################################
 
+When to use this provider
+--------------------------
+
+``common.ai`` is the vendor-neutral way to put LLM and agent steps in a Dag. It is built on
+`pydantic-ai <https://ai.pydantic.dev/>`__, so the model vendor (OpenAI, Anthropic, Google,
+Bedrock, …) is picked by the connection ``llm_conn_id`` points at — switching providers later
+is a connection change, not a Dag rewrite. The AI step is orchestrated by Airflow: the model
+calls, the agent loop, and any tools all run in the Airflow worker, where they get retries,
+logging, and observability like any other task.
+
+Use it when a Dag needs:
+
+* **Generation, classification, summarization, or structured extraction** —
+  :doc:`LLMOperator and @task.llm <operators/llm>`, with Pydantic-typed output pushed to XCom.
+* **Branching on a model's decision** — :doc:`LLMBranchOperator <operators/llm_branch>`.
+* **Agents with tools** — :doc:`AgentOperator <operators/agent>` runs a multi-turn agent loop
+  in the worker, calling Airflow-defined toolsets (SQL, hooks, MCP servers), with optional
+  human-in-the-loop review and durable step replay.
+* **Document pipelines** — loading, file analysis, embeddings, and retrieval for RAG
+  (see :doc:`operators/index`).
+
+Use a vendor's own provider instead when the Dag needs that vendor's **native API surface** —
+a service the vendor runs for you, which no vendor-neutral operator wraps:
+
+* :doc:`apache-airflow-providers-openai:index` — the Embeddings, Responses, and Batch APIs.
+* :doc:`apache-airflow-providers-anthropic:index` — the Claude Message Batches API, and
+  Managed Agents sessions where the agent loop runs on Anthropic's infrastructure rather
+  than in the Airflow worker.
+
+As a rule of thumb: if Airflow should *run* the AI step (and the model should stay
+swappable), use ``common.ai``; if the Dag *submits work to* a vendor-managed service and
+waits for the result, use that vendor's provider.
+
+For example, this ``LLMOperator`` call is unchanged whether ``llm_conn_id`` points at an
+OpenAI, Anthropic, or other pydantic-ai-supported connection:
+
+.. exampleinclude:: /../../ai/src/airflow/providers/common/ai/example_dags/example_llm.py
+    :language: python
+    :start-after: [START howto_operator_llm_basic]
+    :end-before: [END howto_operator_llm_basic]
 
 .. toctree::
     :hidden:

providers/openai/docs/index.rst

@@ -19,6 +19,41 @@
 ``apache-airflow-providers-openai``
 ======================================
 
+When to use this provider
+--------------------------
+
+Use ``openai`` when a Dag needs OpenAI's native API surface — thin wrappers over
+OpenAI-specific endpoints and options:
+
+* ``OpenAIEmbeddingOperator`` — call the Embeddings API directly, e.g. to feed a vector
+  store.
+* ``OpenAIResponseOperator`` — call the
+  `Responses API <https://platform.openai.com/docs/api-reference/responses>`__ with
+  OpenAI-specific parameters.
+* ``OpenAITriggerBatchOperator`` and ``OpenAIHook`` — submit a
+  `Batch API <https://platform.openai.com/docs/guides/batch>`__ job for asynchronous bulk
+  processing and wait for it to complete.
+
+Use :doc:`apache-airflow-providers-common-ai:index` instead when the AI step should be run by
+Airflow itself and stay vendor-neutral:
+
+* Generation, classification, or structured extraction with ``LLMOperator`` — it works with
+  OpenAI models via a connection, and switching to another model provider later is a
+  connection change, not a Dag rewrite.
+* Agents whose loop runs in the Airflow worker with ``AgentOperator`` — Airflow-defined
+  toolsets (SQL, hooks, MCP servers), human-in-the-loop review, and durable step replay.
+* Document-to-vector-store pipelines with its document loader, embedding, and retrieval
+  operators, which are not tied to OpenAI's embedding models.
+
+In short: pick ``openai`` to reach an OpenAI-only endpoint; pick ``common.ai`` to keep the
+Dag portable across model providers.
+
+For example, calling the Responses API directly:
+
+.. exampleinclude:: /../../openai/tests/system/openai/example_openai.py
+    :language: python
+    :start-after: [START howto_operator_openai_response]
+    :end-before: [END howto_operator_openai_response]
 
 .. toctree::
     :hidden: