Add a Retry Policies category to the provider registry - #70499
Conversation
boring-cyborg
Bot
added
area:dev-tools
area:providers
area:registry
backport-to-v3-3-test
6fbf8b8 to
98a9f9d
Compare
potiuk
left a comment
potiuk
left a comment
There was a problem hiding this comment.
Thanks — coherent end-to-end addition: schema, registry tooling, provider data, site data/styling, and a validation check all move together, which is what makes a new registry category actually usable rather than half-wired.
I verified the references resolve rather than assuming: airflow.sdk.definitions.retry_policy.RetryPolicy exists (the ABC at line 92), and providers/common/ai/policies/retry.py contains LLMRetryPolicy(RetryPolicy), matching the declared suffixes: ["RetryPolicy"]. So the registry entry describes something real.
Adding retry-policies to the duplicate-detection list alongside the other categories is easy to forget and you didn't — good.
One observation inline about the directory convention.
Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting
RetryPolicy is a Dag-author-facing pluggable component (task-sdk AIP-105) with a concrete implementation, common-ai's LLMRetryPolicy, but it had no presence in the provider registry -- no module type to classify it under and no provider.yaml section to declare it, so it was undiscoverable when browsing the registry.
The new registry module type shipped without a color token, so its badge fell through to the unstyled base rule and rendered as plain text next to the colored pills for every other type.
The provider.yaml validator had no knowledge of the new retry-policies section, so two mistakes passed CI silently: a typo in a python-modules path yielded an empty category instead of an error, and a provider could ship a RetryPolicy subclass without declaring it. Module-path completeness is deliberately not asserted for this section -- parse_module_data's glob assumes the directory is named after the resource type, which does not hold for retry policies.
The next person adding a registry category is likely to copy this check rather than the standard glob-based one, without realising that the layout divergence is what makes a bespoke check necessary here. Stating the assumption parse_module_data makes, and that retry policies depart from it deliberately, gives that reader the reason rather than just the observation.
98a9f9d to
eab70e8
Compare
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker fbac2f5 v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
4 participants
Why
RetryPolicy is a Dag-author-facing pluggable component (task-sdk AIP-105) with a concrete implementation, common-ai's LLMRetryPolicy, but it had no presence in the provider registry -- no module type to classify it under and no provider.yaml section to declare it, so it was undiscoverable when browsing the registry.
What
Make it searchable in the registry
Was generative AI tooling used to co-author this PR?
Generated-by: [Claude] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.