Support per-run cost limits in common.ai LLM and Agent operators - #71403
Open
Lee-W wants to merge 2 commits into
Open
Support per-run cost limits in common.ai LLM and Agent operators#71403Lee-W wants to merge 2 commits into
Lee-W wants to merge 2 commits into
Conversation
Lee-W
force-pushed
the
common-ai-cost-limit
branch
5 times, most recently
from
August 12, 2026 13:47
eb460d9 to
acb43e9
Compare
Lee-W
marked this pull request as ready for review
August 13, 2026 02:17
Lee-W
commented
Aug 13, 2026
| ``apache-airflow-providers-common-compat`` ``>=1.15.0`` | ||
| ``apache-airflow-providers-standard`` ``>=1.12.1`` | ||
| ``pydantic-ai-slim`` ``>=2.0.0`` | ||
| ``pydantic-ai-slim`` ``>=2.23.0`` |
Member
Author
There was a problem hiding this comment.
I guess we could just upgrade this version since we're still in 0.x.x and the cost feature is something that would be widely used
Dag authors had no way to cap what a single LLM or Agent task is allowed to spend, and no way to see what one cost after the fact. pydantic-ai made both first-class in 2.23.0 via RunUsage.cost and UsageLimits.cost_limit, but this provider's floor was >=2.0.0, so neither existed for users here. Raising the floor is what unlocks the enforcement path — the operators already forward usage_limits untouched to run_sync, so cost_limit takes effect with no interception logic of our own. The max_cost parameter exists for the one thing usage_limits structurally cannot do: a UsageLimits object cannot be templated, so a budget that follows the environment had to be expressible as a scalar. The docs deliberately spell out where the cap stops being a guarantee, since each of these reads as a broken promise to anyone who assumed otherwise: the request that crosses the limit still completes and still bills, self-hosted or unpriced models report no cost at all and the limit silently does nothing, and a durable replay re-counts the cached spend against a fresh attempt's budget.
Because max_cost is templated, its value is often not in the Dag author's control -- an unset Variable or a typo renders to a string the operator then has to make sense of. An "inf" that passed validation left a cap that could never be exceeded, which is worse than no cap at all: it reads as protection that isn't there. "nan" escaped as a bare decimal.InvalidOperation from the negativity comparison, contradicting the documented ValueError contract. The classification call LLMRetryPolicy makes on every task failure is not an operator run, so no cost cap has ever applied to it. That is pre-existing and deliberate, but someone who sets max_cost reasonably expects it to bound everything the task spends, so the exemption is now stated in the docs. The provider requires pydantic-ai-slim 2.23.0 for the cost API, so the extras now say so too rather than leaving a floor the resolver silently overrides.
Lee-W
force-pushed
the
common-ai-cost-limit
branch
from
August 13, 2026 08:45
acb43e9 to
04872a2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Dag authors had no way to cap what a single LLM or Agent task is allowed to spend, and no way to see what one cost after the fact. pydantic-ai made both first-class in 2.23.0 via
RunUsage.costandUsageLimits.cost_limit.What
Raising the floor is what unlocks the enforcement path — the operators already forward
usage_limitsuntouched torun_sync, socost_limittakes effect with no interception logic of our own. Themax_costparameter exists for the one thingusage_limitsstructurally cannot do: aUsageLimitsobject cannot be templated, so a budget that follows the environment had to be expressible as a scalar.The docs deliberately spell out where the cap stops being a guarantee, since each of these reads as a broken promise to anyone who assumed otherwise: the request that crosses the limit still completes and still bills, self-hosted or unpriced models report no cost at all and the limit silently does nothing, and a durable replay re-counts the cached spend against a fresh attempt's budget.
Was generative AI tooling used to co-author this PR?
Generated-by: [Claude] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.