Revert "Suspend Apache Beam Provider due to grpcio limitation (#61926)"#66952
Revert "Suspend Apache Beam Provider due to grpcio limitation (#61926)"#66952olegkachur-e wants to merge 1 commit into
Conversation
3fbd131 to
01bc19c
Compare
|
@olegkachur-e A few things need addressing before review — see our Pull Request quality criteria.
No rush. Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. |
1647831 to
794fc1a
Compare
| # For Python>=3.11, the apache-beam package depends on envoy-data-plane. | ||
| # Currently, there is only one compatible version(1.0.3) of envoy-data-plane available for these | ||
| # Python versions, and it explicitly requires a prerelease version of betterproto. | ||
| "betterproto==2.0.0b6; python_version >= '3.11'", |
There was a problem hiding this comment.
I am not a fan of this.
this is a beta version from 2023. This is not a case where in a few days/weeks things will change and a stable version would be released, is it?
I am not in the details of this so I will leave @potiuk the call here but from my point of view we should not deliever stable release with depenedency on beta packages.
Who gurentees that tommorow the author won't remove them from PyPi with a thought that there won't be any production impact on anyone?
There was a problem hiding this comment.
Not a big fan either.
But I think thee newer version of envoy-data-plane has been questioned by others for example here: apache/beam#37854
And I see that this is what latest beam has:
envoy-data-plane<2,>=1.0.3; python_version >= "3.11"
And the 2.1.0 version of envoy-data-plane uses already betterproto2 - which is a maintained replacement of betterproto.
So maybe we can ask apache-beam to upgrade to the envoy-data-plane>=2 ?
There was a problem hiding this comment.
I am not a big fan of this idea either.
The reason why I decided to add it was in that this beta version exists from 2023 and I am not sure that it will be release in the future. One more point for me to added it was that I noticed that Apache Beam SDK use this beta version in there generated requirements for containers. And I had idea that if they already had hardcoded it in apache-beam requirements, from which our provider already depends, maybe it makes sense temporary hardcoded it in provider as well for fixing constrains problem and unsuspend provider.
There was a problem hiding this comment.
We can't tell beam what to so from thier side but Airflow should not do this.
|
I commented in the issue. |
|
Quickest fix: git fetch upstream main && git rebase upstream/main
rm uv.lock && uv lock
git add uv.lock && git rebase --continue
git push --force-with-leaseAutomated nudge — ignore if you're not ready to rebase. This comment is updated in place on future |
…#61926)" This reverts commit 917abea. - Remove hacks regarding the beam provider suspension, as they are not relevant anymore. ISSUE: apache#66551
794fc1a to
8a4a569
Compare
This reverts commit 917abea.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.Important
🛠️ Maintainer triage note for @olegkachur-e · by
@potiuk· 2026-07-02 17:46 UTCSome review feedback from
@eladkalis waiting on you:@eladkalneed a reply or a fix.The ball is in your court — you've been assigned to this PR. Reply or push a fix in each thread, then mark them resolved.
Automated triage — may be imperfect; a maintainer takes the next look.