Skip to content

Fix metric emitting for triggerer and dag processor#61154

Merged
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:onikolas/fix_metrics_emitting
Jan 30, 2026
Merged

Fix metric emitting for triggerer and dag processor#61154
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:onikolas/fix_metrics_emitting

Conversation

@o-nikolas
Copy link
Copy Markdown
Contributor

After a recent refactoring (#53722) a call to an initializer for stats is required but was not done for the dag processor or triggerer. I noticed this when going to manually test some of my metrics related changes in breeze.

Tested manually in Breeze with and without these changes and metrics appear with these fixes.

I'm curious to hear from @xBis7 since he was the original author of the above changes.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
@xBis7
Copy link
Copy Markdown
Contributor

xBis7 commented Jan 28, 2026

Hi @o-nikolas, thanks for pinging me. The initialize method was the best approach we could come up with, me and @potiuk in order to decouple the code from core.

The result is that every new process has to call Stats.initialize during startup. I was able to identify only 3 places based on the existing tests but I was sure there were more. I see now that it was naive of me to not run further manual tests.

@o-nikolas
Copy link
Copy Markdown
Contributor Author

Hi @o-nikolas, thanks for pinging me. The initialize method was the best approach we could come up with, me and @potiuk in order to decouple the code from core.

The result is that every new process has to call Stats.initialize during startup. I was able to identify only 3 places based on the existing tests but I was sure there were more. I see now that it was naive of me to not run further manual tests.

Thanks for the context @xBis7 I figured that was the case. Do you mind giving these changes a review?

@o-nikolas o-nikolas requested a review from potiuk January 28, 2026 16:58
@xBis7
Copy link
Copy Markdown
Contributor

xBis7 commented Jan 28, 2026

@o-nikolas What are you using for testing? StatsD?

@o-nikolas
Copy link
Copy Markdown
Contributor Author

@o-nikolas What are you using for testing? StatsD?

Yup!

breeze start-airflow --backend postgres --integration statsd

Copy link
Copy Markdown
Contributor

@xBis7 xBis7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM! I tested it manually and I was able to reproduce the issue and verify that the changes in this patch, fix it. Thanks!

@o-nikolas o-nikolas merged commit fe0633d into apache:main Jan 30, 2026
71 checks passed
@o-nikolas o-nikolas deleted the onikolas/fix_metrics_emitting branch January 30, 2026 18:11
morelgeorge pushed a commit to morelgeorge/airflow that referenced this pull request Feb 1, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
shashbha14 pushed a commit to shashbha14/airflow that referenced this pull request Feb 2, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
jason810496 pushed a commit to abhijeets25012-tech/airflow that referenced this pull request Feb 3, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
jhgoebbert pushed a commit to jhgoebbert/airflow_Owen-CH-Leung that referenced this pull request Feb 8, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
After a recent refactoring (apache#53722) a call to an initializer for stats
is required but was not done for the dag processor or triggerer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants