Add REST API metrics by henry3260 · Pull Request #64523 · apache/airflow

henry3260 · 2026-03-31T06:28:25Z

What

adds REST API instrumentation using Airflow's existing metrics abstraction.

Changes included:

Added basic API metrics emission in HttpAccessLogMiddleware.
Emit api.requests, api.request.duration, and api.request.errors for /api/v2 and /ui requests.
Added low-cardinality metric tags: api_surface, method, route, status_code, and status_family.

closes: #56034

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Important

🛠️ Maintainer triage note for @henry3260 · by @potiuk · 2026-06-22 06:31 UTC

Your review threads from @ashb look addressed — please confirm this PR is ready for maintainer review confirmation:

1 thread(s) show your engagement (post-review commits and/or replies).

The ball is in your court — you've been assigned to this PR. Reply yes / ready (and mark the threads resolved) and a maintainer will pick it up from the queue.

_{Automated triage — may be imperfect; a maintainer takes the next look.}

henry3260 · 2026-03-31T15:24:26Z

cc @pierrejeambrun

Copilot

Pull request overview

This PR adds basic REST API request instrumentation to Airflow’s FastAPI/Starlette stack by emitting standardized metrics via Airflow’s existing Stats abstraction, enabling QPS/latency/error monitoring for /api/v2 and /ui traffic.

Changes:

Added api.requests, api.request.duration, and api.request.errors metric definitions to the shared metrics template.
Implemented metric emission in HttpAccessLogMiddleware with bounded tags (api_surface, method, route, status_code, status_family) and guarded failure handling.
Added unit tests validating metric emission behavior for API/UI paths, health checks, non-API paths, failures, and unmatched routes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml	Registers new API request metrics in the shared metrics catalog.
airflow-core/src/airflow/api_fastapi/common/http_access_log.py	Emits API metrics from the access-log middleware after request completion.
airflow-core/tests/unit/api_fastapi/common/test_http_access_log.py	Adds unit tests asserting correct metric emission and tagging behavior.

pierrejeambrun

Cool that looks promising.

A few comments before we can move forward.

Would love another pair of eyes. (@jedcunningham?)

pierrejeambrun · 2026-04-07T13:19:42Z

Also cardinality can be a problem and we need a futher understanding of the impact of such metrics. I believe last time I checked it was around 60k time series introduced, which seems too much.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

ashb

As well as the individaul review comments:

Was using a prometheus style /metricz endpoint considered? That will likely work much better for many deployments rather than needing a push-style metric that the Airflow stats mechanism requires.

Remove the configurable path-to-surface mapping and emit metrics only for core API and UI routes. Use normalized route templates with method and status family tags to keep metric cardinality bounded.

henry3260 · 2026-07-14T16:46:55Z

ready for review :)

boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Mar 31, 2026

henry3260 force-pushed the add-metrics-in-rest-api branch from 4233a57 to 75bbd10 Compare March 31, 2026 14:30

henry3260 force-pushed the add-metrics-in-rest-api branch from 75bbd10 to d18b47b Compare March 31, 2026 21:12

henry3260 marked this pull request as ready for review April 1, 2026 13:18

henry3260 requested review from amoghrajesh, ashb, bugraoz93, choo121600, ephraimbuddy, jason810496, pierrejeambrun, potiuk, rawwar and shubhamraj-git as code owners April 1, 2026 13:18

potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Apr 1, 2026

henry3260 force-pushed the add-metrics-in-rest-api branch from 7fa1d0b to 33604c0 Compare April 1, 2026 17:17

kaxil requested a review from Copilot April 2, 2026 00:41

Copilot started reviewing on behalf of kaxil April 2, 2026 00:42 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/api_fastapi/common/http_access_log.py Outdated

henry3260 force-pushed the add-metrics-in-rest-api branch from 33604c0 to af907c0 Compare April 2, 2026 06:28

pierrejeambrun reviewed Apr 3, 2026

View reviewed changes

pierrejeambrun added this to the Airflow 3.3.0 milestone Apr 3, 2026

pierrejeambrun removed the ready for maintainer review Set after triaging when all criteria pass. label Apr 3, 2026

potiuk force-pushed the add-metrics-in-rest-api branch from af907c0 to 719146c Compare April 6, 2026 21:54

kaxil requested a review from Copilot April 10, 2026 19:55

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/api_fastapi/common/http_access_log.py Outdated

Comment thread airflow-core/src/airflow/api_fastapi/common/http_access_log.py Outdated

Comment thread shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml Outdated

henry3260 force-pushed the add-metrics-in-rest-api branch 2 times, most recently from eb855b2 to 6ca8a2b Compare April 12, 2026 13:03

henry3260 force-pushed the add-metrics-in-rest-api branch from 3552007 to 84fa5a1 Compare June 8, 2026 08:03

jscheffl mentioned this pull request Jun 9, 2026

Explicitly initialize Task SDK Stats in API server lifespan (hardening; backport fix for 3.2.x) #68078

Merged

1 task

henry3260 force-pushed the add-metrics-in-rest-api branch from 84fa5a1 to c997391 Compare June 9, 2026 15:08

add metrics

d1941f4

henry3260 force-pushed the add-metrics-in-rest-api branch from c997391 to 365fdf6 Compare June 10, 2026 07:57

henry3260 requested review from jason810496, jscheffl and pierrejeambrun June 10, 2026 09:36

pierrejeambrun reviewed Jun 10, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/config_templates/config.yml Outdated

Lee-W self-requested a review June 10, 2026 13:44

ashb requested changes Jun 10, 2026

View reviewed changes

allow user define the mapping themself

4523b2d

henry3260 force-pushed the add-metrics-in-rest-api branch from 365fdf6 to 4523b2d Compare June 13, 2026 18:56

vatsrahul1001 modified the milestones: Airflow 3.3.0, Airflow 3.4.0 Jun 15, 2026

Move API metrics config validation into configuration parser

1384714

henry3260 force-pushed the add-metrics-in-rest-api branch from 3387960 to 1384714 Compare June 15, 2026 19:19

henry3260 requested review from ashb and pierrejeambrun June 15, 2026 20:57

ashb reviewed Jun 17, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/api_fastapi/common/http_access_log.py Outdated

Comment thread airflow-core/src/airflow/config_templates/config.yml Outdated

Comment thread airflow-core/src/airflow/configuration.py Outdated

henry3260 force-pushed the add-metrics-in-rest-api branch from dbab760 to fbbee16 Compare June 20, 2026 13:55

henry3260 requested a review from ashb June 20, 2026 14:25

henry3260 force-pushed the add-metrics-in-rest-api branch from fbbee16 to 309e389 Compare June 20, 2026 16:46

potiuk assigned henry3260 Jun 22, 2026

pierrejeambrun mentioned this pull request Jul 6, 2026

Add per-route request metrics to the API server #69449

Closed

1 task

henry3260 force-pushed the add-metrics-in-rest-api branch 2 times, most recently from 77145d4 to 82cc3d7 Compare July 12, 2026 18:03

Simplify REST API metrics tagging

edb9715

Remove the configurable path-to-surface mapping and emit metrics only for core API and UI routes. Use normalized route templates with method and status family tags to keep metric cardinality bounded.

henry3260 force-pushed the add-metrics-in-rest-api branch from 82cc3d7 to edb9715 Compare July 12, 2026 18:03

Uh oh!

Conversation

henry3260 commented Mar 31, 2026 • edited by potiuk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Was generative AI tooling used to co-author this PR?

Uh oh!

henry3260 commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun commented Apr 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ashb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

henry3260 commented Jul 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

henry3260 commented Mar 31, 2026 •

edited by potiuk

Loading