Skip to content

feat: add delay option to CW exporter#4936

Merged
kgeckhart merged 7 commits intografana:mainfrom
tmeijn:feat/add-delay-option-v2
Dec 4, 2025
Merged

feat: add delay option to CW exporter#4936
kgeckhart merged 7 commits intografana:mainfrom
tmeijn:feat/add-delay-option-v2

Conversation

@tmeijn
Copy link
Contributor

@tmeijn tmeijn commented Nov 25, 2025

PR Description

  • Add delay option to prometheus.exporter.cloudwatch component to delay scraping of metrics to account for CloudWatch ingestion latency.

  • Export yace_.* metrics from the underlying YACE Exporter to prometheus.exporter.cloudwatch.

Which issue(s) this PR fixes

Fixes #3929, Partially #398

Notes to the Reviewer

Warning

This is vibe coded, so it might not be the best. However testing a local build it works as I'd expect.

Validation of yace_.* metrics:

image

delay option present in alloy ui:

image

CloudWatch vs Prometheus

image

PR Checklist

  • CHANGELOG.md updated
  • Documentation added
  • Tests updated
  • Config converters updated

@tmeijn tmeijn force-pushed the feat/add-delay-option-v2 branch from 8b86fc1 to 2bfe083 Compare November 25, 2025 20:44
@tmeijn tmeijn marked this pull request as ready for review November 27, 2025 17:50
@tmeijn tmeijn requested review from a team and clayton-cornell as code owners November 27, 2025 17:51
Copy link
Contributor

@kalleep kalleep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes looks mostly fine to me but I left one question, also I can see that @kgeckhart was involved with the original pr so maybe he have more context around this

@clayton-cornell clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Nov 28, 2025
@tmeijn
Copy link
Contributor Author

tmeijn commented Dec 3, 2025

Hey @kalleep, @kgeckhart anything I can do to help this move forwards?

Copy link
Contributor

@kgeckhart kgeckhart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's ship this and I'll open an issue upstream to add a 2 minute delay for 1 minute periods. We know from running our hosted product that 1 minute periods require a 2 minute delay but YACE doesn't do this for you and you must know to do it yourself instead. The referenced alloy issue regarding metrics being too low is for 1 minute periods as well.

We still need to expose delay because there are other use cases where adding delay can be necessary but they are the exception and in most cases the default roundingPeriod logic works as expected.

@kgeckhart kgeckhart merged commit 1ab7f12 into grafana:main Dec 4, 2025
39 checks passed
dehaansa pushed a commit to madhub/alloy that referenced this pull request Dec 10, 2025
* feat: add delay option to CW exporter

* linting + changelog

* fix incorrect support for delay in static jobs

* fix changelog
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 19, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

frozen-due-to-age type/docs Docs Squad label across all Grafana Labs repos

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cloudwatch metrics are too low

4 participants