Skip to content

ref(producer): minor metrics improvements#694

Merged
bmckerry merged 1 commit into
mainfrom
ben/metrics-improvements
Jun 10, 2026
Merged

ref(producer): minor metrics improvements#694
bmckerry merged 1 commit into
mainfrom
ben/metrics-improvements

Conversation

@bmckerry

@bmckerry bmckerry commented Jun 10, 2026

Copy link
Copy Markdown
Member

Currently, taskproducer metrics look strange:

  • latency max spikes up to 1s: https://app.datadoghq.com/s/FH6-Y3/ij5-abd-evg
    • I think the latency issue is because when the worker tries to fetch from an empty queue, it has a 1s timeout before it runs check_task_future_completion()
    • This PR tries to fix this issue by having the worker run check_task_future_completion() before it tries to fetch a new task
    • This also simplifies the logic by only calling check_task_future_completion() from a single place
  • very few activations with pending futures are reported: https://app.datadoghq.com/s/FH6-Y3/cpk-wy4-7mv
    • I think this is because the majority of tasks don't produce anything, so any gauge samples that show a value are getting drowned out by samples reporting 0
    • This PR tries to fix this by only having the gauge report when there are pending activations in the queue

@bmckerry bmckerry requested a review from a team as a code owner June 10, 2026 18:49
@bmckerry bmckerry merged commit 8317fe0 into main Jun 10, 2026
27 checks passed
@bmckerry bmckerry deleted the ben/metrics-improvements branch June 10, 2026 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants