add metric object type by andrewqian2001datadog · Pull Request #837 · DataDog/datadogpy

andrewqian2001datadog · 2024-06-26T19:41:13Z

What does this PR do?

This PR is for adding metric objects that will make aggregation easier by allowing us to have one Metric object instantiation for each unique metric context that we can add samples to which will make aggregation easier. Existing behavior already exists in the client side aggregation for GO

Description of the Change

Create metric object type (MetricAgggregator) that other metric objects (CountMetric, GaugeMetric, etc.) implement that will enforce the objects to implement functions aggregate and potentially flush_unsafe. This should also allow us to group the different metric types together in one list to be sent when we flush the metrics.

Possible Drawbacks

Some future metrics may not have the necessary fields to be of type MetricAggregator

The existing code for flushing metrics does NOT use object oriented programming, as a result they have duplicate code for each metric type.

Maybe i'm missing the potential drawbacks to using object oriented programming (specifically inheritance)?

Update:

Verification Process

Unit tests that verify the new metric objects match the existing behavior in the client side aggregator for Go.

Additional Notes

The python dogstatsd API will need to be refactored in the future to use these metric objects.

Release Notes

N/A, no new behavior until the aggregator class is implemented

Review checklist (to be filled by reviewers)

Feature or bug fix MUST have appropriate tests (unit, integration, etc...)
PR title must be written as a CHANGELOG entry (see why)
Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
PR must have one changelog/ label attached. If applicable it should have the backward-incompatible label attached.
PR should not have do-not-merge/ label attached.
If Applicable, issue must have kind/ and severity/ labels attached at least.

datadog/dogstatsd/metrics.py

gh123man · 2024-06-28T15:15:34Z

datadog/dogstatsd/metrics.py

+        super(SetMetric, self).__init__(name, tags, rate)
+        self.data = set()
+        self.data.add(value)
+        self.lock = Lock()


Similarly - maybe we should wait to add synchronization primitives until we actually have multithreaded code running. We can keep this PR focused to just introducing the metric classes and their associated properties.

the serialization function needs values like namespace and container_id which is set in the base.py class. I think it would be more convenient to leave it as a function in base.py or move it to aggregator.py? Not sure.

This is what aggregator.py looks like so far

Maybe we can move the serialization function there since it should have access to the DogStatsd client (it does in the existing go code) and its properties? It seems more intuitive that aggregator.py would handle serialization of a metric.

gh123man

LGTM!

tests/unit/dogstatsd/test_metrics.py

andrewqian2001datadog · 2024-07-01T17:49:48Z

/merge

dd-devflow · 2024-07-01T17:49:54Z

🚂 MergeQueue: pull request added to the queue

The median merge time in master is 0s.

Use /merge -c to cancel this operation!

andrewqian2001datadog · 2024-07-01T18:49:16Z

/remove

dd-devflow · 2024-07-01T18:49:23Z

🚂 Devflow: /remove

dd-devflow · 2024-07-03T21:50:18Z

❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build took longer than expected. The current limit for the base branch 'master' is 120 minutes.

If you need support, contact us on Slack #devflow with those details!

andrewqian2001datadog · 2024-07-05T15:40:22Z

/merge

dd-devflow · 2024-07-05T15:40:29Z

🚂 MergeQueue: pull request added to the queue

The median merge time in master is 0s.

Use /merge -c to cancel this operation!

dd-devflow · 2024-07-05T17:40:35Z

❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build took longer than expected. The current limit for the base branch 'master' is 120 minutes.

If you need support, contact us on Slack #devflow with those details!

andrewqian2001datadog · 2024-07-08T16:33:16Z

/merge

dd-devflow · 2024-07-08T16:33:23Z

🚂 MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

Use /merge -c to cancel this operation!

dd-devflow · 2024-07-08T16:34:27Z

🚂 MergeQueue: pull request added to the queue

The median merge time in master is 0s.

Use /merge -c to cancel this operation!

dd-devflow · 2024-07-08T18:34:32Z

❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build took longer than expected. The current limit for the base branch 'master' is 120 minutes.

If you need support, contact us on Slack #devflow with those details!

andrewqian2001datadog added 3 commits June 26, 2024 15:33

add metric objects

a92ce4a

fix class definition to work with python2

f4951a8

fix formatting

cce06f0

andrewqian2001datadog self-assigned this Jun 27, 2024

andrewqian2001datadog requested a review from gh123man June 27, 2024 19:17

andrewqian2001datadog marked this pull request as ready for review June 27, 2024 19:18

andrewqian2001datadog requested a review from a team as a code owner June 27, 2024 19:18

andrewqian2001datadog requested a review from a team June 27, 2024 19:18

andrewqian2001datadog added the changelog/Added Added features results into a minor version bump label Jun 27, 2024

andrewqian2001datadog changed the title ~~add metric objects~~ add metric object type Jun 27, 2024

rayz reviewed Jun 27, 2024

View reviewed changes

datadog/dogstatsd/metrics.py Outdated Show resolved Hide resolved

gh123man reviewed Jun 28, 2024

View reviewed changes

comment out potentially unecessary function

6ff91cb

andrewqian2001datadog requested review from gh123man and rayz June 28, 2024 16:47

andrewqian2001datadog added 4 commits June 28, 2024 13:00

remove commented out code

9741d33

fix linter

4849674

rerun tests

7ecc4cc

add optional timestamp property

d76e2dd

gh123man previously approved these changes Jul 1, 2024

View reviewed changes

gh123man reviewed Jul 1, 2024

View reviewed changes

tests/unit/dogstatsd/test_metrics.py Show resolved Hide resolved

dd-devflow bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: queued labels Jul 1, 2024

dd-devflow bot added mergequeue-status: removed and removed mergequeue-status: in_progress labels Jul 1, 2024

dd-devflow bot added the mergequeue-status: rejected label Jul 3, 2024

rayz approved these changes Jul 5, 2024

View reviewed changes

dd-devflow bot added mergequeue-status: queued mergequeue-status: in_progress mergequeue-status: rejected and removed mergequeue-status: rejected mergequeue-status: queued mergequeue-status: in_progress labels Jul 5, 2024

dd-devflow bot added mergequeue-status: waiting mergequeue-status: queued and removed mergequeue-status: rejected mergequeue-status: waiting labels Jul 8, 2024

dd-devflow bot added mergequeue-status: in_progress mergequeue-status: rejected and removed mergequeue-status: queued mergequeue-status: in_progress labels Jul 8, 2024

andrewqian2001datadog merged commit 8ba18e8 into master Jul 8, 2024

andrewqian2001datadog deleted the add-metric-objects branch July 8, 2024 19:21

dbenamydd restored the add-metric-objects branch July 10, 2024 04:26

dbenamydd deleted the add-metric-objects branch July 10, 2024 04:34

andrewqian2001datadog mentioned this pull request Jul 31, 2024

Add client side aggregation #844

Merged

6 tasks

andrewqian2001datadog mentioned this pull request Sep 9, 2024

add buffered_metrics object type #853

Merged

6 tasks

Conversation

andrewqian2001datadog commented Jun 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Description of the Change

Possible Drawbacks

Verification Process

Additional Notes

Release Notes

Review checklist (to be filled by reviewers)

Uh oh!

Uh oh!

Uh oh!

gh123man Jun 28, 2024

Choose a reason for hiding this comment

Uh oh!

andrewqian2001datadog Jun 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gh123man left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrewqian2001datadog commented Jul 1, 2024

Uh oh!

dd-devflow bot commented Jul 1, 2024

Uh oh!

andrewqian2001datadog commented Jul 1, 2024

Uh oh!

dd-devflow bot commented Jul 1, 2024

Uh oh!

dd-devflow bot commented Jul 3, 2024

Uh oh!

andrewqian2001datadog commented Jul 5, 2024

Uh oh!

dd-devflow bot commented Jul 5, 2024

Uh oh!

dd-devflow bot commented Jul 5, 2024

Uh oh!

andrewqian2001datadog commented Jul 8, 2024

Uh oh!

dd-devflow bot commented Jul 8, 2024

Uh oh!

dd-devflow bot commented Jul 8, 2024

Uh oh!

dd-devflow bot commented Jul 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andrewqian2001datadog commented Jun 26, 2024 •

edited

Loading

andrewqian2001datadog Jun 30, 2024 •

edited

Loading