Skip to content

Make the batch processor limit data points rather than metrics.#3141

Merged
tigrannajaryan merged 3 commits intoopen-telemetry:mainfrom
kisieland:change-batch-processing
Jun 2, 2021
Merged

Make the batch processor limit data points rather than metrics.#3141
tigrannajaryan merged 3 commits intoopen-telemetry:mainfrom
kisieland:change-batch-processing

Conversation

@kisieland
Copy link
Copy Markdown
Contributor

@kisieland kisieland commented May 11, 2021

Description:
Changes batch processor logic to limit data points rather than metrics.

Link to tracking Issue: #2754

Testing:
Ran go test ./...
I've added some edge case test and alerted existing tests.

@kisieland kisieland requested a review from a team May 11, 2021 15:19
@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented May 11, 2021

@dashpole @bogdandrutu can you take a look?

@kisieland kisieland force-pushed the change-batch-processing branch from c522c15 to 9fd1d2e Compare May 11, 2021 15:44
dashpole
dashpole reviewed May 11, 2021
View reviewed changes
@dashpole
Copy link
Copy Markdown
Contributor

dashpole commented May 11, 2021

This was approved during the 3/24 sig meeting: #2754 (comment)

@dashpole
Copy link
Copy Markdown
Contributor

dashpole commented May 11, 2021

I think this needs a release note in the CHANGELOG.md under breaking changes

@kisieland kisieland force-pushed the change-batch-processing branch from 9fd1d2e to 25e0a4e Compare May 11, 2021 16:20
@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented May 11, 2021 •
edited
Loading

I think this needs a release note in the CHANGELOG.md under breaking changes

Done.
I guess my editor removed trailing spaces from other lines.

bogdandrutu
bogdandrutu previously requested changes May 11, 2021
View reviewed changes
@bogdandrutu
Copy link
Copy Markdown
Member

bogdandrutu commented May 11, 2021

@dashpole please help us to ensure we don't expose unnecessary public APIs if possible, any public API will cost us in the future.

@kisieland kisieland force-pushed the change-batch-processing branch 2 times, most recently from 88ffead to 3f4a67b Compare May 12, 2021 07:50
@tigrannajaryan
Copy link
Copy Markdown
Member

tigrannajaryan commented May 13, 2021

Is there any measurable performance impact due to this change?

@kisieland kisieland force-pushed the change-batch-processing branch from 3f4a67b to 955952a Compare May 13, 2021 12:57
@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented May 13, 2021 •
edited
Loading

@tigrannajaryan I've added new version of benchmark, It looks like there is about 20% increase on the time it takes to process data. But, as the batch does not take a significant time of the pipeline we can live with this, WDYT?
(Feel free to re-run and verify it)
On main branch I got:

$ GOMAXPROCS=12 go test -bench=BenchmarkBatchMetricProcessor -benchtime=20000x
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/collector/processor/batchprocessor
cpu: Intel(R) Xeon(R) CPU @ 2.30GHz
BenchmarkBatchMetricProcessor-12           20000             78924 ns/op
PASS
ok      go.opentelemetry.io/collector/processor/batchprocessor  22.330s

On change-batch-processing I've got:

$ GOMAXPROCS=12 go test -bench=BenchmarkBatchMetricProcessor -benchtime=20000x
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/collector/processor/batchprocessor
cpu: Intel(R) Xeon(R) CPU @ 2.30GHz
BenchmarkBatchMetricProcessor-12           20000             94004 ns/op
PASS
ok      go.opentelemetry.io/collector/processor/batchprocessor  22.502s

NOTE: This comment was edited, once I updated the Benchmark to exclude data initialisation. With the data initialisation included there was no significant differences.

@kisieland kisieland force-pushed the change-batch-processing branch from 5e9978e to 7e59f72 Compare May 14, 2021 07:46
bogdandrutu
bogdandrutu reviewed May 14, 2021
View reviewed changes
@kisieland kisieland requested a review from bogdandrutu May 18, 2021 08:08
@kisieland kisieland force-pushed the change-batch-processing branch from cc09fa0 to 4b53d58 Compare May 19, 2021 08:03
@tigrannajaryan tigrannajaryan self-assigned this May 19, 2021
@dashpole
Copy link
Copy Markdown
Contributor

dashpole commented May 19, 2021

@jrcamp @Aneurysm9 It would be helpful to have your reviews as well, if you have time.

Aneurysm9
Aneurysm9 reviewed May 19, 2021
View reviewed changes
tigrannajaryan
tigrannajaryan reviewed May 19, 2021
View reviewed changes
@kisieland kisieland requested a review from Aneurysm9 May 20, 2021 10:16
@kisieland kisieland force-pushed the change-batch-processing branch from 7f680d9 to f5e080a Compare May 24, 2021 11:37
@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented May 24, 2021

@bogdandrutu @Aneurysm9 please take a look

bogdandrutu
bogdandrutu reviewed May 24, 2021
View reviewed changes
@kisieland kisieland requested a review from bogdandrutu May 25, 2021 07:44
@tigrannajaryan
Copy link
Copy Markdown
Member

tigrannajaryan commented May 28, 2021

Please resolve the merge conflicts.

@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented May 31, 2021

Merge conflicts resolved.
@tigrannajaryan @bogdandrutu please take a look, I would appreciate if this PR was included in next release.

tigrannajaryan
tigrannajaryan approved these changes May 31, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@tigrannajaryan tigrannajaryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @kisieland

@tigrannajaryan tigrannajaryan dismissed bogdandrutu’s stale review May 31, 2021 15:17

All requested changes addressed

@tigrannajaryan
Copy link
Copy Markdown
Member

tigrannajaryan commented May 31, 2021

@bogdandrutu PTAL, I believe all your comments are addressed and we can merge.

@tigrannajaryan
Copy link
Copy Markdown
Member

tigrannajaryan commented May 31, 2021

@kisieland please squash the commits and add a descriptive commit message. Please also include the results of before/after benchmarking in the commit message.

@kisieland kisieland force-pushed the change-batch-processing branch from 3910a29 to 74aaaf6 Compare June 1, 2021 08:48
@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented Jun 1, 2021

@tigrannajaryan Done

@kisieland kisieland force-pushed the change-batch-processing branch from 74aaaf6 to 61c9ec2 Compare June 2, 2021 07:46
@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented Jun 2, 2021

@bogdandrutu PTAL

@kisieland
Make batch processor limit number of data points rather than number o…
1fd363d 
…f metrics.

This change introduces a BenchmarkBatchMetricProcessor that stress tests batching logic.
Results before:
`BenchmarkBatchMetricProcessor-12           20000             80614 ns/op`
Results after the change:
`BenchmarkBatchMetricProcessor-12           20000             96184 ns/op`
@kisieland kisieland force-pushed the change-batch-processing branch from 61c9ec2 to 1fd363d Compare June 2, 2021 13:24
@tigrannajaryan
Copy link
Copy Markdown
Member

tigrannajaryan commented Jun 2, 2021

@kisieland please fix the changelog conflict one more time (sorry).

@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented Jun 2, 2021

@tigrannajaryan I don't see any conflicts, I resolved them two times today.

bogdandrutu
bogdandrutu approved these changes Jun 2, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@bogdandrutu bogdandrutu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small optimization to avoid creating a new metric then copy it to destination, but directly copy metadata and points to the destination metric.

@kisieland
Copy link
Copy Markdown
Contributor Author

kisieland commented Jun 2, 2021

@bogdandrutu Please merge if everything is ok (merging is blocked for me)

@tigrannajaryan tigrannajaryan merged commit 6d44f0d into open-telemetry:main Jun 2, 2021
@tigrannajaryan
Copy link
Copy Markdown
Member

tigrannajaryan commented Jun 2, 2021

@kisieland Thank you for the contribution and for the patience.

@kisieland kisieland deleted the change-batch-processing branch June 9, 2021 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dashpole dashpole dashpole approved these changes

@carlosalberto carlosalberto carlosalberto approved these changes

@bogdandrutu bogdandrutu bogdandrutu approved these changes

@tigrannajaryan tigrannajaryan tigrannajaryan approved these changes

+1 more reviewer

@Aneurysm9 Aneurysm9 Aneurysm9 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@tigrannajaryan tigrannajaryan

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@kisieland @dashpole @bogdandrutu @tigrannajaryan @carlosalberto @Aneurysm9