Skip to content

Refactor benchmark CI#7873

Merged
XSAM merged 15 commits intoopen-telemetry:mainfrom
XSAM:refactor-benchmark-ci
Mar 3, 2026
Merged

Refactor benchmark CI#7873
XSAM merged 15 commits intoopen-telemetry:mainfrom
XSAM:refactor-benchmark-ci

Conversation

@XSAM
Copy link
Copy Markdown
Member

@XSAM XSAM commented Feb 3, 2026

Resolves #4537

This PR replaces the github-action-benchmark approach with CodSpeed.

CodSpeed addresses:

An example of benchmark result: https://codspeed.io/XSAM/opentelemetry-go/runs/697aa351f5662999a88a8bf3

Benchmark page for opentelemetry-go: https://codspeed.io/open-telemetry/opentelemetry-go

@XSAM XSAM added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Feb 3, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.7%. Comparing base (4ba8200) to head (a1c2448).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #7873   +/-   ##
=====================================
  Coverage   81.7%   81.7%           
=====================================
  Files        304     304           
  Lines      23342   23342           
=====================================
+ Hits       19086   19090    +4     
+ Misses      3867    3864    -3     
+ Partials     389     388    -1     

see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dmathieu
Copy link
Copy Markdown
Member

dmathieu commented Feb 4, 2026

While I have no objection for this (it fixes the conflict issue which is a big improvement), I want to note that it also adds a dependency on an external private company (we've seen cases where external things like this dropped our access overnight).
Also, on the collector, CodSpeed ends up not being very relevant (see open-telemetry/opentelemetry-collector#14521 for example).

@XSAM
Copy link
Copy Markdown
Member Author

XSAM commented Feb 4, 2026

@dmathieu

Also, on the collector, CodSpeed ends up not being very relevant (see open-telemetry/opentelemetry-collector#14521 for example).

My assumption is because the collector repo uses ubuntu-latest, which is VM where it is a noisy environment, to run benchmarks.

https://github.com/open-telemetry/opentelemetry-collector/blob/5e3bf40de1335e61ab1f9550ccf1d9bcf8a8b58f/.github/workflows/go-benchmarks.yml#L13

We use bare-metal to run benchmarks instead. This would give us a stable result.

@pellared
Copy link
Copy Markdown
Member

@XSAM, bump

@XSAM XSAM requested a review from pellared March 3, 2026 12:53
Copy link
Copy Markdown
Member

@pellared pellared left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add https://codspeed.io/open-telemetry/opentelemetry-go somewhere in README.md?

@XSAM
Copy link
Copy Markdown
Member Author

XSAM commented Mar 3, 2026

Should we add https://codspeed.io/open-telemetry/opentelemetry-go somewhere in README.md?

Yeah, I am planning to do that once we can confirm the benchmark result is stable enough to be useful.

@XSAM XSAM merged commit e413dfc into open-telemetry:main Mar 3, 2026
29 checks passed
@XSAM XSAM deleted the refactor-benchmark-ci branch March 3, 2026 18:42
@pellared pellared added this to the v1.42.0 milestone Mar 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add continuous benchmarks

3 participants