Refactoring span benchmarking as per the spec

[srprash]

Making the following changes to span creation according to the span configuration for performance benchmarking:

• Adding a service resource
• Adding the always on sampler
• Adding a single long type attribute
• Adding an event with no attributes

The benchmark result on a m5.xlarge EC2 instance:

Benchmark                                                                         Mode  Cnt     Score     Error   Units
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread                                thrpt   10  3274.915 ±  16.454  ops/ms
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.alloc.rate                 thrpt   10   882.267 ±   3.930  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.alloc.rate.norm            thrpt   10   424.000 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.churn.G1_Eden_Space        thrpt   10   893.436 ± 102.937  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.churn.G1_Eden_Space.norm   thrpt   10   429.436 ±  51.007    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.churn.G1_Old_Gen           thrpt   10     0.001 ±   0.002  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.churn.G1_Old_Gen.norm      thrpt   10     0.001 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.count                      thrpt   10    63.000            counts
SpanBenchmark.simpleSpanStartAddEventEnd_01Thread:·gc.time                       thrpt   10    34.000                ms
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads                               thrpt   10  4865.498 ±  73.035  ops/ms
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.alloc.rate                thrpt   10  1309.461 ±  20.750  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.alloc.rate.norm           thrpt   10   424.001 ±   0.002    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.churn.G1_Eden_Space       thrpt   10  1316.544 ± 126.221  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.churn.G1_Eden_Space.norm  thrpt   10   426.263 ±  39.532    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.churn.G1_Old_Gen          thrpt   10     0.003 ±   0.002  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.churn.G1_Old_Gen.norm     thrpt   10     0.001 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.count                     thrpt   10    77.000            counts
SpanBenchmark.simpleSpanStartAddEventEnd_02Threads:·gc.time                      thrpt   10    46.000                ms
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads                               thrpt   10  6313.366 ±  50.898  ops/ms
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.alloc.rate                thrpt   10  1705.337 ±  14.369  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.alloc.rate.norm           thrpt   10   424.000 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.churn.G1_Eden_Space       thrpt   10  1704.363 ± 152.672  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.churn.G1_Eden_Space.norm  thrpt   10   423.786 ±  38.624    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.churn.G1_Old_Gen          thrpt   10     0.002 ±   0.002  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.churn.G1_Old_Gen.norm     thrpt   10     0.001 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.count                     thrpt   10    84.000            counts
SpanBenchmark.simpleSpanStartAddEventEnd_05Threads:·gc.time                      thrpt   10    63.000                ms
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads                               thrpt   10  6310.809 ± 140.526  ops/ms
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.alloc.rate                thrpt   10  1717.626 ±  41.551  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.alloc.rate.norm           thrpt   10   424.001 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.churn.G1_Eden_Space       thrpt   10  1743.940 ± 118.514  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.churn.G1_Eden_Space.norm  thrpt   10   430.611 ±  31.704    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.churn.G1_Old_Gen          thrpt   10     0.002 ±   0.002  MB/sec
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.churn.G1_Old_Gen.norm     thrpt   10     0.001 ±   0.001    B/op
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.count                     thrpt   10    88.000            counts
SpanBenchmark.simpleSpanStartAddEventEnd_10Threads:·gc.time                      thrpt   10    73.000                ms

[codecov]

Codecov Report

Merging #2186 (833f3fa) into master (6be6337) will increase coverage by 0.75%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2186      +/-   ##
============================================
+ Coverage     85.02%   85.77%   +0.75%     
- Complexity     2168     2209      +41     
============================================
  Files           248      254       +6     
  Lines          8318     8363      +45     
  Branches        924      922       -2     
============================================
+ Hits           7072     7173     +101     
+ Misses          909      860      -49     
+ Partials        337      330       -7     

Impacted Files	Coverage Δ	Complexity Δ
...n/java/io/opentelemetry/api/trace/SpanContext.java	77.77% <0.00%> (-11.70%)	9.00% <0.00%> (-5.00%)
...java/io/opentelemetry/sdk/trace/data/SpanData.java	92.85% <0.00%> (-7.15%)	1.00% <0.00%> (+1.00%)	⬇️
...metry/api/trace/attributes/SemanticAttributes.java	74.00% <0.00%> (-6.44%)	1.00% <0.00%> (ø%)
...telemetry/sdk/trace/export/BatchSpanProcessor.java	85.25% <0.00%> (-3.21%)	9.00% <0.00%> (ø%)
...ntelemetry/sdk/testing/assertj/SpanDataAssert.java	94.96% <0.00%> (-1.47%)	49.00% <0.00%> (-6.00%)
...io/opentelemetry/sdk/trace/config/TraceConfig.java	93.54% <0.00%> (-1.46%)	6.00% <0.00%> (ø%)
.../java/io/opentelemetry/sdk/resources/Resource.java	92.00% <0.00%> (-0.99%)	15.00% <0.00%> (+1.00%)	⬇️
...xtension/incubator/trace/data/SpanDataBuilder.java	62.00% <0.00%> (-0.97%)	7.00% <0.00%> (-1.00%)
...ntelemetry/sdk/trace/RecordEventsReadableSpan.java	99.03% <0.00%> (-0.50%)	72.00% <0.00%> (ø%)
...va/io/opentelemetry/exporter/otlp/SpanAdapter.java	95.78% <0.00%> (-0.33%)	24.00% <0.00%> (+3.00%)	⬇️
... and 38 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6be6337...833f3fa. Read the comment docs.

[jkwatson]

The specification you reference also includes testing the export pipeline through the OTLP exporter, which isn't included in this benchmark (intentionally). What are your thoughts about testing the full scenario that is described by the spec?

I'm not opposed to changing this benchmark code (although now any older benchmarks become irrelevant, because the scenario has changed), but I wouldn't want to say that the performance testing described in that spec is complete, unfortunately.

[srprash]

Yes. I'm working on adding the benchmark with the OTLP exporter in the SpanPipelineBenchmark. I'll add that here in subsequent commits. That would make the benchmark complete.

[srprash]

@jkwatson When you say "although now any older benchmarks become irrelevant, because the scenario has changed", are the older benchmarks published somewhere?

[jkwatson]

@jkwatson When you say "although now any older benchmarks become irrelevant, because the scenario has changed", are the older benchmarks published somewhere?

Hmm. possibly? Maybe on the original PR that submitted them, if anywhere. As I said...it's not a strong objection, just a thought I had.

[jkwatson]

Yes. I'm working on adding the benchmark with the OTLP exporter in the SpanPipelineBenchmark. I'll add that here in subsequent commits. That would make the benchmark complete.

I'd be happy to get this bit in first, if you'd prefer; I just wanted to make sure we didn't think we were done with just this change. :)

[srprash]

Yes. I'm working on adding the benchmark with the OTLP exporter in the SpanPipelineBenchmark. I'll add that here in subsequent commits. That would make the benchmark complete.

I'd be happy to get this bit in first, if you'd prefer; I just wanted to make sure we didn't think we were done with just this change. :)

Sure, I'd prefer this to be merged in. And I acknowledge that we need the benchmarks with OTLP exporter as well, which I'll do in a separate PR.

[jkwatson]

Thanks!