Skip to content

Regressions due to ArgumentOutOfRangeException.Throw helpers #79618

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_Comparison - Duration of single invocation 26.96 μs 29.62 μs 1.10 0.00 False
Array_ComparerStruct - Duration of single invocation 32.68 μs 35.53 μs 1.09 0.01 False
Array_ComparerClass - Duration of single invocation 27.01 μs 29.61 μs 1.10 0.00 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.615432857142853 > 28.2656906.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -24.955613646492015 (T) = (0 -29855.519273809517) / Math.Sqrt((24954.047928222095 / (51)) + (101510.15903279092 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.10614012435866599 = (26990.720810457522 - 29855.519273809517) / 26990.720810457522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntStruct&gt;.Array_ComparerStruct(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.526692000000004 > 34.29732320000001.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -44.01480017251592 (T) = (0 -35511.78942857143) / Math.Sqrt((136824.0049170818 / (51)) + (10502.714230639283 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.08500667059346934 = (32729.55862026861 - 35511.78942857143) / 32729.55862026861 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntStruct&gt;.Array_ComparerClass(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.614556 > 28.425599300000002.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -25.517792861072607 (T) = (0 -29810.798640109893) / Math.Sqrt((139083.4439055703 / (51)) + (58800.15129892933 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.09400656511907288 = (27249.195380162324 - 29810.798640109893) / 27249.195380162324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseInt64 - Duration of single invocation 6.14 ns 7.37 ns 1.20 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.368788642451924 > 6.446153648844837.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 4:14:20 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -83.28896426682208 (T) = (0 -7.3540463909732665) / Math.Sqrt((0.005891914346576905 / (51)) + (0.0007347730182528506 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.1948776656135081 = (6.154643778697916 - 7.3540463909732665) / 6.154643778697916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(Utf8TestCase)
       sub       rsp,48
       xor       eax,eax
       mov       [rsp+28],rax
       vxorps    xmm4,xmm4,xmm4
       vmovdqa   xmmword ptr [rsp+30],xmm4
       mov       [rsp+40],rax
       mov       rcx,[rdx+8]
       test      rcx,rcx
       je        short M00_L01
       lea       rdx,[rcx+10]
       mov       r8d,[rcx+8]
M00_L00:
       mov       [rsp+28],rdx
       mov       [rsp+30],r8d
       lea       rcx,[rsp+28]
       lea       rdx,[rsp+40]
       lea       r8,[rsp+38]
       xor       r9d,r9d
       call      qword ptr [7FFA7BB89210]; System.Buffers.Text.Utf8Parser.TryParse(System.ReadOnlySpan`1<Byte>, Int64 ByRef, Int32 ByRef, Char)
       nop
       add       rsp,48
       ret
M00_L01:
       xor       edx,edx
       xor       r8d,r8d
       jmp       short M00_L00
; Total bytes of code 90
; System.Buffers.Text.Utf8Parser.TryParse(System.ReadOnlySpan`1<Byte>, Int64 ByRef, Int32 ByRef, Char)
       sub       rsp,28
       movzx     eax,r9w
       test      eax,eax
       jne       short M01_L01
M01_L00:
       add       rsp,28
       jmp       qword ptr [7FFA7BB89270]; System.Buffers.Text.Utf8Parser.TryParseInt64D(System.ReadOnlySpan`1<Byte>, Int64 ByRef, Int32 ByRef)
M01_L01:
       movzx     eax,r9w
       or        eax,20
       cmp       eax,67
       jg        short M01_L02
       cmp       eax,64
       je        short M01_L00
       cmp       eax,67
       jne       short M01_L04
       jmp       short M01_L00
M01_L02:
       cmp       eax,6E
       je        short M01_L03
       cmp       eax,72
       je        short M01_L00
       cmp       eax,78
       jne       short M01_L04
       add       rsp,28
       jmp       qword ptr [7FFA7BB89450]
M01_L03:
       add       rsp,28
       jmp       qword ptr [7FFA7BB892D0]
M01_L04:
       call      qword ptr [7FFA7B99F6F0]
       int       3
; Total bytes of code 88

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ElementAt - Duration of single invocation 7.62 ns 8.63 ns 1.13 0.05 False
Range - Duration of single invocation 433.17 ns 489.02 ns 1.13 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.634429277801296 > 8.4986716006387.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -11.611064670743348 (T) = (0 -8.644962220111855) / Math.Sqrt((0.3091825271004132 / (51)) + (0.0003269768718154281 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.11722825793266357 = (7.737865703565919 - 8.644962220111855) / 7.737865703565919 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Range

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 489.0224068855866 > 454.82131945511316.
IsChangePoint: Marked as a change because one of 11/22/2022 2:20:42 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -16.07107367097568 (T) = (0 -485.5668904895548) / Math.Sqrt((41.760882533006324 / (23)) + (284.0239638919368 / (36))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (36) - 2, .025) and -0.1149634271548596 = (435.5002851785141 - 485.5668904895548) / 435.5002851785141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse4 - Duration of single invocation 49.48 ns 53.86 ns 1.09 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Parse4


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.86091103451226 > 52.08762155675408.
IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 11/5/2022 1:28:30 PM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -21.330719328272366 (T) = (0 -54.24342790236393) / Math.Sqrt((0.2721948367779376 / (51)) + (0.2728934077687653 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.08472151012200824 = (50.006778141850155 - 54.24342790236393) / 50.006778141850155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IntObj - Duration of single invocation 187.30 μs 226.33 μs 1.21 0.17 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
Details

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.IntObj


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 226.3340773809524 > 209.57269610829684.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 12/3/2022 7:24:45 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -23.85552183777217 (T) = (0 -226392.71607574186) / Math.Sqrt((95137082.2262806 / (51)) + (924618.7116872688 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.17413584962476428 = (192816.4582898082 - 226392.71607574186) / 192816.4582898082 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions