Skip to content

[Perf] Changes at 2/26/2022 2:38:27 AM #3784

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.12
Baseline 0c33372da6e56b1150d6111ee279ab6f52231a98
Compare c8da2fd5d8b841fc34845218158399cf35eb8bcf
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 4.85 ms 4.49 ms 0.92 0.01 False
Count - Duration of single invocation 5.13 ms 4.67 ms 0.91 0.02 False
Count - Duration of single invocation 3.75 ms 3.31 ms 0.88 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.48737708030303 < 4.6141636706958264.
IsChangePoint: Marked as a change because one of 2/8/2022 12:37:33 AM, 2/11/2022 9:28:46 PM, 2/25/2022 11:02:46 PM, 3/1/2022 5:49:50 AM falls between 2/20/2022 5:50:14 AM and 3/1/2022 5:49:50 AM.
IsImprovementStdDev: Marked as improvement because 64.34634832129687 (T) = (0 -4500460.316874301) / Math.Sqrt((10627152922.546461 / (299)) + (196129875.5654471 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.0997509558949912 = (4999128.126093682 - 4500460.316874301) / 4999128.126093682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.669069198660715 < 4.949552988700397.
IsChangePoint: Marked as a change because one of 1/15/2022 9:05:03 AM, 2/15/2022 12:48:49 AM, 2/26/2022 10:27:55 AM, 3/1/2022 5:49:50 AM falls between 2/20/2022 5:50:14 AM and 3/1/2022 5:49:50 AM.
IsImprovementStdDev: Marked as improvement because 6.302454281717755 (T) = (0 -4728492.344083105) / Math.Sqrt((28863042239.856823 / (299)) + (9380445112.794176 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.05588967449838557 = (5008410.793061514 - 4728492.344083105) / 5008410.793061514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.306345409777778 < 3.546261253896588.
IsChangePoint: Marked as a change because one of 2/15/2022 12:48:49 AM, 2/26/2022 10:27:55 AM, 3/1/2022 5:49:50 AM falls between 2/20/2022 5:50:14 AM and 3/1/2022 5:49:50 AM.
IsImprovementStdDev: Marked as improvement because 20.6297238685053 (T) = (0 -3283154.7027353174) / Math.Sqrt((16881288828.491781 / (299)) + (235785646.5320235 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06011603460739471 = (3493148.966919431 - 3283154.7027353174) / 3493148.966919431 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.12
Baseline 0c33372da6e56b1150d6111ee279ab6f52231a98
Compare c8da2fd5d8b841fc34845218158399cf35eb8bcf
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstWithPredicate_LastElementMatches - Duration of single invocation 518.21 ns 459.86 ns 0.89 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 459.85831484385005 < 491.0330063684158.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/17/2022 3:48:53 PM, 2/19/2022 1:08:37 AM, 2/25/2022 11:02:46 PM, 3/1/2022 5:49:50 AM falls between 2/20/2022 5:50:14 AM and 3/1/2022 5:49:50 AM.
IsImprovementStdDev: Marked as improvement because 3.7533798392740776 (T) = (0 -466.3102948640107) / Math.Sqrt((375.5884232080384 / (282)) + (350.5934243325014 / (8))) is greater than 1.9682351736146575 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (282) + (8) - 2, .975) and 0.05131311636195156 = (491.5323516182623 - 466.3102948640107) / 491.5323516182623 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 0c33372da6e56b1150d6111ee279ab6f52231a98
Compare c8da2fd5d8b841fc34845218158399cf35eb8bcf
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 117.10 ms 101.09 ms 0.86 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 101.0861945 < 108.23783633633242.
IsChangePoint: Marked as a change because one of 2/8/2022 12:37:33 AM, 2/11/2022 9:28:46 PM, 2/25/2022 11:02:46 PM, 3/1/2022 5:49:50 AM falls between 2/20/2022 5:50:14 AM and 3/1/2022 5:49:50 AM.
IsImprovementStdDev: Marked as improvement because 81.58236125039869 (T) = (0 -101233818.40238094) / Math.Sqrt((6385515615224.782 / (299)) + (29264686117.397625 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.11304852716683375 = (114136817.51833881 - 101233818.40238094) / 114136817.51833881 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.12
Baseline 0c33372da6e56b1150d6111ee279ab6f52231a98
Compare c8da2fd5d8b841fc34845218158399cf35eb8bcf
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 126.00 ns 118.00 ns 0.94 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 118.002386307482 < 120.53650479435872.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/25/2022 11:02:46 PM, 3/1/2022 5:49:50 AM falls between 2/20/2022 5:50:14 AM and 3/1/2022 5:49:50 AM.
IsImprovementStdDev: Marked as improvement because 32.25764577201101 (T) = (0 -118.86229416933939) / Math.Sqrt((3.74158353594347 / (299)) + (0.49709969444614915 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.06903067099119957 = (127.67584330183212 - 118.86229416933939) / 127.67584330183212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions