System.Tests.Perf_Version.Parse2 has regressed on Unix

[adamsitnik]

System.Tests.Perf_Version.Parse2 has regressed on Unix:

but not on Windows:

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter System.Tests.Perf_Version.Parse2

Details

System.Tests.Perf_Version.Parse2

Result	Base	Diff	Ratio	Operating System	Bit	Processor Name
Same	24.86	23.83	1.04	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	16.60	16.57	1.00	Windows 11	X64	AMD Ryzen 9 5900X
Same	30.12	28.99	1.04	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	40.05	44.17	0.91	Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same	23.24	23.15	1.00	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	25.28	25.82	0.98	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same	25.86	28.70	0.90	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Same	38.79	38.17	1.02	Windows 11	X64	Unknown processor
Same	34.06	36.19	0.94	Windows 11	X64	Unknown processor
Slower	20.51	23.54	0.87	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	28.60	33.92	0.84	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	60.71	65.94	0.92	centos 7	X64	Intel Xeon CPU E5530 2.40GHz
Slower	45.11	50.60	0.89	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	23.23	28.10	0.83	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	28.86	31.77	0.91	ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	28.80	34.15	0.84	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	37.57	36.67	1.02	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	36.64	34.33	1.07	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	53.16	57.09	0.93	Windows 10	Arm	Microsoft SQ1 3.0 GHz
Slower	36.42	43.70	0.83	macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower	34.79	40.25	0.86	macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

System.Tests.Perf_Version.Parse2 has regressed on [Unix]https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_ubuntu%2018.04%2fSystem.Tests.Perf_Version.Parse2.html):

but not on Windows:

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter System.Tests.Perf_Version.Parse2

Details

System.Tests.Perf_Version.Parse2

Result	Base	Diff	Ratio	Operating System	Bit	Processor Name
Same	24.86	23.83	1.04	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	16.60	16.57	1.00	Windows 11	X64	AMD Ryzen 9 5900X
Same	30.12	28.99	1.04	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	40.05	44.17	0.91	Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same	23.24	23.15	1.00	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	25.28	25.82	0.98	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same	25.86	28.70	0.90	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Same	38.79	38.17	1.02	Windows 11	X64	Unknown processor
Same	34.06	36.19	0.94	Windows 11	X64	Unknown processor
Slower	20.51	23.54	0.87	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	28.60	33.92	0.84	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	60.71	65.94	0.92	centos 7	X64	Intel Xeon CPU E5530 2.40GHz
Slower	45.11	50.60	0.89	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	23.23	28.10	0.83	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	28.86	31.77	0.91	ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	28.80	34.15	0.84	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	37.57	36.67	1.02	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	36.64	34.33	1.07	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	53.16	57.09	0.93	Windows 10	Arm	Microsoft SQ1 3.0 GHz
Slower	36.42	43.70	0.83	macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower	34.79	40.25	0.86	macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

Author:	adamsitnik
Assignees:	-
Labels:	area-System.Runtime, os-linux, tenet-performance
Milestone:	-

[stephentoub]

I don't see any changes to Version.Parse or number parsing in general in or around Sept 2021. Unless I just missed it, if there's actually a regression here, seems likely it's somewhere in code gen?

[adamsitnik]

The regression has most likely been caused by PGO data update (#58621)

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

System.Tests.Perf_Version.Parse2 has regressed on Unix:

but not on Windows:

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter System.Tests.Perf_Version.Parse2

Details

System.Tests.Perf_Version.Parse2

Result	Base	Diff	Ratio	Operating System	Bit	Processor Name
Same	24.86	23.83	1.04	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	16.60	16.57	1.00	Windows 11	X64	AMD Ryzen 9 5900X
Same	30.12	28.99	1.04	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	40.05	44.17	0.91	Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same	23.24	23.15	1.00	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	25.28	25.82	0.98	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same	25.86	28.70	0.90	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Same	38.79	38.17	1.02	Windows 11	X64	Unknown processor
Same	34.06	36.19	0.94	Windows 11	X64	Unknown processor
Slower	20.51	23.54	0.87	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	28.60	33.92	0.84	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	60.71	65.94	0.92	centos 7	X64	Intel Xeon CPU E5530 2.40GHz
Slower	45.11	50.60	0.89	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	23.23	28.10	0.83	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	28.86	31.77	0.91	ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	28.80	34.15	0.84	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	37.57	36.67	1.02	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	36.64	34.33	1.07	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	53.16	57.09	0.93	Windows 10	Arm	Microsoft SQ1 3.0 GHz
Slower	36.42	43.70	0.83	macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower	34.79	40.25	0.86	macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

Author:	adamsitnik
Assignees:	-
Labels:	os-linux, tenet-performance, area-CodeGen-coreclr, untriaged
Milestone:	-

[JulieLeeMSFT]

The regression has most likely been caused by PGO data update (#58621)

CC @AndyAyersMS @EgorBo.