GC.AllocateUninitializedArray has regressed on macOS

[adamsitnik]

GC.AllocateUninitializedArray benchmarks have regressed by 15% for "smaller size" (1000 elements) only on macOS (x64, I don't have arm64 data). Other Unixes are not affected and larger sizes (10000 elements) are not affected..

Details

.NET 6:

Type	Method	length	pinned	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
Perf_GC<Byte>	AllocateUninitializedArray	1000	False	99.14 ns	2.948 ns	3.276 ns	97.28 ns	96.46 ns	108.0 ns	0.4891	-	-	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	False	192.16 ns	0.779 ns	0.729 ns	191.89 ns	191.14 ns	193.4 ns	0.9669	-	-	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	1000	True	378.25 ns	4.336 ns	3.843 ns	376.98 ns	373.63 ns	385.4 ns	1.9396	1.9396	0.3246	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	True	527.51 ns	35.474 ns	34.840 ns	537.93 ns	440.44 ns	558.6 ns	3.8341	3.8341	0.6409	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	False	460.30 ns	13.229 ns	12.993 ns	465.62 ns	433.22 ns	472.1 ns	4.7617	-	-	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	False	863.26 ns	28.192 ns	31.336 ns	868.93 ns	799.42 ns	905.1 ns	9.5207	-	-	19.55 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	True	2,079.43 ns	193.755 ns	215.358 ns	2,184.01 ns	1,616.18 ns	2,218.1 ns	18.7916	18.7916	3.1319	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	True	4,294.48 ns	339.558 ns	391.035 ns	4,411.58 ns	3,194.01 ns	4,566.0 ns	36.7647	36.7647	6.1275	19.56 KB

.NET 7 Preview 1:

Type	Method	length	pinned	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
Perf_GC<Byte>	AllocateUninitializedArray	1000	False	115.0 ns	3.62 ns	4.02 ns	115.7 ns	107.7 ns	120.7 ns	0.4894	-	-	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	False	218.5 ns	10.22 ns	11.76 ns	221.6 ns	197.3 ns	238.9 ns	0.9663	-	-	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	1000	True	408.4 ns	44.27 ns	49.21 ns	403.0 ns	314.0 ns	498.6 ns	1.9366	1.9366	0.3239	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	True	622.0 ns	7.05 ns	6.25 ns	621.5 ns	607.3 ns	630.9 ns	3.8293	3.8293	0.6402	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	False	502.1 ns	36.32 ns	40.37 ns	500.5 ns	441.9 ns	573.4 ns	4.7603	-	-	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	False	834.6 ns	24.70 ns	25.36 ns	844.3 ns	785.4 ns	860.9 ns	9.5210	-	-	19.55 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	True	2,328.8 ns	272.49 ns	313.79 ns	2,413.5 ns	1,747.3 ns	2,679.2 ns	18.7466	18.7466	3.1301	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	True	4,128.7 ns	395.66 ns	439.78 ns	4,314.0 ns	3,130.7 ns	4,385.4 ns	36.7561	36.7561	6.1260	19.56 KB

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter '*AllocateUninitializedArray' --bdn-arguments "--join true"

Details

System.Tests.Perf_GC.AllocateArray(length: 10000, pinned: False)

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Same	248.92	243.83	1.02	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	179.83	184.15	0.98	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	459.82	461.10	1.00	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Faster	914.81	709.96	1.29	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	312.96	304.02	1.03	+0	bimodal	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	298.12	327.61	0.91	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Same	307.02	307.68	1.00	+0	several?	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Same	584.23	542.31	1.08	+0	several?	Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	420.18	431.82	0.97	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	301.75	307.80	0.98	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	483.25	487.37	0.99	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	1248.19	1157.88	1.08	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	807.84	747.88	1.08	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Same	372.58	389.82	0.96	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	391.98	388.72	1.01	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	365.02	365.98	1.00	+0		ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	610.43	598.45	1.02	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Same	269.72	275.66	0.98	+0		Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	489.28	492.36	0.99	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	827.06	821.72	1.01	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	853.39	1028.78	0.83	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	734.86	859.05	0.86	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

System.Tests.Perf_GC.AllocateUninitializedArray(length: 1000, pinned: False)

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Same	28.63	28.43	1.01	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	19.49	20.59	0.95	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	47.97	48.42	0.99	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Faster	96.60	73.23	1.32	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	32.77	32.21	1.02	+0	several?	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	30.69	32.90	0.93	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Same	31.55	32.59	0.97	+0	several?	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Same	59.60	62.74	0.95	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	42.11	46.20	0.91	+0	several?	Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	37.46	38.63	0.97	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	53.49	54.05	0.99	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	148.24	138.95	1.07	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	87.90	85.15	1.03	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Same	43.12	43.76	0.99	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	50.21	48.88	1.03	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	46.33	46.50	1.00	+0	multimodal	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	64.90	65.37	0.99	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Same	29.11	30.75	0.95	+0		Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	50.59	52.71	0.96	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	94.16	96.78	0.97	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	92.65	109.98	0.84	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	77.08	91.51	0.84	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

cc @VSadov

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

GC.AllocateUninitializedArray benchmarks have regressed by 15% for "smaller size" (1000 elements) only on macOS (x64, I don't have arm64 data). Other Unixes are not affected.

Details

.NET 6:

Type	Method	length	pinned	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
Perf_GC<Byte>	AllocateUninitializedArray	1000	False	99.14 ns	2.948 ns	3.276 ns	97.28 ns	96.46 ns	108.0 ns	0.4891	-	-	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	False	192.16 ns	0.779 ns	0.729 ns	191.89 ns	191.14 ns	193.4 ns	0.9669	-	-	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	1000	True	378.25 ns	4.336 ns	3.843 ns	376.98 ns	373.63 ns	385.4 ns	1.9396	1.9396	0.3246	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	True	527.51 ns	35.474 ns	34.840 ns	537.93 ns	440.44 ns	558.6 ns	3.8341	3.8341	0.6409	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	False	460.30 ns	13.229 ns	12.993 ns	465.62 ns	433.22 ns	472.1 ns	4.7617	-	-	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	False	863.26 ns	28.192 ns	31.336 ns	868.93 ns	799.42 ns	905.1 ns	9.5207	-	-	19.55 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	True	2,079.43 ns	193.755 ns	215.358 ns	2,184.01 ns	1,616.18 ns	2,218.1 ns	18.7916	18.7916	3.1319	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	True	4,294.48 ns	339.558 ns	391.035 ns	4,411.58 ns	3,194.01 ns	4,566.0 ns	36.7647	36.7647	6.1275	19.56 KB

.NET 7 Preview 1:

Type	Method	length	pinned	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
Perf_GC<Byte>	AllocateUninitializedArray	1000	False	115.0 ns	3.62 ns	4.02 ns	115.7 ns	107.7 ns	120.7 ns	0.4894	-	-	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	False	218.5 ns	10.22 ns	11.76 ns	221.6 ns	197.3 ns	238.9 ns	0.9663	-	-	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	1000	True	408.4 ns	44.27 ns	49.21 ns	403.0 ns	314.0 ns	498.6 ns	1.9366	1.9366	0.3239	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	True	622.0 ns	7.05 ns	6.25 ns	621.5 ns	607.3 ns	630.9 ns	3.8293	3.8293	0.6402	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	False	502.1 ns	36.32 ns	40.37 ns	500.5 ns	441.9 ns	573.4 ns	4.7603	-	-	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	False	834.6 ns	24.70 ns	25.36 ns	844.3 ns	785.4 ns	860.9 ns	9.5210	-	-	19.55 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	True	2,328.8 ns	272.49 ns	313.79 ns	2,413.5 ns	1,747.3 ns	2,679.2 ns	18.7466	18.7466	3.1301	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	True	4,128.7 ns	395.66 ns	439.78 ns	4,314.0 ns	3,130.7 ns	4,385.4 ns	36.7561	36.7561	6.1260	19.56 KB

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter '*AllocateUninitializedArray' --bdn-arguments "--join true"

Details

System.Tests.Perf_GC.AllocateArray(length: 10000, pinned: False)

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Same	248.92	243.83	1.02	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	179.83	184.15	0.98	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	459.82	461.10	1.00	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Faster	914.81	709.96	1.29	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	312.96	304.02	1.03	+0	bimodal	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	298.12	327.61	0.91	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Same	307.02	307.68	1.00	+0	several?	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Same	584.23	542.31	1.08	+0	several?	Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	420.18	431.82	0.97	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	301.75	307.80	0.98	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	483.25	487.37	0.99	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	1248.19	1157.88	1.08	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	807.84	747.88	1.08	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Same	372.58	389.82	0.96	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	391.98	388.72	1.01	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	365.02	365.98	1.00	+0		ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	610.43	598.45	1.02	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Same	269.72	275.66	0.98	+0		Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	489.28	492.36	0.99	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	827.06	821.72	1.01	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	853.39	1028.78	0.83	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	734.86	859.05	0.86	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

System.Tests.Perf_GC.AllocateUninitializedArray(length: 1000, pinned: False)

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Same	28.63	28.43	1.01	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	19.49	20.59	0.95	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	47.97	48.42	0.99	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Faster	96.60	73.23	1.32	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	32.77	32.21	1.02	+0	several?	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	30.69	32.90	0.93	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Same	31.55	32.59	0.97	+0	several?	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Same	59.60	62.74	0.95	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	42.11	46.20	0.91	+0	several?	Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	37.46	38.63	0.97	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	53.49	54.05	0.99	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	148.24	138.95	1.07	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	87.90	85.15	1.03	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Same	43.12	43.76	0.99	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	50.21	48.88	1.03	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	46.33	46.50	1.00	+0	multimodal	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	64.90	65.37	0.99	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Same	29.11	30.75	0.95	+0		Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	50.59	52.71	0.96	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	94.16	96.78	0.97	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	92.65	109.98	0.84	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	77.08	91.51	0.84	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

cc @VSadov

Author:	adamsitnik
Assignees:	-
Labels:	area-System.Runtime, os-mac-os-x, tenet-performance
Milestone:	-

[adamsitnik]

System.Tests.Perf_GC<Byte>.NewOperator_Array(length: 1000) and few similar benchmarks has regressed as well.

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

GC.AllocateUninitializedArray benchmarks have regressed by 15% for "smaller size" (1000 elements) only on macOS (x64, I don't have arm64 data). Other Unixes are not affected and larger sizes (10000 elements) are not affected..

Details

.NET 6:

Type	Method	length	pinned	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
Perf_GC<Byte>	AllocateUninitializedArray	1000	False	99.14 ns	2.948 ns	3.276 ns	97.28 ns	96.46 ns	108.0 ns	0.4891	-	-	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	False	192.16 ns	0.779 ns	0.729 ns	191.89 ns	191.14 ns	193.4 ns	0.9669	-	-	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	1000	True	378.25 ns	4.336 ns	3.843 ns	376.98 ns	373.63 ns	385.4 ns	1.9396	1.9396	0.3246	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	True	527.51 ns	35.474 ns	34.840 ns	537.93 ns	440.44 ns	558.6 ns	3.8341	3.8341	0.6409	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	False	460.30 ns	13.229 ns	12.993 ns	465.62 ns	433.22 ns	472.1 ns	4.7617	-	-	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	False	863.26 ns	28.192 ns	31.336 ns	868.93 ns	799.42 ns	905.1 ns	9.5207	-	-	19.55 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	True	2,079.43 ns	193.755 ns	215.358 ns	2,184.01 ns	1,616.18 ns	2,218.1 ns	18.7916	18.7916	3.1319	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	True	4,294.48 ns	339.558 ns	391.035 ns	4,411.58 ns	3,194.01 ns	4,566.0 ns	36.7647	36.7647	6.1275	19.56 KB

.NET 7 Preview 1:

Type	Method	length	pinned	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
Perf_GC<Byte>	AllocateUninitializedArray	1000	False	115.0 ns	3.62 ns	4.02 ns	115.7 ns	107.7 ns	120.7 ns	0.4894	-	-	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	False	218.5 ns	10.22 ns	11.76 ns	221.6 ns	197.3 ns	238.9 ns	0.9663	-	-	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	1000	True	408.4 ns	44.27 ns	49.21 ns	403.0 ns	314.0 ns	498.6 ns	1.9366	1.9366	0.3239	1 KB
Perf_GC<Char>	AllocateUninitializedArray	1000	True	622.0 ns	7.05 ns	6.25 ns	621.5 ns	607.3 ns	630.9 ns	3.8293	3.8293	0.6402	1.98 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	False	502.1 ns	36.32 ns	40.37 ns	500.5 ns	441.9 ns	573.4 ns	4.7603	-	-	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	False	834.6 ns	24.70 ns	25.36 ns	844.3 ns	785.4 ns	860.9 ns	9.5210	-	-	19.55 KB
Perf_GC<Byte>	AllocateUninitializedArray	10000	True	2,328.8 ns	272.49 ns	313.79 ns	2,413.5 ns	1,747.3 ns	2,679.2 ns	18.7466	18.7466	3.1301	9.79 KB
Perf_GC<Char>	AllocateUninitializedArray	10000	True	4,128.7 ns	395.66 ns	439.78 ns	4,314.0 ns	3,130.7 ns	4,385.4 ns	36.7561	36.7561	6.1260	19.56 KB

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter '*AllocateUninitializedArray' --bdn-arguments "--join true"

Details

System.Tests.Perf_GC.AllocateArray(length: 10000, pinned: False)

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Same	248.92	243.83	1.02	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	179.83	184.15	0.98	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	459.82	461.10	1.00	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Faster	914.81	709.96	1.29	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	312.96	304.02	1.03	+0	bimodal	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	298.12	327.61	0.91	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Same	307.02	307.68	1.00	+0	several?	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Same	584.23	542.31	1.08	+0	several?	Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	420.18	431.82	0.97	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	301.75	307.80	0.98	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	483.25	487.37	0.99	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	1248.19	1157.88	1.08	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	807.84	747.88	1.08	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Same	372.58	389.82	0.96	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	391.98	388.72	1.01	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	365.02	365.98	1.00	+0		ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	610.43	598.45	1.02	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Same	269.72	275.66	0.98	+0		Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	489.28	492.36	0.99	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	827.06	821.72	1.01	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	853.39	1028.78	0.83	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	734.86	859.05	0.86	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

System.Tests.Perf_GC.AllocateUninitializedArray(length: 1000, pinned: False)

Result	Base	Diff	Ratio	Alloc Delta	Modality	Operating System	Bit	Processor Name	Base V	Diff V
Same	28.63	28.43	1.01	+0		Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	19.49	20.59	0.95	+0		Windows 11	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	47.97	48.42	0.99	+0		Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Faster	96.60	73.23	1.32	+0		Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)	6.0.121.56705	7.0.22.7608
Same	32.77	32.21	1.02	+0	several?	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	30.69	32.90	0.93	+0		Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	6.0.121.56705	7.0.22.7608
Same	31.55	32.59	0.97	+0	several?	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	6.0.121.56705	7.0.22.7608
Same	59.60	62.74	0.95	+0		Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	42.11	46.20	0.91	+0	several?	Windows 11	X64	Unknown processor	6.0.121.56705	7.0.22.7608
Same	37.46	38.63	0.97	+0		ubuntu 20.04	X64	AMD Ryzen 9 5900X	6.0.121.56705	7.0.22.7608
Same	53.49	54.05	0.99	+0		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	148.24	138.95	1.07	+0		centos 7	X64	Intel Xeon CPU E5530 2.40GHz	6.0.121.56705	7.0.22.7608
Same	87.90	85.15	1.03	+0		ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)	6.0.121.56705	7.0.22.7608
Same	43.12	43.76	0.99	+0		alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	50.21	48.88	1.03	+0		ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	46.33	46.50	1.00	+0	multimodal	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	6.0.121.56705	7.0.22.7608
Same	64.90	65.37	0.99	+0		Windows 10	Arm64	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Same	29.11	30.75	0.95	+0		Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores	6.0.121.56705	7.0.22.7608
Same	50.59	52.71	0.96	+0		Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz	6.0.121.56705	7.0.22.7608
Same	94.16	96.78	0.97	+0		Windows 10	Arm	Microsoft SQ1 3.0 GHz	6.0.121.56705	7.0.22.7608
Slower	92.65	109.98	0.84	+0		macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)	6.0.121.56705	7.0.22.7608
Slower	77.08	91.51	0.84	+0		macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	6.0.121.56705	7.0.22.7608

cc @VSadov

Author:	adamsitnik
Assignees:	-
Labels:	os-mac-os-x, tenet-performance, area-GC-coreclr, untriaged
Milestone:	-

[adamsitnik]

The issue still persists with preview2, some benchmarks are even up to 5 times slower. It's specific to macOS x64.

Type	Method	Job	Runtime	Toolchain	Size	Mean	Error	StdDev	Median	Min	Max	Ratio	RatioSD	Gen 0	Allocated	Alloc Ratio
CtorGivenSize<Int32>	Array	Job-YIPOBQ	.NET 6.0	net6.0	512	196.4 ns	4.55 ns	5.24 ns	193.7 ns	190.9 ns	209.7 ns	1.00	0.00	0.9899	2.02 KB	1.00
CtorGivenSize<Int32>	Array	Job-FCGZAV	.NET 7.0	net7.0	512	1,062.0 ns	71.56 ns	82.40 ns	1,005.3 ns	992.6 ns	1,224.5 ns	5.40	0.34	12.6558	2.02 KB	1.00
CtorGivenSize<String>	Array	Job-YIPOBQ	.NET 6.0	net6.0	512	414.2 ns	8.11 ns	8.32 ns	415.9 ns	382.5 ns	419.2 ns	1.00	0.00	1.9677	4.02 KB	1.00
CtorGivenSize<String>	Array	Job-FCGZAV	.NET 7.0	net7.0	512	2,074.3 ns	7.78 ns	6.07 ns	2,075.9 ns	2,066.8 ns	2,086.6 ns	5.02	0.14	25.0000	4.02 KB	1.00

[Maoni0]

thanks for reporting! @MokoSan will be doing investigation on this.