Background
This issue page is opened in response to Intel's following inquiry (@helloguo):
I have one question about the perf tests SsePerformanceTests.cs. I saw ‘EXP_RANGE’ is initialized as private const int EXP_RANGE = EXP_MAX / 2;. It usually works very well since it tests a large range of floating point values. However, in some cases such as SumSqU, it causes a lot of FP_ASSIST, a hardware event causing some perf issue ( you can find more details here or the section ‘Floating-Point Performance Ratios’ in intel optimization manual). Basically, FP_ASSIST is caused by denormals, underflow numbers, NAN.
I would suggest to initialize ‘EXP_RANGE’ as private const int EXP_RANGE = EXP_MAX / 4; or private const int EXP_RANGE = EXP_MAX / 8;. On my machine, when I changed ‘EXP_RANGE’ to private const int EXP_RANGE = EXP_MAX / 4; the execution time reduced to 227 us from 469 us. Do you mind testing it on your machine and changing the perf tests if it makes sense to you?
Results (original, after Change 1, after Change 2)
Note: Since these results are obtained with the ShortRun mode, the mean is an average of only 3 numbers and can encapsulate a sizeable error.
- Original performance test results:
BenchmarkDotNet=v0.10.14, OS=Windows 10.0.17134
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.0.100-alpha1-20180720-2
[Host] : .NET Core 3.0.0-preview1-26710-03 (CoreCLR 4.6.26710.05, CoreFX 4.6.26708.04), 64bit RyuJIT
Toolchain=InProcessToolchain LaunchCount=1 TargetCount=3
WarmupCount=3
| Method |
Mean |
Error |
StdDev |
| NativeAddScalarUPerf |
259.8 us |
363.18 us |
20.520 us |
| ManagedAddScalarUPerf |
309.1 us |
434.44 us |
24.546 us |
| NativeScaleUPerf |
312.4 us |
1,012.74 us |
57.222 us |
| ManagedScaleUPerf |
259.0 us |
788.02 us |
44.525 us |
| NativeScaleSrcUPerf |
417.2 us |
969.85 us |
54.798 us |
| ManagedScaleSrcUPerf |
387.1 us |
1,137.38 us |
64.264 us |
| NativeScaleAddUPerf |
333.3 us |
670.78 us |
37.900 us |
| ManagedScaleAddUPerf |
306.6 us |
859.01 us |
48.536 us |
| NativeAddScaleUPerf |
413.0 us |
141.54 us |
7.997 us |
| ManagedAddScaleUPerf |
552.9 us |
2,980.25 us |
168.389 us |
| NativeAddScaleSUPerf |
4,734.3 us |
527.31 us |
29.794 us |
| ManagedAddScaleSUPerf |
4,942.8 us |
3,993.27 us |
225.627 us |
| NativeAddScaleCopyUPerf |
662.1 us |
1,882.68 us |
106.375 us |
| ManagedAddScaleCopyUPerf |
557.3 us |
424.36 us |
23.977 us |
| NativeAddUPerf |
372.0 us |
253.56 us |
14.326 us |
| ManagedAddUPerf |
351.3 us |
178.05 us |
10.060 us |
| NativeAddSUPerf |
4,865.2 us |
1,176.29 us |
66.463 us |
| ManagedAddSUPerf |
4,959.9 us |
4,223.49 us |
238.635 us |
| NativeMulElementWiseUPerf |
569.9 us |
370.85 us |
20.954 us |
| ManagedMulElementWiseUPerf |
544.0 us |
431.84 us |
24.400 us |
| NativeSumUPerf |
305.4 us |
75.23 us |
4.251 us |
| ManagedSumUPerf |
302.6 us |
90.87 us |
5.134 us |
| NativeSumSqUPerf |
605.7 us |
220.48 us |
12.457 us |
| ManagedSumSqUPerf |
627.1 us |
31.06 us |
1.755 us |
| NativeSumSqDiffUPerf |
308.9 us |
105.91 us |
5.984 us |
| ManagedSumSqDiffUPerf |
310.8 us |
121.70 us |
6.876 us |
| NativeSumAbsUPerf |
307.0 us |
156.36 us |
8.835 us |
| ManagedSumAbsUPerf |
306.9 us |
150.29 us |
8.492 us |
| NativeSumAbsDiffUPerf |
310.7 us |
71.38 us |
4.033 us |
| ManagedSumAbsDiffUPerf |
314.4 us |
105.13 us |
5.940 us |
| NativeMaxAbsUPerf |
305.2 us |
96.67 us |
5.462 us |
| ManagedMaxAbsUPerf |
308.2 us |
106.53 us |
6.019 us |
| NativeMaxAbsDiffUPerf |
308.5 us |
149.43 us |
8.443 us |
| ManagedMaxAbsDiffUPerf |
315.4 us |
39.40 us |
2.226 us |
| NativeDotUPerf |
370.4 us |
189.90 us |
10.730 us |
| ManagedDotUPerf |
499.3 us |
1,135.48 us |
64.157 us |
| NativeDotSUPerf |
4,048.1 us |
4,474.43 us |
252.814 us |
| ManagedDotSUPerf |
5,175.8 us |
18,528.29 us |
1,046.883 us |
| NativeDist2Perf |
439.8 us |
1,530.90 us |
86.499 us |
| ManagedDist2Perf |
418.8 us |
613.55 us |
34.667 us |
| NativeSdcaL1UpdateUPerf |
797.2 us |
1,032.66 us |
58.347 us |
| ManagedSdcaL1UpdateUPerf |
836.8 us |
2,398.49 us |
135.519 us |
| NativeSdcaL1UpdateSUPerf |
14,388.6 us |
5,955.75 us |
336.511 us |
| ManagedSdcaL1UpdateSUPerf |
15,092.4 us |
8,062.04 us |
455.520 us |
- Performance test results after changing
private const int EXP_RANGE = EXP_MAX / 2 to EXP_MAX / 4 in test\Microsoft.ML.CpuMath.PerformanceTests\SsePerformanceTests.cs:
BenchmarkDotNet=v0.10.14, OS=Windows 10.0.17134
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.0.100-alpha1-20180720-2
[Host] : .NET Core 3.0.0-preview1-26710-03 (CoreCLR 4.6.26710.05, CoreFX 4.6.26708.04), 64bit RyuJIT
Toolchain=InProcessToolchain LaunchCount=1 TargetCount=3
WarmupCount=3
| Method |
Mean |
Error |
StdDev |
| NativeAddScalarUPerf |
241.5 us |
528.68 us |
29.872 us |
| ManagedAddScalarUPerf |
254.9 us |
436.65 us |
24.672 us |
| NativeScaleUPerf |
225.6 us |
551.65 us |
31.169 us |
| ManagedScaleUPerf |
232.9 us |
199.31 us |
11.261 us |
| NativeScaleSrcUPerf |
309.6 us |
119.61 us |
6.758 us |
| ManagedScaleSrcUPerf |
443.4 us |
2,016.66 us |
113.945 us |
| NativeScaleAddUPerf |
267.6 us |
1,021.77 us |
57.732 us |
| ManagedScaleAddUPerf |
220.9 us |
174.62 us |
9.866 us |
| NativeAddScaleUPerf |
367.9 us |
270.39 us |
15.277 us |
| ManagedAddScaleUPerf |
362.1 us |
423.48 us |
23.927 us |
| NativeAddScaleSUPerf |
4,687.2 us |
2,677.88 us |
151.305 us |
| ManagedAddScaleSUPerf |
5,702.7 us |
9,787.96 us |
553.038 us |
| NativeAddScaleCopyUPerf |
571.9 us |
731.91 us |
41.354 us |
| ManagedAddScaleCopyUPerf |
520.9 us |
310.43 us |
17.540 us |
| NativeAddUPerf |
371.3 us |
181.94 us |
10.280 us |
| ManagedAddUPerf |
390.0 us |
265.46 us |
14.999 us |
| NativeAddSUPerf |
4,880.2 us |
4,622.75 us |
261.194 us |
| ManagedAddSUPerf |
5,215.2 us |
2,576.32 us |
145.567 us |
| NativeMulElementWiseUPerf |
573.8 us |
211.73 us |
11.963 us |
| ManagedMulElementWiseUPerf |
607.4 us |
692.36 us |
39.119 us |
| NativeSumUPerf |
317.1 us |
221.46 us |
12.513 us |
| ManagedSumUPerf |
311.5 us |
63.41 us |
3.583 us |
| NativeSumSqUPerf |
314.4 us |
20.76 us |
1.173 us |
| ManagedSumSqUPerf |
311.5 us |
43.73 us |
2.471 us |
| NativeSumSqDiffUPerf |
358.2 us |
675.55 us |
38.170 us |
| ManagedSumSqDiffUPerf |
366.4 us |
151.24 us |
8.545 us |
| NativeSumAbsUPerf |
320.3 us |
57.05 us |
3.224 us |
| ManagedSumAbsUPerf |
370.5 us |
810.48 us |
45.793 us |
| NativeSumAbsDiffUPerf |
333.6 us |
237.97 us |
13.446 us |
| ManagedSumAbsDiffUPerf |
330.6 us |
212.14 us |
11.986 us |
| NativeMaxAbsUPerf |
314.5 us |
159.91 us |
9.035 us |
| ManagedMaxAbsUPerf |
322.0 us |
267.66 us |
15.123 us |
| NativeMaxAbsDiffUPerf |
328.3 us |
128.60 us |
7.266 us |
| ManagedMaxAbsDiffUPerf |
372.7 us |
1,107.30 us |
62.564 us |
| NativeDotUPerf |
374.5 us |
273.08 us |
15.430 us |
| ManagedDotUPerf |
383.4 us |
146.85 us |
8.297 us |
| NativeDotSUPerf |
3,792.4 us |
839.94 us |
47.458 us |
| ManagedDotSUPerf |
4,351.2 us |
3,492.81 us |
197.351 us |
| NativeDist2Perf |
377.7 us |
247.30 us |
13.973 us |
| ManagedDist2Perf |
447.2 us |
1,652.67 us |
93.379 us |
| NativeSdcaL1UpdateUPerf |
723.1 us |
833.90 us |
47.117 us |
| ManagedSdcaL1UpdateUPerf |
661.7 us |
412.54 us |
23.309 us |
| NativeSdcaL1UpdateSUPerf |
17,905.3 us |
22,845.78 us |
1,290.829 us |
| ManagedSdcaL1UpdateSUPerf |
15,535.5 us |
4,587.61 us |
259.209 us |
- Performance test results after changing
private const int EXP_RANGE = EXP_MAX / 2 to EXP_MAX / 8 in test\Microsoft.ML.CpuMath.PerformanceTests\SsePerformanceTests.cs:
BenchmarkDotNet=v0.10.14, OS=Windows 10.0.17134
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.0.100-alpha1-20180720-2
[Host] : .NET Core 3.0.0-preview1-26710-03 (CoreCLR 4.6.26710.05, CoreFX 4.6.26708.04), 64bit RyuJIT
Toolchain=InProcessToolchain LaunchCount=1 TargetCount=3
WarmupCount=3
| Method |
Mean |
Error |
StdDev |
| NativeAddScalarUPerf |
229.2 us |
131.203 us |
7.4132 us |
| ManagedAddScalarUPerf |
200.3 us |
105.466 us |
5.9590 us |
| NativeScaleUPerf |
230.7 us |
96.228 us |
5.4371 us |
| ManagedScaleUPerf |
224.0 us |
45.792 us |
2.5873 us |
| NativeScaleSrcUPerf |
296.4 us |
167.351 us |
9.4557 us |
| ManagedScaleSrcUPerf |
292.9 us |
160.873 us |
9.0896 us |
| NativeScaleAddUPerf |
245.2 us |
29.924 us |
1.6908 us |
| ManagedScaleAddUPerf |
210.9 us |
105.411 us |
5.9559 us |
| NativeAddScaleUPerf |
360.0 us |
249.734 us |
14.1104 us |
| ManagedAddScaleUPerf |
345.0 us |
88.350 us |
4.9919 us |
| NativeAddScaleSUPerf |
5,138.2 us |
9,108.634 us |
514.6547 us |
| ManagedAddScaleSUPerf |
7,292.9 us |
8,231.585 us |
465.0998 us |
| NativeAddScaleCopyUPerf |
607.0 us |
176.048 us |
9.9470 us |
| ManagedAddScaleCopyUPerf |
579.1 us |
229.272 us |
12.9543 us |
| NativeAddUPerf |
356.1 us |
150.430 us |
8.4996 us |
| ManagedAddUPerf |
365.8 us |
364.408 us |
20.5898 us |
| NativeAddSUPerf |
4,552.6 us |
2,174.675 us |
122.8732 us |
| ManagedAddSUPerf |
4,857.6 us |
3,113.260 us |
175.9050 us |
| NativeMulElementWiseUPerf |
567.1 us |
60.800 us |
3.4353 us |
| ManagedMulElementWiseUPerf |
524.6 us |
260.834 us |
14.7376 us |
| NativeSumUPerf |
300.6 us |
87.834 us |
4.9628 us |
| ManagedSumUPerf |
300.1 us |
100.474 us |
5.6770 us |
| NativeSumSqUPerf |
301.2 us |
134.832 us |
7.6182 us |
| ManagedSumSqUPerf |
301.0 us |
147.377 us |
8.3271 us |
| NativeSumSqDiffUPerf |
304.2 us |
145.773 us |
8.2364 us |
| ManagedSumSqDiffUPerf |
304.8 us |
176.255 us |
9.9587 us |
| NativeSumAbsUPerf |
301.8 us |
138.377 us |
7.8186 us |
| ManagedSumAbsUPerf |
311.9 us |
140.698 us |
7.9497 us |
| NativeSumAbsDiffUPerf |
309.2 us |
8.792 us |
0.4968 us |
| ManagedSumAbsDiffUPerf |
359.3 us |
647.514 us |
36.5857 us |
| NativeMaxAbsUPerf |
319.1 us |
249.761 us |
14.1120 us |
| ManagedMaxAbsUPerf |
356.6 us |
206.461 us |
11.6655 us |
| NativeMaxAbsDiffUPerf |
383.1 us |
465.175 us |
26.2833 us |
| ManagedMaxAbsDiffUPerf |
364.9 us |
107.476 us |
6.0726 us |
| NativeDotUPerf |
432.3 us |
75.997 us |
4.2940 us |
| ManagedDotUPerf |
429.2 us |
70.930 us |
4.0077 us |
| NativeDotSUPerf |
4,622.5 us |
1,927.120 us |
108.8859 us |
| ManagedDotSUPerf |
4,801.0 us |
1,360.842 us |
76.8901 us |
| NativeDist2Perf |
384.5 us |
95.121 us |
5.3745 us |
| ManagedDist2Perf |
377.8 us |
55.466 us |
3.1340 us |
| NativeSdcaL1UpdateUPerf |
812.2 us |
425.857 us |
24.0617 us |
| ManagedSdcaL1UpdateUPerf |
700.9 us |
381.663 us |
21.5646 us |
| NativeSdcaL1UpdateSUPerf |
14,386.1 us |
1,931.457 us |
109.1309 us |
| ManagedSdcaL1UpdateSUPerf |
17,178.5 us |
17,825.752 us |
1,007.1880 us |
Background
This issue page is opened in response to Intel's following inquiry (@helloguo):
Results (original, after Change 1, after Change 2)
Note: Since these results are obtained with the
ShortRunmode, the mean is an average of only 3 numbers and can encapsulate a sizeable error.private const int EXP_RANGE = EXP_MAX / 2toEXP_MAX / 4intest\Microsoft.ML.CpuMath.PerformanceTests\SsePerformanceTests.cs:private const int EXP_RANGE = EXP_MAX / 2toEXP_MAX / 8intest\Microsoft.ML.CpuMath.PerformanceTests\SsePerformanceTests.cs: