Perf results for SDCA Benchmark

[briancylui]

Recent: Netcoreapp 3.0 (3.0 SDK) without my change

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.0.100-alpha1-20180720-2
  [Host] : .NET Core 3.0.0-preview1-26710-03 (CoreCLR 4.6.26710.05, CoreFX 4.6.26708.04), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	390.087 ms	22.6311 ms	26.0620 ms	0.98	8000.0000	1000.0000	-	18291.36 KB
PredictIris	1.162 ms	0.0234 ms	0.0269 ms	0.98	23.4375	11.7188	1.9531	87.21 KB
PredictIrisBatchOf1	1.159 ms	0.0253 ms	0.0270 ms	0.98	23.4375	11.7188	1.9531	87.18 KB
PredictIrisBatchOf2	1.181 ms	0.0311 ms	0.0346 ms	0.98	25.3906	13.6719	1.9531	87.29 KB
PredictIrisBatchOf5	1.242 ms	0.0406 ms	0.0452 ms	0.98	27.3438	13.6719	1.9531	87.59 KB
TrainSentiment	1,730.975 ms	31.0080 ms	29.0049 ms	0.98	172000.0000	47000.0000	5000.0000	733281.14 KB

// * Legends *
  Mean          : Arithmetic mean of all measurements
  Error         : Half of 99.9% confidence interval
  StdDev        : Standard deviation of all measurements
  AccuracyMacro : Macro-average accuracy of the model
  Gen 0         : GC Generation 0 collects per 1k Operations
  Gen 1         : GC Generation 1 collects per 1k Operations
  Gen 2         : GC Generation 2 collects per 1k Operations
  Allocated     : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 ms          : 1 Millisecond (0.001 sec)

Recent: Netcoreapp 3.0 (3.0 SDK) with my change

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.0.100-alpha1-20180720-2
  [Host] : .NET Core 3.0.0-preview1-26710-03 (CoreCLR 4.6.26710.05, CoreFX 4.6.26708.04), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	353.046 ms	79.1765 ms	91.1797 ms	0.98	9000.0000	1000.0000	-	21455.45 KB
PredictIris	1.155 ms	0.0481 ms	0.0553 ms	0.98	25.3906	13.6719	1.9531	86.87 KB
PredictIrisBatchOf1	1.157 ms	0.0345 ms	0.0397 ms	0.98	25.3906	13.6719	1.9531	86.84 KB
PredictIrisBatchOf2	1.140 ms	0.0531 ms	0.0612 ms	0.98	23.4375	11.7188	1.9531	86.94 KB
PredictIrisBatchOf5	1.115 ms	0.0276 ms	0.0307 ms	0.98	25.3906	13.6719	1.9531	87.25 KB
TrainSentiment	1,702.674 ms	39.4201 ms	45.3963 ms	0.98	172000.0000	47000.0000	5000.0000	733326.96 KB

Netcoreapp 3.0 (3.0 SDK)

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.0.100-alpha1-20180720-2
  [Host] : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20  

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	252.784 ms	5.2311 ms	6.0241 ms	0.98	8500.0000	500.0000	-	16417.97 KB
PredictIris	1.123 ms	0.0282 ms	0.0313 ms	0.98	25.3906	13.6719	1.9531	87.31 KB
PredictIrisBatchOf1	1.122 ms	0.0393 ms	0.0453 ms	0.98	25.3906	13.6719	1.9531	87.29 KB
PredictIrisBatchOf2	1.107 ms	0.0180 ms	0.0168 ms	0.98	25.3906	13.6719	1.9531	87.39 KB
PredictIrisBatchOf5	1.139 ms	0.0425 ms	0.0490 ms	0.98	27.3438	13.6719	1

Netcoreapp 3.0 (2.2 SDK)

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host] : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20  

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	286.192 ms	27.6633 ms	30.7477 ms	0.98	9500.0000	500.0000	-	18975.91 KB
PredictIris	1.111 ms	0.0201 ms	0.0178 ms	0.98	25.3906	13.6719	1.9531	85.57 KB
PredictIrisBatchOf1	1.155 ms	0.0178 ms	0.0158 ms	0.98	25.3906	13.6719	1.9531	85.54 KB
PredictIrisBatchOf2	1.182 ms	0.0362 ms	0.0417 ms	0.98	25.3906	13.6719	1.9531	85.64 KB
PredictIrisBatchOf5	1.125 ms	0.0207 ms	0.0173 ms	0.98	25.3906	13.6719	1.9531	85.94 KB

Netcoreapp 2.1 (2.2 SDK)

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host] : .NET Core 2.1.2 (CoreCLR 4.6.26628.05, CoreFX 4.6.26629.01), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20  

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	215.400 ms	8.1704 ms	9.0813 ms	0.98	8500.0000	500.0000	-	16269.34 KB
PredictIris	1.151 ms	0.0350 ms	0.0403 ms	0.98	25.3906	13.6719	1.9531	86.17 KB
PredictIrisBatchOf1	1.114 ms	0.0224 ms	0.0249 ms	0.98	25.3906	13.6719	1.9531	86.14 KB
PredictIrisBatchOf2	1.128 ms	0.0181 ms	0.0161 ms	0.98	25.3906	13.6719	1.9531	86.24 KB
PredictIrisBatchOf5	1.153 ms	0.0398 ms	0.0458 ms	0.98	23.4375	11.7188	1.9531	86.54 KB

Netcoreapp 2.0 (2.2 SDK)

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host] : .NET Core 2.0.9 (CoreCLR 4.6.26614.01, CoreFX 4.6.26614.01), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20  

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	213.557 ms	4.0471 ms	3.7857 ms	0.98	9000.0000	333.3333	-	12865.26 KB
PredictIris	1.249 ms	0.0723 ms	0.0833 ms	0.98	23.4375	11.7188	1.9531	84.68 KB
PredictIrisBatchOf1	1.265 ms	0.0332 ms	0.0369 ms	0.98	23.4375	11.7188	1.9531	84.64 KB
PredictIrisBatchOf2	1.131 ms	0.0185 ms	0.0164 ms	0.98	23.4375	11.7188	1.9531	84.75 KB
PredictIrisBatchOf5	1.133 ms	0.0222 ms	0.0238 ms	0.98	23.4375	11.7188	1.9531	85.04 KB

Netcoreapp 2.0 (2.1 SDK)

BenchmarkDotNet=v0.11.0, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.1.200
  [Host] : .NET Core 2.0.7 (CoreCLR 4.6.26328.01, CoreFX 4.6.26403.03), 64bit RyuJIT

Toolchain=InProcessToolchain  MaxIterationCount=20  

Method	Mean	Error	StdDev	AccuracyMacro	Gen 0	Gen 1	Gen 2	Allocated
TrainIris	259.502 ms	10.0133 ms	11.1298 ms	0.98	9000.0000	500.0000	-	12839.83 KB
PredictIris	1.208 ms	0.0454 ms	0.0505 ms	0.98	23.4375	11.7188	1.9531	84.67 KB
PredictIrisBatchOf1	1.146 ms	0.0285 ms	0.0292 ms	0.98	23.4375	11.7188	1.9531	84.64 KB
PredictIrisBatchOf2	1.153 ms	0.0281 ms	0.0312 ms	0.98	23.4375	11.7188	1.9531	84.75 KB
PredictIrisBatchOf5	1.163 ms	0.0297 ms	0.0330 ms	0.98	23.4375	11.7188	1.9531	85.05 KB

[briancylui]

@eerhardt: Here are the perf results for the SDCA benchmark you may be interested in. Special thanks to @safern who helped me through all the runtime and SDK versions!

[danmoseley]

So your changes take train from 286->253 but unexplained is why 2.1 was 215? Which must be unrelated to your changes?