Regression in R2R compilation time

[sebastienros]

Linux and Windows, PlaintextPlatform benchmark in this example, Single file, Trimmed and Self-contained

Binlogs before/after show

msbuild.binlog.zip

[EgorBo]

@sebastienros is there a commit range? (I don't see an ability to view it on the e.g. Single File tab)

[EgorBo]

Likely, ffb2578...e6c1a49

[sebastienros]

That's what I have yes, we could probably go shorter by trying builds from within the range if you don't find anything

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/plaintext.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --scenario plaintext --profile intel-lin-app --profile intel-load-load --application.buildArguments "/p:PublishReadyToRun=true /p:PublishSingleFile=true" --application.framework net9.0 --application.options.collectCounters true --load.options.reuseBuild true --application.aspNetCoreVersion 9.0.0-preview.4.24205.2 --application.runtimeVersion 9.0.0-preview.4.24204.3 --application.sdkVersion 9.0.100-preview.4.24208.11

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/plaintext.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --scenario plaintext --profile intel-lin-app --profile intel-load-load --application.buildArguments "/p:PublishReadyToRun=true /p:PublishSingleFile=true" --application.framework net9.0 --application.options.collectCounters true --load.options.reuseBuild true --application.aspNetCoreVersion 9.0.0-preview.4.24208.6 --application.runtimeVersion 9.0.0-preview.4.24209.8 --application.sdkVersion 9.0.100-preview.4.24211.4

[EgorBo]

TP regressed on NativeAOT too:

[sebastienros]

Haven't had time to file this one, are you saying it's related? I wouldn't think so

[EgorBo]

Haven't had time to file this one, are you saying it's related? I wouldn't think so

Dates match

[EgorBo]

The thing is - when something goes wrong for AOT'd code - JIT can just re-jit it, while NAOT cannot, so it explains why TP only got hurt for NAOT.

[sebastienros]

Here are the commands for the AOT one:

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/goldilocks.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/steadystate.profile.yml --scenario todosapipublishaot --profile intel-win-app --profile intel-lin-load --profile amd-lin2-db --application.packageReferences Microsoft.DotNet.ILCompiler=9.0.0-preview.4.24204.3 --application.environmentVariables DOTNET_GCDynamicAdaptationMode=1 --application.framework net9.0 --application.options.collectCounters true --load.options.reuseBuild true --application.aspNetCoreVersion 9.0.0-preview.4.24205.2 --application.runtimeVersion 9.0.0-preview.4.24204.3 --application.sdkVersion 9.0.100-preview.4.24208.11

crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/goldilocks.benchmarks.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/steadystate.profile.yml --scenario todosapipublishaot --profile intel-win-app --profile intel-lin-load --profile amd-lin2-db --application.packageReferences Microsoft.DotNet.ILCompiler=9.0.0-preview.4.24209.5 --application.environmentVariables DOTNET_GCDynamicAdaptationMode=1 --application.framework net9.0 --application.options.collectCounters true --load.options.reuseBuild true --application.aspNetCoreVersion 9.0.0-preview.4.24208.6 --application.runtimeVersion 9.0.0-preview.4.24209.5 --application.sdkVersion 9.0.100-preview.4.24209.32

[EgorBo]

The main suspect is #99982

[sebastienros]

Minimal changeset identified for the nativeaot benchmark:

a8c2a5e...404b286

[EgorBo]

Oh, so #100728 then?

[sebastienros]

nativeaot may be special, not sure if moving the runtime version without the corresponding sdk might change the outcome. But at least I set the framework reference and Microsoft.DotNet.ILCompiler to the same runtime. On 9.0.0-preview.4.24208.9+a8c2a5e5bbc2 we get 398K RPS and on 9.0.0-preview.4.24208.10+404b286b2309 only 178K RPS.