Native AOT Trimming & Perf Improvements in .NET 8

[agocke]

For .NET 8 we want to continue improving Native AOT performance, particularly in the areas that Native AOT excels at, namely fast startup, small binaries, and smaller working set. Native AOT shares the JIT and GC with CoreCLR so there's a lot of overlap with CoreCLR in these areas.

For .NET 8 we'll target basic APIs that are commonly used by customers and touch multiple areas of the stack (web/http stack and base framework libraries). There are two toy apps that we can use as benchmarks.

1. Albums app
   This is an ASP.NET Core Minimal APIs app that simply returns lists of in-memory objects, JSON serialized.):
   i. Disk size: 10 MB
   ii. Startup time: <50 ms
   iii. Working set (before load): <50 MB
   iv. Working set (at load): <50 MB
   v. Throughput: Within 5% of default CoreCLR RPS on Citrine perf environment ("default" here means compared to the default configuration of a CoreCLR-based deployment of the app, e.g. including tiered JIT)

2. Todo API app
   This app is more representative of something “real” and thus uses much more of the framework surface area:
   i. Disk size: 20 MB
   ii. Startup time: <150 ms
   iii. Working set (before load): <60MB
   iv. Working set (at load): <60MB
   v. Throughput: Within 5% of default CoreCLR RPS on Citrine perf environment
   vi. Container smallest possible constraint: <100MB

The ASP.NET work here is at dotnet/aspnetcore#45910

Planned perf work in Native AOT includes

Size:

• Reduce impact of relocations Investigate size of .rela.dyn section on Linux #76657
  • Reloc in EmbeddedPointerIndirectionNode Convert pointers in a couple tables to relative #78801
  • Relocs in GCStaticsNode Remove pointer relocs from GC statics #78794
  • Reloc in ThreadStaticsNode
  • Reloc in NonGCStaticsNode Place non-GC statics in the dehydratable section #78795
  • Relocs in ModuleInitializerListNode Convert pointers in a couple tables to relative #78801
  • Relocs in PInvokeMethodFixup/PInvokeModuleFixup
  • Frozen strings and frozen objects - Place frozen objects into dehydrated section #78688
  • InterfaceDispatchCellNode - Place dispatch cells in dehydrated section #78748
• Shrink metadata overhead for user code if possible
• Componentize fixed overheads in the lowest portions of the framework: Bringing down the size of Hello World under 2 MB #80165

Add benchmarking:

These items don't have improvements planned, but need to be tracked for regressions:

• Startup perf
• Build time (ILLink and Native AOT)
• Throughput

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

For .NET 8 we want to continue improving Native AOT performance, particularly in the areas that Native AOT excels at, namely fast startup, small binaries, and smaller working set. Native AOT shares the JIT and GC with CoreCLR so there's a lot of overlap with CoreCLR in these areas.

For .NET 8 we'll target basic APIs that are commonly used by customers and touch multiple areas of the stack (web/http stack and base framework libraries). There are two toy apps that we can use as benchmarks.

1. Albums app
   This is an ASP.NET Core Minimal APIs app that simply returns lists of in-memory objects, JSON serialized.):
   i. Disk size: 10 MB
   ii. Startup time: <50 ms
   iii. Working set (before load): <50 MB
   iv. Working set (at load): <50 MB
   v. Throughput: >5 million RPS (Citrine) (TBD)

2. Todo API app
   This app is more representative of something “real” and thus uses much more of the framework surface area:
   i. Disk size: 20 MB
   ii. Startup time: <150 ms
   iii. Working set (before load): <60MB
   iv. Working set (at load): <60MB
   v. Throughput: >5 million RPS (Citrine) (TBD)
   vi. Container smallest possible constraint: <100MB

Author:	agocke
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr
Milestone:	-

[LLT21]

Native AOT performance is already excellent today. I have been testing it for over a year and it has been fast and stable in this timeframe. To track the performance, I made an early NativeAOT implementation at TechEmpower Benchmarks (named appmpower). One thing I noticed while upgrading last week to .NET 7 at the same time as the asp.net core middleware implementation in the TechEmpower Benchmarks is that the non AOT implementation of the standard middleware plaintext test has improved from 5,703,769 responses per second in .NET 6 to 5,954,942 in .NET 7. My AOT middleware implementation has remained identical (5,024,661 vs 5,026,387 responses per second). I did some testing locally and the whole difference seems due to the <TieredPGO>true</TieredPGO> project option with according parameters in the docker file of the standard asp.net core middleware implementation. I guess this is a benefit of generating IL, so not sure whether AOT could still improve in this respect or whether other options exist to mimic this performance advantage of TieredPGO in AOT compilation. In any case, the TieredPGO implementation in .NET 7 has created some excellent performance improvement for non AOT projects.

[jkotas]

whether other options exist to mimic this performance advantage of TieredPGO in AOT compilation

The AOT code generation can benefit from profile data in similar way as TieredPGO. It requires the profile data to be collected ahead of time and passed to the AOT toolchain. The support is partially implemented - for example, see

runtime/src/coreclr/tools/aot/ILCompiler/ILCompilerRootCommand.cs

Lines 29 to 30 in 1412ee7

	public Option<string[]> MibcFilePaths { get; } =
	new(new[] { "--mibc", "-m" }, Array.Empty<string>, "Mibc file(s) for profile guided optimization");

. However, the end-to-end experience for this is incomplete and undocumented.

Tagging subscribers to 'linkable-framework': @eerhardt, @vitek-karas, @LakshanF, @sbomer, @joperezr
See info in area-owners.md if you want to be subscribed.

Issue Details

---

For .NET 8 we want to continue improving Native AOT performance, particularly in the areas that Native AOT excels at, namely fast startup, small binaries, and smaller working set. Native AOT shares the JIT and GC with CoreCLR so there's a lot of overlap with CoreCLR in these areas.

For .NET 8 we'll target basic APIs that are commonly used by customers and touch multiple areas of the stack (web/http stack and base framework libraries). There are two toy apps that we can use as benchmarks.

1. Albums app
   This is an ASP.NET Core Minimal APIs app that simply returns lists of in-memory objects, JSON serialized.):
   i. Disk size: 10 MB
   ii. Startup time: <50 ms
   iii. Working set (before load): <50 MB
   iv. Working set (at load): <50 MB
   v. Throughput: >5 million RPS (Citrine) (TBD)

2. Todo API app
   This app is more representative of something “real” and thus uses much more of the framework surface area:
   i. Disk size: 20 MB
   ii. Startup time: <150 ms
   iii. Working set (before load): <60MB
   iv. Working set (at load): <60MB
   v. Throughput: >5 million RPS (Citrine) (TBD)
   vi. Container smallest possible constraint: <100MB

Planned perf work in Native AOT includes

Size:

• Reduce impact of relocations Investigate size of .rela.dyn section on Linux #76657
  • Reloc in EmbeddedPointerIndirectionNode Convert pointers in a couple tables to relative #78801
  • Relocs in GCStaticsNode Remove pointer relocs from GC statics #78794
  • Reloc in ThreadStaticsNode
  • Reloc in NonGCStaticsNode Place non-GC statics in the dehydratable section #78795
  • Relocs in ModuleInitializerListNode Convert pointers in a couple tables to relative #78801
  • Relocs in PInvokeMethodFixup/PInvokeModuleFixup
  • Frozen strings and frozen objects - Place frozen objects into dehydrated section #78688
  • InterfaceDispatchCellNode - Place dispatch cells in dehydrated section #78748
• Shrink metadata overhead for user code if possible

Startup perf:

• Start measuring startup perf (no improvement plans at the moment)

Build time:

• Start measuring AOT and linker compile times to check for regressions (no plans for improvements at the moment)

Author:	agocke
Assignees:	-
Labels:	tenet-performance, linkable-framework, area-NativeAOT-coreclr
Milestone:	8.0.0