COMPlus_PartialNGen is not being set since repo consolidation

[Anipik]

we are setting this flag _partialCrossgenFlag for some assemblies here https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk/targets/framework.dependency.targets#L427-L431

Before this change these lines of code was never been executed dotnet/arcade#4911 becuase $(OSGroup) can never be linux in installer. we either set it to unix or osx https://github.com/dotnet/runtime/blob/master/src/installer/Directory.Build.props#L63

The failure due to this flag being set can be found in #32858

the failure message is

Error compiling /root/runtime/artifacts/bin/pkg/netcoreapp5.0/runtime/Linux-Debug-x64/System.ComponentModel.TypeConverter.dll: Cannot compile using the PartialNgen flag because no IBC data was found. (0x8013200C)

I checked the flag value and its being set to 1

This was introduced in dotnet/core-setup#5252
@MichalStrehovsky can you take a look and help me fix this ?

cc @ericstj @dagood @ViktorHofer @safern

[MichalStrehovsky]

This is either because CoreFX stopped embedding IBC data again like in dotnet/corefx#38343 (comment) (when Release builds of CoreFX were actually Debug builds), or because the optimization infra is not generating the IBC data for CoreFX to embed.

Cc @brianrob who own the area.

[Anipik]

@brianrob any pointers here to fix this ?

[Anipik]

cc @ericstj

[brianrob]

It looks like we are publishing IBC data for System.ComponentModel.TypeConverter.dll so this implies that the IBC data isn't being embedded into the IL image before the call to crossgen occurs. Note that this only happens for official builds so if you try this on your dev box you'll never see the IBC data.

I would recommend looking for the call to ibcmerge.exe in build logs and make sure that this is succeeding for System.ComponentModel.TypeConverter.dll.

[ericstj]

I believe @Anipik is seeing the failure on debug build in CI which would explain why the IBC data is missing.

I checked the binaries from a recent official build and confirm that IBC data is present:

It did get an OSGroup value of Linux when it was in core-setup, and the binaries it got from the corefx transport package would have always had IBC data (since they were release binaries)
dotnet/core-setup@0d80fe3

Regardless the fix would be to condition that code. I think we need to match these conditions:

runtime/eng/Tools.props

Line 46 in 2458930

<Import Project="$(RepositoryEngineeringDir)restore\optimizationData.targets" Condition="'$(DotNetBuildFromSource)' != 'true' and '$(EnableNgenOptimization)' == 'true'" />

@brianrob as far as I can tell it might just be a release/debug difference. I can't find anywhere else we condition this to be disabled on official vs not. To be sure, try a release build on Linux and make sure it is successful

Separately: @dagood do we really want library specific configurations to exist in Arcade? Seems to me this should live in the shared framework project. This is leaking repo-specific state into shared tooling: https://github.com/dotnet/arcade/blob/4d9720ccb07cc01068c2f7c19cb2ded64c44b1a1/src/Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk/targets/framework.dependency.targets#L427-L431

[brianrob]

Historically, the conditioning has been on whether or not ibcmerge.exe is present, which only happens on release official builds since it is not publicly available.

[Anipik]

@ericstj I tried with the release flag still the same error

Error compiling /runtime/artifacts/bin/pkg/netcoreapp5.0/runtime/Linux-Release-x64/System.ComponentModel.TypeConverter.dll: Cannot compile using the PartialNgen flag because no IBC data was found. (0x8013200C)

[ericstj]

I see, we are probably missing another condition then as @brianrob suggests.

[ericstj]

Found it: https://github.com/dotnet/arcade/blob/f6121fcaf62777bc2288f7f0917b48a130a0174e/src/Microsoft.DotNet.Arcade.Sdk/tools/OptimizationData.targets#L59

[ericstj]

@brianrob I don't like copying a bunch of this conditional logic or running the targets. Is there a way for these installer targets to "observe" that IBCMerge was run and only then specify the param? Or for crossgen to ignore the option if the binary doesn't have IBC merge data?