MacOS arm64 builds are failing on M1 devices

[mangod9]

Description

After the latest change of using crossgen2 to build System.Private.CoreLib.dll. The issue seems to be related to using the right bitness of crossgen2 while compiling. Cross compilation from Intel machines should continue to work

Configuration

Build runtime on an M1 device with ./build.sh -subset clr -arch arm64 fails with

  Unhandled exception. System.DllNotFoundException: Unable to load shared library 'jitinterface_x64' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libjitinterface_x64, 1): image not found
     at System.Runtime.InteropServices.NativeLibrary.LoadByName(String libraryName, QCallAssembly callingAssembly, Boolean hasDllImportSearchPathFlag, UInt32 dllImportSearchPathFlag, Boolean throwOnError)
     at System.Runtime.InteropServices.NativeLibrary.LoadLibraryByName(String libraryName, Assembly assembly, Nullable`1 searchPath, Boolean throwOnError)
     at System.Runtime.InteropServices.NativeLibrary.Load(String libraryName, Assembly assembly, Nullable`1 searchPath)
     at Internal.JitInterface.JitConfigProvider.<>c__DisplayClass5_0.<Initialize>b__0(String libName, Assembly assembly, Nullable`1 searchPath) in /Users/usr/git/runtime/src/coreclr/tools/Common/JitInterface/JitConfigProvider.cs:line 61
     at System.Runtime.InteropServices.NativeLibrary.LoadLibraryCallbackStub(String libraryName, Assembly assembly, Boolean hasDllImportSearchPathFlags, UInt32 dllImportSearchPathFlags)
     at Internal.JitInterface.CorInfoImpl.GetJitHost(IntPtr configProvider)
     at Internal.JitInterface.CorInfoImpl.Startup() in /Users/usr/git/runtime/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 116
     at Internal.JitInterface.JitConfigProvider.Initialize(TargetDetails target, IEnumerable`1 jitFlags, IEnumerable`1 parameters, String jitPath) in /Users/usr/git/runtime/src/coreclr/tools/Common/JitInterface/JitConfigProvider.cs:line 69
     at ILCompiler.ReadyToRunCodegenCompilationBuilder.ToCompilation() in /Users/usr/git/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilationBuilder.cs:line 254
     at ILCompiler.Program.Run(String[] args) in /Users/usr/git/runtime/src/coreclr/tools/aot/crossgen2/Program.cs:line 602
     at ILCompiler.Program.Main(String[] args) in /Users/usr/git/runtime/src/coreclr/tools/aot/crossgen2/Program.cs:line 754

Regression?

Building on M1 is new, but it was working before the switch to crossgen2. using arch -x86_64 ./build.sh -subset clr -arch arm64 doesnt help either. Switching back to using cg1 is a workaround for now.

Other information

This is most likely since the default .dotnet used for build is x64, but the build logic assumes its arm64 since that is the host its running on. Might need to special case to always assume x64 during builds.

[mangod9]

Tagging to #43313

[mangod9]

Hoping this should be fixed by #47245

[sdmaclea]

With macOS 11.3 Beta 6, the preview3 SDK is stable enough to build.

So I updated global.json to reference preview3.

diff --git a/global.json b/global.json
index f2672dc65e4..136090228cd 100644
--- a/global.json
+++ b/global.json
@@ -1,11 +1,11 @@
 {
   "sdk": {
-    "version": "6.0.100-preview.2.21155.3",
+    "version": "6.0.100-preview.3.21202.5",
     "allowPrerelease": true,
     "rollForward": "major"
   },
   "tools": {
-    "dotnet": "6.0.100-preview.2.21155.3"
+    "dotnet": "6.0.100-preview.3.21202.5"
   },
   "native-tools": {
     "cmake": "3.16.4",

Then built using

./build.sh clr -arch arm64 -c Release -runtimeconfiguration checked
./build.sh libs -arch arm64 -c Release -runtimeconfiguration checked

I kept the commands separate because I had a crash the first build attempt.

crossgen2 appears to be present and built for arm64

The big difference is probably macOS 11.3 Beta 6. I didn't try preview2. It might work too. It just wasn't installed on my machine.

[vcsjones]

For what it's worth, I still run in to this building main.

EXEC : error : Unable to load shared library 'jitinterface_x64' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libjitinterface_x64, 1): image not found [/Users/kjones/Projects/dotnet/runtime/src/coreclr/crossgen-corelib.proj]

and building with: ./build.sh -s clr+libs -rc release -arch arm64

I did notice that the build is using the .NET 6 preview that is installed locally for me, it doesn't restore one in to .dotnet. Since my locally installed previews are x64, perhaps that is related.

[vcsjones]

Ah, indeed doing use_installed_dotnet_cli=false ./build.sh -s clr+libs -rc release -arch arm64 did get me a complete build. So it seems that if an installed dotnet CLI is present, but the wrong architecture, the error will occur. I presume the error would go away if I removed my x64 installs and installed the ARM64 previews.

[sdmaclea]

This is fixed when the global runtime is osx-arm64. Closing