Enable sccache for macOS CoreCLR legs - #128675
Conversation
…ch wrapper) Consolidate the -DCMAKE_C/CXX_COMPILER_LAUNCHER injection into eng/native/build-commons.sh so every native component invoked through build_native picks up sccache when USE_SCCACHE=true (CoreCLR, corehost, native libraries, etc.) instead of only the components that hand-rolled the flag themselves. On macOS, CMake wraps PCH includes in -Xarch_<arch> -include<path>, which sccache cannot parse and which also strips bare -include during preprocessing. Use a thin wrapper script that drops -Xarch_<arch> (safe in single-architecture builds) and rewrites -include<path> to -Xclang -include -Xclang <path>, which sccache passes through to the clang frontend correctly. Remove the now-redundant per-component sccache flag injection from src/coreclr/build-runtime.sh; build-commons.sh handles it for all build_native invocations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Widen the gate in setup-sccache.yml and sccache-stats.yml to also include osx + arm64 alongside linux + x64. The CoreCLR_Libraries leg in runtime.yml already lists osx_arm64 and references both templates, so widening the gate activates Mac sccache there without any change to runtime.yml. On linux/freebsd we keep using the linux-musl-x64 sccache NuGet package (also works for arm64 cross-compile on x64 hosts, per dotnet#128065). The NuGet package has no Mac binary, so on macOS we download the official tarball from the mozilla/sccache GitHub release matching the host CPU (uname -m, not archType: the sccache executable must match the agent's CPU). A pinned sha256 guards against tarball tampering and surfaces transient GitHub-side issues. --strip-components=1 places the binary directly in the tools directory so the existing prependpath line still finds it. Bump SCCACHE_CACHE_SIZE to 5120M to match dotnet#128065. The 3584M limit was evicting useful entries; the larger cache keeps hit rates consistent across the now-larger fleet of jobs. Starting narrow on osx_arm64 only: the xarch wrapper in eng/native/sccache-xarch-wrapper.sh assumes single-arch builds, so we widen to osx_x64 / additional Mac legs in follow-ups after validating server startup and cache hits in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two changes together actually activate sccache on the x64 leg: 1. Widen the gate in setup-sccache.yml and sccache-stats.yml to also include (osGroup == osx && archType == x64). Without this, the templates short-circuit and do nothing on the x64 platform. 2. Reference both templates from the osx_x64 CoreCLR_Libraries job in runtime.yml (the one that builds for installer tests). That job previously had no sccache plumbing at all, so widening the gate alone would have been a no-op. The xarch wrapper script is target-arch agnostic — it strips any -Xarch_<arch> (matches both -Xarch_arm64 cross-builds and -Xarch_x86_64 native builds) and its only correctness precondition is single-arch builds, which build-commons.sh already enforces via single-valued CMAKE_OSX_ARCHITECTURES. The setup-sccache.yml binary picker uses uname -m to select the sccache executable, so an x64 host that cross-compiles arm64 still gets the correct x86_64-apple- darwin sccache binary while the cache key continues to differentiate target archs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wire setup-sccache.yml / sccache-stats.yml into the macOS legs in runtime.yml whose buildArgs are clean CoreCLR/libs/host invocations. Mono-flavored legs are deliberately excluded for this PR (see below). Legs added: - Libraries_CheckedCoreCLR (osx_arm64, osx_x64) - NativeAOT (osx_arm64) - NativeAOT_Libraries (osx_arm64) - Installer_Build_And_Test (osx_x64, osx_arm64) For Installer_Build_And_Test the new setup-sccache template is appended to the existing preBuildSteps (which downloads the CoreCLR_Libraries artifact); sccache-stats is inserted as the first postBuildStep so stats are emitted even if later steps fail. Mono legs (AllSubsets_Mono, AllSubsets_Mono_LLVMAOT, the *_RuntimeTests variants, Mono_MiniJIT_LibrariesTests) are intentionally NOT wired. src/mono/mono.proj invokes cmake directly via _MonoCMakeArgs and bypasses eng/native/build-commons.sh, so the launcher injection on this branch does not reach Mono's native build. Wiring sccache into those legs would download the binary and report a misleading hit rate dominated by Mono cache-misses without speeding the dominant work. Extending Mono itself to use sccache (by adding the launcher flags to _MonoCMakeArgs when USE_SCCACHE=true) is left as a follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
the
area-Infrastructure-coreclr
There was a problem hiding this comment.
Pull request overview
This PR wires up sccache for additional macOS CI legs by centralizing the compiler-launcher injection in eng/native/build-commons.sh, adding a macOS-specific argument-rewrite wrapper, and enabling the existing sccache setup/stats templates across the targeted pipeline jobs.
Changes:
- Move
CMAKE_*_COMPILER_LAUNCHERinjection behindUSE_SCCACHE=trueintoeng/native/build-commons.sh(and remove the older injection fromsrc/coreclr/build-runtime.sh). - Add
eng/native/sccache-xarch-wrapper.shto rewrite macOS-Xarch_*/-include...argument patterns before invokingsccache. - Extend pipeline templates and job definitions to download/configure sccache on macOS and print cache stats in the selected CI legs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coreclr/build-runtime.sh | Removes redundant sccache launcher injection now handled in shared native build logic. |
| eng/native/build-commons.sh | Adds centralized USE_SCCACHE handling and selects the macOS wrapper as the launcher on Apple targets. |
| eng/native/sccache-xarch-wrapper.sh | New wrapper script to rewrite Apple-specific compiler flags for sccache compatibility. |
| eng/pipelines/runtime.yml | Enables sccache setup + stats templates on additional macOS-related pipeline legs. |
| eng/pipelines/coreclr/templates/setup-sccache.yml | Adds macOS download path (GitHub release + SHA256 pin) and bumps cache size. |
| eng/pipelines/coreclr/templates/sccache-stats.yml | Expands stats collection gating to include macOS legs. |
| preBuildSteps: | ||
| - template: /eng/pipelines/coreclr/templates/setup-sccache.yml | ||
| postBuildSteps: | ||
| - template: /eng/pipelines/coreclr/templates/sccache-stats.yml |
| # sccache version. On linux/freebsd we use the linux-musl-x64 NuGet | ||
| # package; on macOS we download the matching tag from the mozilla/sccache | ||
| # GitHub release. Keep the version in sync across platforms so all | ||
| # jobs cache against the same compiler-launcher behavior. | ||
| sccacheVersion: '0.15.0' |
|
I ran builds on this PR twice. On the second run it shows sccache being used. Note the installer leg was still running when I collected this data.
|
|
@steveisok Awesome! Sorry, I merged and now you have merge conflicts :) |
|
Workflow state for the Holistic Review Orchestrator. {
"version": 5,
"last_dispatched_commit": "f89193586ddc6cd6f223ee6122f393868a911106",
"last_dispatched_base_ref": "main",
"last_dispatched_base_sha": "54cfeb453970e794043c71448a60a39b4fee3a38",
"last_reviewed_commit": "f89193586ddc6cd6f223ee6122f393868a911106",
"last_reviewed_base_ref": "main",
"last_reviewed_base_sha": "54cfeb453970e794043c71448a60a39b4fee3a38",
"last_recorded_worker_run_id": "29675342180",
"review_attempt_commit": "",
"review_attempt_base_ref": "",
"review_attempt_count": 0,
"max_review_attempts": 5,
"review_history_format": "holistic-review-disclosure-v1",
"review_history": [
{
"commit": "f89193586ddc6cd6f223ee6122f393868a911106",
"review_id": 4730526823
}
]
} |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Holistic Review
Motivation: Solid and real. This extends the existing sccache compile-caching rollout (#128065 for linux/freebsd) to 8 macOS CoreCLR/NativeAOT/Installer CI legs, cutting native rebuild times. macOS native builds are a well-known CI bottleneck, so the value is clear.
Approach: Reasonable and consistent with the prior rollout. The two pipeline templates are gated on osGroup==osx && archType∈(arm64,x64), so adding them to mixed-platform matrix entries is a no-op elsewhere. The sccache-xarch-wrapper.sh cleanly works around sccache's inability to parse Apple's -Xarch_<arch> PCH flags, and the SHA-256-pinned GitHub-release download for macOS is appropriate since no NuGet package exists for that host.
Summary: build-commons.sh always pins a single CMAKE_OSX_ARCHITECTURES, so stripping -Xarch_ is safe). My one substantive concern is scope: relocating the launcher injection from src/coreclr/build-runtime.sh into build_native() silently broadens sccache wrapping to corehost, native-libs, and Mono native builds on all platforms — including existing linux legs — which the PR frames as macOS-only. A maintainer should confirm that broadening is intended. No blocking bugs found.
Detailed Findings
⚠️ Scope — Launcher injection moved to shared build_native()
See the inline comment on eng/native/build-commons.sh. Removing the USE_SCCACHE block from build-runtime.sh and adding it to build_native() means every native build sourcing build-commons.sh (corehost, native libs, Mono) now gets the compiler launcher whenever USE_SCCACHE=true, on all platforms rather than only CoreCLR on the previously-enabled linux legs. Likely a positive expansion, but it should be an explicit, acknowledged decision rather than an implicit side effect of the refactor.
✅ Wrapper correctness — -Xarch_/-include rewriting
The wrapper strips -Xarch_<arch> and rewrites the following glued -include<path> into -Xclang -include -Xclang <path>. The safety argument is verified: lines 94/96, 147/149, 161, 173/175, and 187 of build-commons.sh each set a single-valued CMAKE_OSX_ARCHITECTURES, so there is exactly one target arch per compiler invocation and dropping the -Xarch_ selector is lossless. A non--include argument after -Xarch_ is preserved correctly by the else branch.
✅ Pipeline gating & cache isolation
The if or(and(linux...), and(osx, arm64|x64)) conditions in setup-sccache.yml and sccache-stats.yml correctly preserve the existing linux behavior while adding osx. The cache key already includes osGroup/osSubgroup/archType, so macOS entries stay isolated from linux ones. The 3584M→5120M size bump matches #128065.
💡 Download hardening
The macOS path uses set -euo pipefail, curl -fsSL, and shasum -a 256 -c - against a pinned digest — good. One nit: the pinned sha256 values will need updating in lockstep whenever sccacheVersion bumps; a comment already notes the version must stay in sync, which is sufficient.
Note
This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.
Generated by Holistic Review · 108 AIC · ⌖ 12 AIC · ⊞ 10K
| __sccacheLauncher="$__RepoRootDir/eng/native/sccache-xarch-wrapper.sh" | ||
| fi | ||
| cmakeArgs="-DCMAKE_C_COMPILER_LAUNCHER=$__sccacheLauncher -DCMAKE_CXX_COMPILER_LAUNCHER=$__sccacheLauncher $cmakeArgs" | ||
| fi |
There was a problem hiding this comment.
src/coreclr/build-runtime.sh (removed in this PR), so it applied to the CoreCLR runtime component build. By moving it into build_native() in build-commons.sh, the launcher is now injected for every native build that sources this file — src/native/corehost/build.sh, src/native/libs/build-native.sh, and Mono via src/tests/build.sh — whenever USE_SCCACHE=true is in the environment, on all platforms (not just macOS). On linux/freebsd legs where USE_SCCACHE is already set by setup-sccache.yml, corehost and native-libs compilations will now also be wrapped by sccache, which is a behavioral change not mentioned in the PR description. This is likely a beneficial expansion of caching, but please confirm it's intended and that the corehost/native-libs builds don't emit any flags sccache chokes on (the -Xarch_ handling is only applied on the osx/maccatalyst path). Worth calling out explicitly since the PR title/description scope this as a macOS-only change.
Bring up sccache compile caching for macOS CI legs in dotnet/runtime, mirroring the linux/freebsd rollout from #128065.
Scope
Wires sccache into 8 macOS CI legs that build native code through
eng/native/build-commons.sh:The two template files (
setup-sccache.yml,sccache-stats.yml) are gated onosGroup == osx && archType ∈ (arm64, x64), so adding them to mixed-platform matrix entries is a no-op on non-macOS platforms.How it works
eng/native/build-commons.sh— whenUSE_SCCACHE=trueis set in the environment,build_native()injects-DCMAKE_C_COMPILER_LAUNCHER=…and-DCMAKE_CXX_COMPILER_LAUNCHER=…into the CMake arguments. On linux/freebsd the launcher is baresccache; on macOS/maccatalyst it is the new wrapper described below.eng/native/sccache-xarch-wrapper.sh(new) — On macOS, CMake emits-Xarch_<arch> -include<path>for precompiled headers. Two issues for sccache:-Xarch_<arch>is an Apple-specific flag sccache doesn't recognize.-include<path>(glued together) prevents sccache from seeing the PCH path and hashing it correctly.The wrapper strips
-Xarch_<arch>and rewrites-include<path>into-Xclang -include -Xclang <path>beforeexec-ing sccache. Stripping-Xarch_<arch>is safe becausebuild-commons.shalways setsCMAKE_OSX_ARCHITECTURESto a single value (lines 94/96, 147/149, 161, 173/175, 187), so each compiler invocation has exactly one target arch.eng/pipelines/coreclr/templates/setup-sccache.yml— Downloads the sccache binary per-osGroup: NuGet for linux/freebsd (existing), GitHub release tarball for macOS with SHA-256 pinning. Tarball usestar --strip-components=1. Cache size bumped 3584M → 5120M (matching Enable sccache for arm64 and FreeBSD cross-builds #128065). Cache key includes${osGroup}${osSubgroup}|${archType}|${nameSuffix}|$(Build.BuildId)with rolling restoreKeys so per-platform/arch caches stay isolated.eng/pipelines/coreclr/templates/sccache-stats.yml— Same gate; runssccache --show-statsafter the build.Inspired by
#128065 (sccache for linux/freebsd cross-compile scenarios) — we adopt its cache-size bump and scope-widening pattern.