Skip to content

[ci] cache Gradle distributions on MSBuild test agents - #12244

Merged
jonathanpeppers merged 4 commits into
mainfrom
jonathanpeppers-cache-gradle-wrapper-dists
Jul 28, 2026
Merged

[ci] cache Gradle distributions on MSBuild test agents#12244
jonathanpeppers merged 4 commits into
mainfrom
jonathanpeppers-cache-gradle-wrapper-dists

Conversation

@jonathanpeppers

Copy link
Copy Markdown
Member

Summary

Internal build 3030679 had six AndroidGradleProjectTests failures spread across two macOS test agents (Xamarin.Android.Build.Tests - macOS-9 and macOS-10). Every one reported:

error XAGRDL1000: Executable 'gradlew' not found in project directory
'.../temp/gradle/BuildAppNativeAOT/'

The real failure was upstream, in the Gradle wrapper invoked by AndroidGradleProject.Create():

Running: .../build-tools/gradle/gradlew init --dsl kotlin ... --no-daemon
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 504 for URL:
  https://github.com/gradle/gradle-distributions/releases/download/v9.4.1/gradle-9.4.1-bin.zip
Exit Code: 1

services.gradle.org redirects distribution downloads to GitHub releases, and GitHub returned a transient 504 on those two agents. Every Gradle test on those agents failed; the same test cases passed on all other shards.

Two things made a single transient 504 fatal:

  • cache-gradle.yaml only cached $HOME/.gradle/caches. The Gradle wrapper stores downloaded distributions in $HOME/.gradle/wrapper/dists, which was never cached — so every agent re-downloaded the distribution from GitHub on every run.
  • run-msbuild-tests.yaml did not include cache-gradle.yaml at all, even though AndroidGradleProjectTests runs the Gradle wrapper there. The MSBuild test agents had no Gradle caching whatsoever.

Changes

  • cache-gradle.yaml gains a second Cache@2 task for $HOME/.gradle/wrapper/dists, keyed only on build-tools/gradle/gradle/wrapper/gradle-wrapper.properties since that file alone determines the distribution version. The existing dependency cache keeps its broader key.
  • run-msbuild-tests.yaml now includes cache-gradle.yaml, after setup-test-environment.yaml (which performs the checkout the cache key globs against).

Notes

This is complementary to #12199. That PR removes the gradle init call and the extra 9.3.1 distribution download, but the repo wrapper's own distribution (9.4.1) is still fetched from services.gradle.org on first use, so caching it is still needed.

The Linux Xamarin.Android.Build.Tests jobs are filtered to BuildTest/PackagingTest/XASdkTests/AndroidDependenciesTests and do not run AndroidGradleProjectTests, so they are left alone.

Testing

  • Both templates parse as valid YAML.
  • Caching behavior is only observable on CI; the effect should be a cache Gradle distributions hit on subsequent MSBuild test runs.

Build 3030679 saw six AndroidGradleProjectTests failures across two macOS
agents, all reporting:

    error XAGRDL1000: Executable 'gradlew' not found in project directory
    '.../temp/gradle/BuildAppNativeAOT/'

The real failure was upstream, in the Gradle wrapper invoked by the test
setup:

    Exception in thread "main" java.io.IOException: Server returned HTTP
    response code: 504 for URL:
    https://github.com/gradle/gradle-distributions/releases/download/v9.4.1/gradle-9.4.1-bin.zip

Two things made a single transient HTTP 504 fatal:

* `cache-gradle.yaml` only cached `$HOME/.gradle/caches`. The Gradle wrapper
  stores downloaded distributions in `$HOME/.gradle/wrapper/dists`, which was
  never cached, so every agent re-downloaded the distribution.
* `run-msbuild-tests.yaml` did not include `cache-gradle.yaml` at all, even
  though AndroidGradleProjectTests runs the Gradle wrapper there.

Cache `$HOME/.gradle/wrapper/dists` in a second `Cache@2` task, keyed only on
`gradle-wrapper.properties` since that is what determines the distribution
version, and include the template from the MSBuild test jobs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fa12aa62-fd2f-4e7f-b37c-f666fb6e40cb
Copilot AI review requested due to automatic review settings July 27, 2026 14:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves CI reliability for AndroidGradleProjectTests by ensuring the Gradle wrapper’s downloaded distribution ZIPs are cached on MSBuild test agents, reducing sensitivity to transient download failures from services.gradle.org → GitHub redirects.

Changes:

  • Add a second Cache@2 entry in cache-gradle.yaml to persist $HOME/.gradle/wrapper/dists (Gradle distributions), keyed solely on gradle-wrapper.properties.
  • Include cache-gradle.yaml in run-msbuild-tests.yaml so MSBuild test jobs get the same Gradle caching behavior as other pipelines.
  • Update cache-gradle.yaml’s default xaSourcePath to $(System.DefaultWorkingDirectory) to match the MSBuild test template defaults.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
build-tools/automation/yaml-templates/run-msbuild-tests.yaml Adds the Gradle cache template so MSBuild test agents restore/save Gradle caches before running tests that invoke gradlew.
build-tools/automation/yaml-templates/cache-gradle.yaml Introduces a dedicated cache for Gradle wrapper distributions (~/.gradle/wrapper/dists) keyed by gradle-wrapper.properties.

Accidental regression from rewriting the template: the `xaSourcePath`
parameter default is `$(System.DefaultWorkingDirectory)/android`, not
`$(System.DefaultWorkingDirectory)`.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fa12aa62-fd2f-4e7f-b37c-f666fb6e40cb
@jonathanpeppers

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM

Clean, well-scoped CI change that correctly addresses the root cause from build 3030679 (transient GitHub 504 while the Gradle wrapper downloaded its distribution).

Verified:

  • The dists cache path $HOME/.gradle/wrapper/dists matches the wrapper config — build-tools/gradle/gradle/wrapper/gradle-wrapper.properties sets distributionPath=wrapper/dists with distributionBase=GRADLE_USER_HOME (→ $HOME/.gradle). ✔️
  • Keying the dists cache solely on gradle-wrapper.properties is correct — that file's distributionUrl (currently gradle-9.4.1-bin.zip) fully determines the distribution version. ✔️
  • Separating the two caches (dependencies keyed on build files + Java.Interop hash; distributions keyed only on the wrapper) is the right invalidation model. ✔️
  • Placement in run-msbuild-tests.yaml after setup-test-environment.yaml is correct — that template performs the submodules: recursive checkout the cache key globs and the git log external/Java.Interop step depend on. ✔️
  • Windows pwsh and macOS/Linux bash branches both set GRADLE_DISTS_DIR consistently. ✔️

Notes:

  • CI is still pending on the head commit; this review reflects a static analysis only.
  • One 💡 inline suggestion about monotonic growth of the dists cache across version bumps — informational, not blocking.

Severity counts: ❌ 0 · ⚠️ 0 · 💡 1

Nice, targeted fix with excellent inline documentation explaining the why.

Generated by Android PR Reviewer for #12244 · 80.7 AIC · ⌖ 18.5 AIC · ⊞ 6.9K
Comment /review to run again

Comment thread build-tools/automation/yaml-templates/cache-gradle.yaml Outdated
Copilot AI added 2 commits July 27, 2026 09:29
A partial restore of $HOME/.gradle/wrapper/dists has no value: the wrapper
only uses the distribution named in gradle-wrapper.properties, so restoring
an older one is dead weight. Worse, Cache@2 then saves the restored old
distribution alongside the newly downloaded one under the new key, so the
cache grows by ~150 MB on every Gradle version bump and never shrinks.

Removing restoreKeys means a version bump is a clean cache miss that
re-downloads and stores only the version actually in use.

The Maven dependency cache keeps its restoreKeys, where a partial restore is
genuinely useful because Gradle reuses whichever modules are already present.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fa12aa62-fd2f-4e7f-b37c-f666fb6e40cb
The previous commit added cache-gradle.yaml to run-msbuild-tests.yaml, but
that template is only referenced by azure-pipelines.yaml. The two pipelines
that actually run AndroidGradleProjectTests define their MSBuild test jobs
inline:

  * azure-pipelines-public.yaml   -> dnceng-public "dotnet-android"
  * azure-pipelines-internal.yaml -> dnceng "dotnet-android-internal"

Build 1528252 confirmed the miss: "cache Gradle distributions" appeared only
in the four build jobs, and every "macOS > Tests > MSBuild N" job still had
no Gradle cache step.

Add the template to mac_msbuild_tests and win_msbuild_tests in both
pipelines, and revert the ineffective run-msbuild-tests.yaml change.

These jobs check out to $(System.DefaultWorkingDirectory), so xaSourcePath is
passed explicitly rather than using the template default, which has an
/android suffix for the MAUI integration checkout layout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fa12aa62-fd2f-4e7f-b37c-f666fb6e40cb
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Jul 27, 2026
@jonathanpeppers
jonathanpeppers merged commit fb30bbf into main Jul 28, 2026
44 checks passed
@jonathanpeppers
jonathanpeppers deleted the jonathanpeppers-cache-gradle-wrapper-dists branch July 28, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants