Skip to content

perf: speed up shorebird_tests with template project and Gradle cache#108

Merged
eseidel merged 2 commits into
shorebird/devfrom
es/speed-up-shorebird-tests
Feb 3, 2026
Merged

perf: speed up shorebird_tests with template project and Gradle cache#108
eseidel merged 2 commits into
shorebird/devfrom
es/speed-up-shorebird-tests

Conversation

@eseidel

@eseidel eseidel commented Feb 3, 2026

Copy link
Copy Markdown

Summary

  • Create a template Flutter project once in setUpAll and copy it per test, avoiding repeated flutter create calls
  • Run a warm-up flutter build apk in setUpAll (outside per-test timeout) to prime the Gradle cache
  • Add actions/cache for ~/.gradle so subsequent CI runs start warm
  • Include VERBOSE env var and failure output logging from feat: add verbose mode and failure logging to shorebird_tests #107

This should fix the flaky 6-minute timeout failures on macOS by moving the expensive first-build overhead outside the per-test timeout.

Test plan

  • Verify CI passes on macOS (previously flaky)
  • Verify individual test times are reduced
  • Verify Gradle cache is populated and restored on subsequent runs

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
@eseidel eseidel enabled auto-merge (squash) February 3, 2026 04:22
@eseidel eseidel requested a review from bdero February 3, 2026 04:22
@eseidel

eseidel commented Feb 3, 2026

Copy link
Copy Markdown
Author

We should immediately see this in faster CI times... (with less flake) if we don't then something didn't work.

When the GHA Gradle cache is restored, skip the throwaway APK build
since the cache is already warm. This saves ~4 minutes on cache-hit runs.

@bdero bdero left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor concern, but not blocking.
PR mentions adding a VERBOSE env var, but I don't see it in the diff?

Comment thread packages/shorebird_tests/test/shorebird_tests.dart
@eseidel eseidel merged commit a1d4df6 into shorebird/dev Feb 3, 2026
4 of 5 checks passed
@eseidel eseidel deleted the es/speed-up-shorebird-tests branch February 3, 2026 05:07
bdero pushed a commit that referenced this pull request Feb 3, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Feb 11, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Feb 13, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Feb 21, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Mar 2, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Mar 5, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Mar 10, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Mar 24, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Mar 27, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Apr 20, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Apr 27, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Apr 30, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Jun 1, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
mrnpro pushed a commit to Flutter-Code-Push/flutter that referenced this pull request Jun 16, 2026
…shorebirdtech#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from shorebirdtech#107
bdero pushed a commit that referenced this pull request Jun 22, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Jun 25, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
bdero pushed a commit that referenced this pull request Jul 7, 2026
…#108)

- Create a template Flutter project once in setUpAll and copy it per
  test, avoiding repeated `flutter create` calls
- Run a warm-up `flutter build apk` in setUpAll (outside per-test
  timeout) to prime the Gradle cache
- Add actions/cache for ~/.gradle so subsequent CI runs start warm
- Add VERBOSE env var and failure output logging from #107
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants