Skip to content

shorebird: consume our published Dart SDK on mac-arm64 (skip rebuilding it)#155

Merged
eseidel merged 3 commits into
shorebird/devfrom
es/consume-shorebird-dart-macos
Jun 5, 2026
Merged

shorebird: consume our published Dart SDK on mac-arm64 (skip rebuilding it)#155
eseidel merged 3 commits into
shorebird/devfrom
es/consume-shorebird-dart-macos

Conversation

@eseidel

@eseidel eseidel commented Jun 2, 2026

Copy link
Copy Markdown

What

Two changes, both scoped to mac-arm64:

  1. DEPS: point the macos-arm64 dart-sdk prebuilt at Shorebird's own published SDK (gs://shorebird-dart-sdk-prebuilt, dep_type: gcs) instead of Google's CIPD.
  2. shorebird/ci/shards/macos.json: drop --no-prebuilt-dart-sdk from the mac-arm64 shard.

Why — build time

The mac-arm64 shard currently builds the Dart SDK from source (--no-prebuilt-dart-sdk + ninja dart_sdk). Dropping that flag turns dart_sdk into copy_dart_sdk — a ~1.5s copy of the prebuilt instead of a full from-source compile. So the engine now:

  • builds against our Dart fork end to end (the prebuilt is our SDK), and
  • skips rebuilding Dart on mac-arm64 → significantly shorter shard time.

The published dart-sdk-darwin-arm64.zip engine artifact is now our SDK (the copy), so downstream consumers are unchanged.

How to evaluate

  • Correctness: an engine build at this revision should produce a working mac-arm64 engine + framework; the dart-sdk-darwin-arm64.zip artifact's revision should be our published dart sha.
  • Speed: compare the mac-arm64 shard wall/CPU time against a baseline build — the from-source dart_sdk step should be gone, replaced by a ~seconds copy.

Verified locally

gn --runtime-mode=release --mac-cpu=arm64 --prebuilt-dart-sdk + ninja dart_sdkcopy_dart_sdk copies our SDK (revision 3bc26d9, bin/dart executable, 3.12.1) in ~1.5s. The full consume path (authenticated private GCS fetch → extract → gn uses our Dart → our Dart compiles the platform kernel) was confirmed end to end.

Sequencing (important)

Merge the _build_engine reader-auth PR first (shorebirdtech/_build_engine#240). Once DEPS points at the private bucket, gclient sync needs the keyless-WIF reader credential; without flutter#240 a mac-arm64 build would hit 401 anonymous at sync. mac-x64/linux/windows are untouched (still Google CIPD / from-source) pending their own published prebuilts.

Depends on: shorebirdtech/_build_engine#240. Reader infra: shorebirdtech/_shorebird#2277 (merged). Broader WIF migration: shorebirdtech/_shorebird#2276.

eseidel added 2 commits June 2, 2026 12:29
…ng it

Point the macos-arm64 dart-sdk prebuilt at Shorebird's own published SDK
(gs://shorebird-dart-sdk-prebuilt, dep_type: gcs) instead of Google's CIPD,
and drop --no-prebuilt-dart-sdk from the mac-arm64 engine shard.

Effect: the mac-arm64 shard stops building the Dart SDK from source (the
dart_sdk gn target was a full from-source compile). With the prebuilt it
becomes copy_dart_sdk -- a ~1.5s copy of our SDK -- so the engine builds
against our Dart fork end to end and the mac-arm64 build is significantly
shorter. The published dart-sdk-darwin-arm64.zip artifact is now our SDK.

Verified locally: gn --mac-cpu=arm64 --prebuilt-dart-sdk + ninja dart_sdk
copies our SDK (revision 3bc26d9, executable) in ~1.5s.

Bot auth to the private bucket is keyless WIF via shorebird-dart-sdk-reader
(shorebirdtech/_build_engine sync.yaml). mac-x64/linux/windows are
unchanged (still build Dart from source / use Google CIPD) pending their
own published prebuilts.
Use the existing dart_sdk_revision var for the macos-arm64 gcs prebuilt
object name instead of hardcoding the sha, so the source clone and the
prebuilt always reference the same fork revision. Bump dart_sdk_revision to
the published sha (3bc26d9); a published prebuilt must exist for whatever
this var points at.

Not dart_revision: that's Google's upstream revision and is load-bearing
for the linux/windows/mac-x64 CIPD entries; our GCS objects are keyed by
our fork's sha.

gclient validate passes; object resolves to
gs://shorebird-dart-sdk-prebuilt/<dart_sdk_revision>/dart-sdk-darwin-arm64.tar.gz.
@eseidel eseidel requested a review from bdero June 4, 2026 00:01
@eseidel eseidel merged commit 01782da into shorebird/dev Jun 5, 2026
3 of 7 checks passed
@eseidel eseidel deleted the es/consume-shorebird-dart-macos branch June 5, 2026 02:20
bdero added a commit that referenced this pull request Jun 5, 2026
Partial revert of #155 for the 3.44.1-rc1 release. With the flag
dropped, the kernel-compiling host steps (const_finder.dart.snapshot,
flutter_patched_sdk dills) resolved the upstream CIPD Dart prebuilt
(dart_revision fc3da898ea1) instead of our fork, stamping kernels with
an SDK hash our VM (0e6acdc83e) refuses to load:
'ConstFinder failure: Can't load Kernel binary: Invalid SDK hash' on
every release-mode build.

The DEPS macos-arm64 GCS prebuilt dep stays (inert with the flag
restored). #155 can return once every kernel-producing step is covered
by a fork prebuilt and the engine build verifies artifact kernel
hashes against the shipped VM.
bdero added a commit that referenced this pull request Jun 5, 2026
Built on dart-sdk 0e6acdc83e3 from flutter_release_ex/3.44.1-rc1, with
--no-prebuilt-dart-sdk restored on the mac-arm64 dart_sdk step (#155
partial revert). Artifact kernel-hash audit matches the rc0 known-good
pattern: zero-hash VM platform dills, const_finder per upstream
prebuilt (long-standing latent quirk, lenient-loaded by the zero-hash
VM). Updater unchanged at a591b7f6b96.
mrnpro pushed a commit to Flutter-Code-Push/flutter that referenced this pull request Jun 16, 2026
…ng it) (shorebirdtech#155)

* shorebird: consume our published Dart SDK on mac-arm64, skip rebuilding it

Point the macos-arm64 dart-sdk prebuilt at Shorebird's own published SDK
(gs://shorebird-dart-sdk-prebuilt, dep_type: gcs) instead of Google's CIPD,
and drop --no-prebuilt-dart-sdk from the mac-arm64 engine shard.

Effect: the mac-arm64 shard stops building the Dart SDK from source (the
dart_sdk gn target was a full from-source compile). With the prebuilt it
becomes copy_dart_sdk -- a ~1.5s copy of our SDK -- so the engine builds
against our Dart fork end to end and the mac-arm64 build is significantly
shorter. The published dart-sdk-darwin-arm64.zip artifact is now our SDK.

Verified locally: gn --mac-cpu=arm64 --prebuilt-dart-sdk + ninja dart_sdk
copies our SDK (revision 3bc26d9, executable) in ~1.5s.

Bot auth to the private bucket is keyless WIF via shorebird-dart-sdk-reader
(shorebirdtech/_build_engine sync.yaml). mac-x64/linux/windows are
unchanged (still build Dart from source / use Google CIPD) pending their
own published prebuilts.

* shorebird: key the dart-sdk prebuilt object on Var('dart_sdk_revision')

Use the existing dart_sdk_revision var for the macos-arm64 gcs prebuilt
object name instead of hardcoding the sha, so the source clone and the
prebuilt always reference the same fork revision. Bump dart_sdk_revision to
the published sha (3bc26d9); a published prebuilt must exist for whatever
this var points at.

Not dart_revision: that's Google's upstream revision and is load-bearing
for the linux/windows/mac-x64 CIPD entries; our GCS objects are keyed by
our fork's sha.

gclient validate passes; object resolves to
gs://shorebird-dart-sdk-prebuilt/<dart_sdk_revision>/dart-sdk-darwin-arm64.tar.gz.
mrnpro pushed a commit to Flutter-Code-Push/flutter that referenced this pull request Jun 16, 2026
Partial revert of shorebirdtech#155 for the 3.44.1-rc1 release. With the flag
dropped, the kernel-compiling host steps (const_finder.dart.snapshot,
flutter_patched_sdk dills) resolved the upstream CIPD Dart prebuilt
(dart_revision fc3da898ea1) instead of our fork, stamping kernels with
an SDK hash our VM (0e6acdc83e) refuses to load:
'ConstFinder failure: Can't load Kernel binary: Invalid SDK hash' on
every release-mode build.

The DEPS macos-arm64 GCS prebuilt dep stays (inert with the flag
restored). shorebirdtech#155 can return once every kernel-producing step is covered
by a fork prebuilt and the engine build verifies artifact kernel
hashes against the shipped VM.
bdero pushed a commit that referenced this pull request Jun 22, 2026
…ng it) (#155)

* shorebird: consume our published Dart SDK on mac-arm64, skip rebuilding it

Point the macos-arm64 dart-sdk prebuilt at Shorebird's own published SDK
(gs://shorebird-dart-sdk-prebuilt, dep_type: gcs) instead of Google's CIPD,
and drop --no-prebuilt-dart-sdk from the mac-arm64 engine shard.

Effect: the mac-arm64 shard stops building the Dart SDK from source (the
dart_sdk gn target was a full from-source compile). With the prebuilt it
becomes copy_dart_sdk -- a ~1.5s copy of our SDK -- so the engine builds
against our Dart fork end to end and the mac-arm64 build is significantly
shorter. The published dart-sdk-darwin-arm64.zip artifact is now our SDK.

Verified locally: gn --mac-cpu=arm64 --prebuilt-dart-sdk + ninja dart_sdk
copies our SDK (revision 3bc26d9, executable) in ~1.5s.

Bot auth to the private bucket is keyless WIF via shorebird-dart-sdk-reader
(shorebirdtech/_build_engine sync.yaml). mac-x64/linux/windows are
unchanged (still build Dart from source / use Google CIPD) pending their
own published prebuilts.

* shorebird: key the dart-sdk prebuilt object on Var('dart_sdk_revision')

Use the existing dart_sdk_revision var for the macos-arm64 gcs prebuilt
object name instead of hardcoding the sha, so the source clone and the
prebuilt always reference the same fork revision. Bump dart_sdk_revision to
the published sha (3bc26d9); a published prebuilt must exist for whatever
this var points at.

Not dart_revision: that's Google's upstream revision and is load-bearing
for the linux/windows/mac-x64 CIPD entries; our GCS objects are keyed by
our fork's sha.

gclient validate passes; object resolves to
gs://shorebird-dart-sdk-prebuilt/<dart_sdk_revision>/dart-sdk-darwin-arm64.tar.gz.
bdero added a commit that referenced this pull request Jun 22, 2026
Partial revert of #155 for the 3.44.1-rc1 release. With the flag
dropped, the kernel-compiling host steps (const_finder.dart.snapshot,
flutter_patched_sdk dills) resolved the upstream CIPD Dart prebuilt
(dart_revision fc3da898ea1) instead of our fork, stamping kernels with
an SDK hash our VM (0e6acdc83e) refuses to load:
'ConstFinder failure: Can't load Kernel binary: Invalid SDK hash' on
every release-mode build.

The DEPS macos-arm64 GCS prebuilt dep stays (inert with the flag
restored). #155 can return once every kernel-producing step is covered
by a fork prebuilt and the engine build verifies artifact kernel
hashes against the shipped VM.
bdero pushed a commit that referenced this pull request Jun 25, 2026
…ng it) (#155)

* shorebird: consume our published Dart SDK on mac-arm64, skip rebuilding it

Point the macos-arm64 dart-sdk prebuilt at Shorebird's own published SDK
(gs://shorebird-dart-sdk-prebuilt, dep_type: gcs) instead of Google's CIPD,
and drop --no-prebuilt-dart-sdk from the mac-arm64 engine shard.

Effect: the mac-arm64 shard stops building the Dart SDK from source (the
dart_sdk gn target was a full from-source compile). With the prebuilt it
becomes copy_dart_sdk -- a ~1.5s copy of our SDK -- so the engine builds
against our Dart fork end to end and the mac-arm64 build is significantly
shorter. The published dart-sdk-darwin-arm64.zip artifact is now our SDK.

Verified locally: gn --mac-cpu=arm64 --prebuilt-dart-sdk + ninja dart_sdk
copies our SDK (revision 3bc26d9, executable) in ~1.5s.

Bot auth to the private bucket is keyless WIF via shorebird-dart-sdk-reader
(shorebirdtech/_build_engine sync.yaml). mac-x64/linux/windows are
unchanged (still build Dart from source / use Google CIPD) pending their
own published prebuilts.

* shorebird: key the dart-sdk prebuilt object on Var('dart_sdk_revision')

Use the existing dart_sdk_revision var for the macos-arm64 gcs prebuilt
object name instead of hardcoding the sha, so the source clone and the
prebuilt always reference the same fork revision. Bump dart_sdk_revision to
the published sha (3bc26d9); a published prebuilt must exist for whatever
this var points at.

Not dart_revision: that's Google's upstream revision and is load-bearing
for the linux/windows/mac-x64 CIPD entries; our GCS objects are keyed by
our fork's sha.

gclient validate passes; object resolves to
gs://shorebird-dart-sdk-prebuilt/<dart_sdk_revision>/dart-sdk-darwin-arm64.tar.gz.
bdero added a commit that referenced this pull request Jun 25, 2026
Partial revert of #155 for the 3.44.1-rc1 release. With the flag
dropped, the kernel-compiling host steps (const_finder.dart.snapshot,
flutter_patched_sdk dills) resolved the upstream CIPD Dart prebuilt
(dart_revision fc3da898ea1) instead of our fork, stamping kernels with
an SDK hash our VM (0e6acdc83e) refuses to load:
'ConstFinder failure: Can't load Kernel binary: Invalid SDK hash' on
every release-mode build.

The DEPS macos-arm64 GCS prebuilt dep stays (inert with the flag
restored). #155 can return once every kernel-producing step is covered
by a fork prebuilt and the engine build verifies artifact kernel
hashes against the shipped VM.
bdero pushed a commit that referenced this pull request Jul 7, 2026
…ng it) (#155)

* shorebird: consume our published Dart SDK on mac-arm64, skip rebuilding it

Point the macos-arm64 dart-sdk prebuilt at Shorebird's own published SDK
(gs://shorebird-dart-sdk-prebuilt, dep_type: gcs) instead of Google's CIPD,
and drop --no-prebuilt-dart-sdk from the mac-arm64 engine shard.

Effect: the mac-arm64 shard stops building the Dart SDK from source (the
dart_sdk gn target was a full from-source compile). With the prebuilt it
becomes copy_dart_sdk -- a ~1.5s copy of our SDK -- so the engine builds
against our Dart fork end to end and the mac-arm64 build is significantly
shorter. The published dart-sdk-darwin-arm64.zip artifact is now our SDK.

Verified locally: gn --mac-cpu=arm64 --prebuilt-dart-sdk + ninja dart_sdk
copies our SDK (revision 3bc26d9, executable) in ~1.5s.

Bot auth to the private bucket is keyless WIF via shorebird-dart-sdk-reader
(shorebirdtech/_build_engine sync.yaml). mac-x64/linux/windows are
unchanged (still build Dart from source / use Google CIPD) pending their
own published prebuilts.

* shorebird: key the dart-sdk prebuilt object on Var('dart_sdk_revision')

Use the existing dart_sdk_revision var for the macos-arm64 gcs prebuilt
object name instead of hardcoding the sha, so the source clone and the
prebuilt always reference the same fork revision. Bump dart_sdk_revision to
the published sha (3bc26d9); a published prebuilt must exist for whatever
this var points at.

Not dart_revision: that's Google's upstream revision and is load-bearing
for the linux/windows/mac-x64 CIPD entries; our GCS objects are keyed by
our fork's sha.

gclient validate passes; object resolves to
gs://shorebird-dart-sdk-prebuilt/<dart_sdk_revision>/dart-sdk-darwin-arm64.tar.gz.
bdero added a commit that referenced this pull request Jul 7, 2026
Partial revert of #155 for the 3.44.1-rc1 release. With the flag
dropped, the kernel-compiling host steps (const_finder.dart.snapshot,
flutter_patched_sdk dills) resolved the upstream CIPD Dart prebuilt
(dart_revision fc3da898ea1) instead of our fork, stamping kernels with
an SDK hash our VM (0e6acdc83e) refuses to load:
'ConstFinder failure: Can't load Kernel binary: Invalid SDK hash' on
every release-mode build.

The DEPS macos-arm64 GCS prebuilt dep stays (inert with the flag
restored). #155 can return once every kernel-producing step is covered
by a fork prebuilt and the engine build verifies artifact kernel
hashes against the shipped VM.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants