Vendor the SDK's SwiftPM-only deps as a pre-built static binary - #69
Vendor the SDK's SwiftPM-only deps as a pre-built static binary#69peachbits wants to merge 7 commits into
Conversation
|
✅ Validation passed. A full
This is the same artifact set the manual spike built — which builds + runs a ZEC wallet on the simulator. |
af4b872 to
e5bcf7e
Compare
|
The branch has been amended ( |
|
✅ GUI integration validation complete (Edge app, static frameworks, against the regenerated
One integration note for Edge: after |
e5bcf7e to
128a823
Compare
|
🔎 Major correction + second commit ( What we found (while validating SDK 2.6.0-alpha.6 against develop in Release config): the podspec's The fix: deps now ship as Also in the commit: SDK bump to 2.6.0-alpha.6 (bridge needed zero changes — API surface is source-compatible; new Voting/PIR module + expanded FFI compile in), per-run DerivedData cleaning in Caveats: (1) full runtime wallet validation of the isolated build is blocked on this workstation — SentinelOne EDR deletes the ad-hoc-signed Release simulator binary (from build dirs and even sim containers); needs an S1 exclusion or a CI/second machine. Boot-level launch did succeed before the agent intervened. (2) Shipping decision still open: with both this package (grpc 1.27 static) and piratechain (grpc 1.8 pods) in one app, lazy archive semantics mean symbol resolution is link-order-dependent — the clean paths are prelink+symbol-hiding for zcash, or upgrading piratechain onto the same vendored-deps architecture. |
|
History rewrite: re-split into two clean commits (tip The previous split leaked mechanism into the version bump: commit 2 carried the xcframework/
Tip tree vs the previous tip differs only by two stale header comments in |
3b0ca95 to
d237b00
Compare
|
📌 Open item found during review discussion: react-native-piratechain coordination is required before this ships. piratechain still declares the Resolution options (one copy of the deps must exist, exactly one force_load):
Until then, the current state (pirate on 1.8 pods) is the tested configuration. |
The modern ZcashLightClientKit SDK pulls grpc-swift (1.24+), SwiftNIO, SwiftProtobuf and SQLite.swift via SwiftPM only -- grpc-swift's CocoaPods releases stopped at 1.8.0, so it can no longer be a CocoaPods dependency. Consuming the SDK via spm_dependency would force the entire host app onto dynamic frameworks. Instead, scripts/buildVendoredDeps.ts pre-builds those leaf dependencies in Release per platform (device arm64; simulator arm64+x86_64, matching the libzcashlc baseline), packages them as ios/vendored/libZcashDeps.xcframework, and harvests their Swift/C modules -- generated by 'npm run update-sources' and shipped in the npm tarball. Dependency versions are pinned to the SDK's Package.resolved (-disableAutomaticPackageResolution), and DerivedData is cleaned per run (stale precompiled modules of the libzcashlc header poison SDK bumps). The ZcashLightClientKit source keeps compiling in-pod exactly as before, and the host app stays on static frameworks. The archive ships as an XCFramework consumed via vendored_frameworks because that is a real link input: CocoaPods places it on the app link line, where the linker pulls members on demand to satisfy the in-pod SDK source's references. (pod_target_xcconfig OTHER_LDFLAGS is NOT viable for this: a static-framework pod's Libtool step ignores OTHER_LDFLAGS, and pod-target settings never propagate to the app link -- flags there are silently dead.) The binary .swiftmodule format only loads under the exact Swift compiler that produced it (library-evolution .swiftinterface is unavailable: swift-nio doesn't compile under evolution), so the build stamps ios/vendored/swift-version.txt and the podspec fails fast with rebuild instructions when the consuming Xcode doesn't match. Note: sqlite3 is not part of this binary -- SQLite.swift on Apple platforms links the system sqlite3 module, so there is nothing to exclude. Edge's pre-existing duplicate-sqlite3 ld warnings come from libzcashlc vs libpiratelc (both Rust cores embed sqlite3) and are unrelated to this change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First hardfork-era prerelease: new Voting/PIR module, expanded libzcashlc FFI, updated checkpoints. Pin changes only -- the bridge needs no changes (the SDK's bridge-facing API surface is source-compatible with 2.5.2) and the dependency graph pins are identical, so the vendored deps binary is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Points update-sources at the Swift SDK's 2.7.0-rc.4 tag and bumps the Android SDK (and the incubator) to the same release - the first published line carrying Ironwood. The Swift side is pinned by commit rather than tag name, so the checkout is immutable even if the tag is ever moved. The FFI is upstream's own release asset now, so there is no Edge-hosted xcframework to build or host. update-sources does not repeat that asset's URL: it reads the url/checksum pair out of the pinned checkout's own binaryTarget, which is the one place upstream writes down which FFI build matches this source - so a source/binary mismatch is unrepresentable. LIBZCASHLC_XCFRAMEWORK_SHA256 remains as the reviewed copy of that checksum: the build stops if the two disagree, so bumping the SDK pin has to change native code deliberately rather than silently, and the download is verified against it before anything is unpacked. Also carries the one other build input Ironwood needs: TreeState.ironwoodTree (field 7) in the bundled lightwalletd proto. update-checkpoints writes the GetTreeState response verbatim, so a field missing from our proto is never decoded and silently never written - every checkpoint generated after activation would have shipped without Ironwood tree state. Pre-activation output is unchanged, so existing checkpoints need no regeneration.
Reports the Ironwood pool alongside transparent/sapling/orchard on both platforms, and folds it into the deprecated summed fields so funds do not appear to vanish from them once value starts crossing into Ironwood.
Detection plus the v1 sweep, identical on both platforms. Synchronizer.proposeOrchardToIronwoodMigration builds the sweep: the SDK spends every Orchard note to the account's own address with the fee chosen so no Orchard change remains, leaving Sapling and transparent funds untouched, and it executes through the ordinary createTransfer pipeline. Neither SDK reports a payment value on the proposal, so both platforms quote the crossing amount the same way - the Orchard balance minus that fee - and fail rather than quote a figure they cannot source. Tools.getIronwoodActivationHeight is stateless and served from consensus constants (ZIP 258), which neither SDK exposes, so it answers on both platforms without an SDK call. There is deliberately no migration state to poll: the app decides whether to offer a sweep from the activation height, sync state and Orchard balance, and broadcasting the sweep spends those notes, which withdraws the offer on its own.
The synchronizer's event stream only carries transactions newly found in scanned blocks or newly mined, and the native module drops every event until JavaScript attaches a listener. A transaction that settled while nothing was listening - mined while the app was closed, or during a failed sync - is therefore neither newly found nor newly mined on the next launch, so it was never reported again: it stayed at height 0, "pending", forever. Synchronizer.subscribe now asks native for the current transaction set once its listeners are attached. Driving it from JavaScript is what makes the delivery reliable: emitting during initialize would race the app's own subscribe call and lose, because hasListeners is still false there. Both platforms are affected. iOS never dumps the existing set outside its restart path. Android's collector does deliver the full list, but on its first emission during initialize, which is the same race - one it usually wins, which is not the same as being correct. Re-sending known transactions is harmless: the app updates only those whose height or amount actually changed, and its new-transaction notification path is not reached for transactions it already has.
The checkpoint shipped in 0.13.0 is above the NU6.3 activation height (3428143) but was generated with the old lightwalletd proto, which lacked TreeState.ironwoodTree, so the Ironwood commitment tree was silently dropped. Regenerated with the fixed proto; the sapling and orchard trees are unchanged.
2e7be00 to
0bc93ba
Compare
|
Verified this branch end-to-end against a local build of the full Ironwood stack (no publishing, no pin repointing). Stack under test (all built locally, uncommitted
Bundling verified at the binary layer, not just node_modules: the installed Real on-chain migration executed (edge-funds / My Zcash, mainnet, NU6.3 active at tip 3,432,352 vs activation 3,428,143):
The explorer was reached through the app's own Show in Explorer link on the tx detail scene, which correctly opened blockchair; that link works. One blocking integration finding (details in the gui PR): |
2 participants
Summary
The modern ZcashLightClientKit SDK pulls grpc-swift (1.24+), SwiftNIO, SwiftProtobuf and SQLite.swift via SwiftPM only — grpc-swift's CocoaPods releases stopped at 1.8.0 — so those can no longer be CocoaPods
dependencys. Consuming the SDK through React Native'sspm_dependencywould force the entire host app onto dynamic frameworks (and, on RN 0.85, an open-ended community-pod link cascade).This vendors just those leaf dependencies as a pre-built static binary instead, so the host app stays on static frameworks and the ZcashLightClientKit source keeps compiling in-pod exactly as it does today.
What changed
scripts/buildVendoredDeps.ts(new): builds the SDK's SwiftPM dependency graph (excluding ZcashLightClientKit itself) in Release config for device (arm64) and simulator (arm64 + x86_64 — same arch baseline aslibzcashlc.xcframework, so Intel Macs still build), merging raw per-target objects per arch andlipo-ing the simulator slices. Harvests the Swift.swiftmodules + Cmodule.modulemaps the in-pod SDK source imports. Dependency versions are pinned to the SDK's ownPackage.resolved(-disableAutomaticPackageResolutionhard-fails on drift). Driven bynpm run update-sources; output is gitignored and shipped in the npm tarball (same model aslibzcashlc.xcframework).react-native-zcash.podspec: drops thegRPC-Swift/SQLite.swiftpod dependencies; vendors the per-platform deps binary via SDK-conditional-force_loadplus the module search paths. Bridge, SDK-source vendoring, checkpoints, andlibzcashlcare unchanged..swiftmodulefiles only load under the exact Swift compiler that produced them (the stable alternative — library-evolution.swiftinterface— is unavailable because swift-nio doesn't compile under evolution). The build stampsios/vendored/swift-version.txt; the podspec fails fast at pod-install time with actionable instructions (npm run update-sourcesrebuilds locally, ~10–15 min) when the consuming Xcode doesn't match. This does mean the published tarball's prebuilt deps are coupled to the publishing Xcode — the guard turns that from a cryptic compile error into a one-command fix..gitignore: treatsios/vendored/as generated.Tradeoff: this deliberately gives up Xcode-agnosticism
Today's package is compiler-agnostic by construction: every Swift line (bridge + vendored SDK source + the gRPC/SQLite pods) compiles from source on the consumer's machine, and the only prebuilt binary (
libzcashlc) sits behind a C interface, which is ABI-stable across all toolchains — any Xcode works. This PR introduces the package's first prebuilt Swift binary, and consuming prebuilt Swift means reading.swiftmodulefiles, a compiler-version-locked format (Swift's machine-code ABI is stable; its module format is not). The portable alternative (library-evolution.swiftinterface) is off the table by swift-nio upstream policy (apple/swift-nio#2470, #2897 — closed "not planned"), so the lock is permanent while NIO is in the SDK's dependency graph.The rejected alternative that would preserve Xcode-agnosticism: vendor the grpc/NIO/SwiftProtobuf source into the pod the way the SDK source is vendored. Rejected because it means hand-maintaining a CocoaPods replica of SwiftPM's ~35-target build graph (including the CNIOBoringSSL C target and per-target module maps) and re-deriving it on every SDK bump. The prebuilt binary trades that permanent maintenance burden for: coupling to the publishing Xcode, mitigated by the stamp/fail-fast guard and a one-command local rebuild. If sustained mixed-Xcode pain ever exceeds that maintenance cost, the revisit levers are (a) source-vendoring the deps, or (b) per-compiler prebuilt variants as GitHub release assets (same download pattern as
libzcashlc).sqlite3 note (correcting an earlier claim)
An earlier revision of this PR claimed it deduplicated sqlite3 symbols. That was a misdiagnosis: SQLite.swift on Apple platforms has no C-shim target (it imports the system
sqlite3clang module), so the deps binary never contained sqlite3 definitions in the first place. The duplicate-sqlite3_*ld warnings visible in Edge builds come fromlibzcashlcvs react-native-piratechain'slibpiratelc(both Rust cores embed sqlite3); they are pre-existing and unrelated to this change.Validation
npm run update-sourcesruns clean end-to-end (SDK clone → libzcashlc → source vendoring → deps build for both platforms)..a= arm64; simulator.a= arm64 + x86_64 (lipo -infoverified);swift-version.txtstamped; GRPC/NIO symbols present; 0 sqlite3 definitions; ZcashLightClientKit correctly absent (compiled from source in-pod).strip -S) is a possible follow-up.Pairs with the GUI's RN 0.85 upgrade: EdgeApp/edge-react-gui#6064
🤖 Generated with Claude Code