feat: txe speedup#23578
Merged
Merged
Conversation
… audit bundle Three layered changes to make the token_contract suite tractable when nargo runs it at high parallelism (e.g. -j64/-j128): 1. worker_thread dispatcher pool. `TXEDispatcherPool` spawns N worker threads, each running its own `TXEDispatcher`. nargo's per-test RPC connections are routed sticky-by-session to a worker, so different tests execute truly in parallel. The pool builds a shared protocol- contracts LMDB on main once; each worker clones the data.mdb file and skips `loadContractArtifact` + `computeArtifactHash` for the 6 protocol contracts plus the SchnorrAccount artifact. RPC dispatch is over msgpackr — see `msgpackr_fr_extension.ts` for the Fr extension reconciliation that's required because `@aztec/native` is externalised from the worker bundle, so the bundled `Fr` class identity differs from the one `@aztec/native/msgpack_channel` registers. 2. WASM bb.js singletons in workers. Before this change, each worker isolate's first crypto call lazy-spawned two bb subprocesses (one Native-Unix-Socket for `Barretenberg.initSingleton`, one Native- SharedMemory for `BarretenbergSync.initSingleton`). At w128 that was 258 bb processes loading SRS in parallel, dominating warm-up and timing-out nargo's RPC connect on 4 of 60 tests. `worker.ts` now seeds both singletons with `BackendType.Wasm` + `skipSrsInit: true` + `threads: 1` before any other import, so every subsequent `initSingleton()` call joins the WASM singleton instead of spawning a subprocess. `threads: 1` is critical at w64+ — with threads > 1 each WASM backend spawns its own thread workers and OOM-kills the host process. 3. Bundle audit. `esbuild.config.mjs` ships TXE as two ESM bundles (`bin/index.js` + `worker.bundle.js`) with aggressive stubbing of modules the worker never executes (telemetry-client, @noble/curves table precomputes, protocol-contracts/providers/bundle, zod locales). Two onResolve filters that previously matched only the resolved-path form (`.../dest/class-registry/index.js`) now also match the package-specifier form (`@aztec/protocol-contracts/ class-registry`), eliminating a dual-load that put the protocol contract JSONs into both their lazy chunks AND the eager startup chunk. Strips file_map / debug_symbols from artifacts inlined into the bundle. token_contract suite (60 tests; pool-warm + test-execute): master + -j8: ~30s + flaky bb-socket timeouts this branch w8: 18s, 60/60 this branch w32: 18s, 60/60 this branch w64: 23s, 60/60 this branch w128: 31s, 60/60 Open follow-ups: w128 warm is still bundle-parse-bound (~17s of 128 isolates each parsing the 2.2MB chunk). Worker count > test count wastes spawn cost; either cap the pool or fall back to a V8 startup snapshot for the next round of gains.
Thunkar
added
the
ci-no-fail-fast
alexghr
reviewed
May 27, 2026
nventuro
reviewed
May 27, 2026
…ges into gj/txe_speedup
…ges into gj/txe_speedupt push
nventuro
approved these changes
May 29, 2026
Comment on lines
+11
to
+13
| receivedTx(_durationMs: number, _isAccepted: boolean): void {} | ||
| recordSnapshot(_durationMs: number): void {} | ||
| recordSnapshotError(): void {} |
Contributor
There was a problem hiding this comment.
You 👍 'd this comment but didn't change the behavior, is that intented or was this an oversight?
Co-authored-by: Nicolás Venturo <nicolas.venturo@gmail.com>
This was referenced May 29, 2026
Thunkar
added a commit
that referenced
this pull request
May 29, 2026
…tion (#23700) ## Why CI on `merge-train/fairies` is red after `feat: txe speedup (#23578)` merged with `next`. The yarn-project compile fails: ``` src/index.ts:43:73 - error TS2322: Type '"AuthRegistry"' is not assignable to type '"ContractClassRegistry" | "ContractInstanceRegistry" | "FeeJuice" | "MultiCallEntrypoint"'. ``` (log: http://ci.aztec-labs.com/1780061823709173 → http://ci.aztec-labs.com/bd7e1fc18ebbb263) This is a semantic merge conflict. #23578 was based on an older `next` where `AuthRegistry` was a protocol contract. On the train it was demoted from a protocol contract to a standard contract (#23106), so `ProtocolContractName` no longer includes it and it is now deployed per-session in `txe_session.ts`. ## Changes - **`txe/src/index.ts`** — drop `'AuthRegistry'` from `TXE_REQUIRED_PROTOCOL_CONTRACTS` (now empty); it is no longer a protocol contract. - **`txe/src/dispatcher_pool.ts`** — load the standard auth registry the same way protocol contracts are loaded: `buildSharedContractStore` now parses and registers its artifact/instance **once in the main thread, in parallel**, so every worker clones the result instead of re-parsing the ~1 MB JSON per session. - **`txe/src/txe_session.ts` + `txe/src/oracle/txe_oracle_top_level_context.ts`** — session init no longer calls `getStandardAuthRegistry()` (it imports only the cheap address-only `/constants`); `deployManyInSingleBlock` is replaced by `mineDeploymentNullifiers`, which only emits the auth-registry deployment nullifier into the baseline block (artifact/instance already come from the cloned shared store). The artifact leaves the worker startup bundle entirely. - **`txe/esbuild/plugins/strip_artifact_debug.mjs`** — extend the existing debug-strip filter to `standard-contracts/artifacts`, so the auth registry (942 → 103 KB) sheds `debug_symbols`/`file_map` when bundled, keeping the TXE bundle under its existing size limits (no limit bumps). ## Verification (local) Built the upstream chain from the artifact cache and ran the failing yarn-project compile step: - `yarn tsgo -b --emitDeclarationOnly` (full-project typecheck) — pass - `cd txe && yarn build` (tsc + esbuild + size guard) — pass; bundle 12.7 MiB, guard does not trip; auth-registry artifact confirmed absent from `worker.bundle` - `format --check` and txe `eslint` — pass Full `./bootstrap.sh ci` (which runs the entire repo incl. C++/circuit tests and the TXE test suites) was not run end-to-end: the noir build toolchain (cargo-binstall) is unavailable on this host, so upstream components were sourced from the build cache and only the yarn-project compile — the failing step — was reproduced locally. The TXE test suites that exercise the auth-registry deployment path should be confirmed by CI. --------- Co-authored-by: Gregorio Juliana <gregojquiros@gmail.com>
danielntmd
pushed a commit
to danielntmd/aztec-packages
that referenced
this pull request
Jun 4, 2026
BEGIN_COMMIT_OVERRIDE docs: restore two-crate testing guidance + /errors/1 anchor (AztecProtocol#23632) feat: txe speedup (AztecProtocol#23578) fix(ci): don't serve cached results for disabled-cache test commands (AztecProtocol#23658) fix(txe): reconcile txe speedup with auth-registry/public-checks demotion (AztecProtocol#23700) feat(aztec-nr)!: remove set_sender_for_tags oracle, use builder pattern for sender override (AztecProtocol#23619) feat!: demote multi_call_entrypoint to non-protocol contract (AztecProtocol#23197) feat(pxe): add class IDs and view context to auth request (AztecProtocol#23633) fix: bb format.sh re-staging inside worktrees (AztecProtocol#23711) feat!: shift remaining protocol-contract addresses to 1-3 (AztecProtocol#23218) feat(standard-contracts): show expected-vs-actual diff in drift error (AztecProtocol#23710) feat(standard-contracts): add v4-style pin-build mechanism for standard contracts (AztecProtocol#23262) END_COMMIT_OVERRIDE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds: