refactor: cut TS world state and NAPI AVM over to WSDB IPC; delete NAPI WSDB#23036
Open
charlielye wants to merge 21 commits into
Open
refactor: cut TS world state and NAPI AVM over to WSDB IPC; delete NAPI WSDB#23036charlielye wants to merge 21 commits into
charlielye wants to merge 21 commits into
Conversation
charlielye
force-pushed
the
cl/ipc-3-avm-wsdb-cutover
branch
from
144f6c2 to
d2d4ae7
Compare
charlielye
requested review from
IlyasRidhuan,
MirandaWood and
jeanmon
as code owners
charlielye
force-pushed
the
cl/ipc-3-avm-wsdb-cutover
branch
2 times, most recently
from
b7a6517 to
4941f2b
Compare
charlielye
force-pushed
the
cl/ipc-3-avm-wsdb-cutover
branch
3 times, most recently
from
67d3c6c to
d904a74
Compare
5 tasks
AztecBot
force-pushed
the
cl/ipc-1-mpsc-shm
branch
2 times, most recently
from
17b93f4 to
e668757
Compare
This was referenced May 12, 2026
charlielye
force-pushed
the
cl/ipc-3-avm-wsdb-cutover
branch
from
0d58c3e to
679c622
Compare
This was referenced Jun 10, 2026
…PI WSDB The C++ AVM (NAPI) used to dereference an in-process `WorldState*` handle (the NAPI External returned by `NativeWorldState`). With WSDB now running as a standalone aztec-wsdb process, that pointer no longer exists. This PR cuts everything over to IPC: C++: - AvmSimAPI::simulate now takes `LowLevelMerkleDBInterface&` instead of `WorldState&`; AvmSimulationHelper::simulate_fast_internal is promoted to public so callers can plug in their own merkle DB. - nodejs_module/avm_simulate accepts a WSDB UDS socket path (string) as its third arg instead of a NAPI External handle, constructs a WsdbIpcClient + WsdbIpcMerkleDB per simulation, and runs the simulation against that. - nodejs_module/world_state/* deleted — there is no in-process WorldState any more. nodejs_module links wsdb_client + wsdb_ipc_client instead. TypeScript (bb.js): - `findWsdbBinary()` added next to `findBbBinary()` / `findNapiBinary()` for resolving the standalone aztec-wsdb binary on disk. TypeScript (yarn-project): - `@aztec/native` no longer exports `NativeWorldState`. `avmSimulate` takes a `wsdbSocketPath: string` instead of `worldStateHandle: any`. - `NativeWorldStateInstance` interface trades `getHandle()` for `getSocketPath()` and gains `close()`. The legacy in-process `NativeWorldState` class in `world-state/src/native/native_world_state_instance.ts` is gone — only `IpcWorldState` remains. - `NativeWorldStateService.new()`/`.tmp()` now spawn aztec-wsdb under the hood and wrap it in `IpcWorldState`. They preserve the existing public API so callers (tests, factories, the node) don't need to change. - `MerkleTreesFacade.getRevision()` returns plain `WorldStateRevision` again; a new `getSocketPath()` exposes the WSDB socket path so the C++ AVM can attach to the same world state instance the TS layer is using. `GuardedMerkleTreeOperations` and `HintingMerkleWriteOperations` proxy `getSocketPath` through. - `CppPublicTxSimulator` and `CppVsTsPublicTxSimulator` ask the merkle tree for its socket path and pass that to `avmSimulate` instead of the removed handle. - `WorldStateRevisionWithHandle` deleted from stdlib. After this PR, the only path between TS and the world state is IPC, and the only path between the C++ AVM and the world state is also IPC.
Promote `simulate_for_hint_collection_internal` to a public method on `AvmSimulationHelper` that takes a `LowLevelMerkleDBInterface&` directly, mirroring `simulate_fast_internal`. `simulate_for_hint_collection` (the WorldState-taking convenience method) now delegates to it. `AvmSimAPI::simulate(merkle_db)` dispatches based on `config.collect_hints` instead of asserting against it. The previous guard required an in-process WorldState for hint collection, which broke the NAPI AVM after the WSDB cutover (WorldState is no longer in-process). The hinting proxies wrap any `LowLevelMerkleDBInterface`, so `WsdbIpcMerkleDB` works as the backing DB.
After the WSDB cutover, NativeWorldStateService is backed by an out-of-process aztec-wsdb subprocess instead of in-process NAPI. Tests must explicitly call close() to terminate the subprocess; otherwise the IPC socket and child remain open and Jest hangs after the suite completes. The orchestrator_errors test created 9 IPC-backed world states (one per test beforeEach) and leaked all of them, which is the wall-clock 600s timeout the test was hitting in CI.
After the WSDB cutover, NativeWorldStateService is backed by an out-of-
process aztec-wsdb subprocess. Tests must call close() explicitly;
otherwise the IPC subprocess and socket pin Node's event loop and Jest
hangs.
Five `it()`s in native_world_state.test.ts ('manually clears the
database', 'retrieves leaf indices', 'retrieves leaf sibling paths',
'correctly reports block numbers', 'can re-org') created a local ws
without disposing it. Wrap each body in try/finally so the close
happens even when an assertion fails.
Reverts the unref() that was tried in this same commit slot earlier:
unref'ing the wsdb child during startup caused the sandbox container to
exit 0 silently (aztec-wsdb is spawned before the HTTP server is up, so
the socket is briefly the only ref'd handle on the event loop). The
right fix is to close in tests, not unref in the backend.
The beforeAll hook spun up a NativeWorldStateService to generate mock checkpoints and never closed it, leaking an aztec-wsdb child for the entire suite. Wrap in try/finally so the db closes even if checkpoint generation throws.
The function creates a fresh NativeWorldStateService for the rerun but never closes it, leaking an aztec-wsdb subprocess. e2e_epochs/ epochs_upload_failed_proof was hanging Jest because of this — the test calls rerunEpochProvingJob after tearing down its EpochsTestContext, so the leaked world state has no fixture to clean it up.
Move the bb.js binary discovery + WsdbBackend construction + waitUntilReady ceremony into a single static factory on IpcWorldState. NativeWorldStateService new() and tmp() now call IpcWorldState.spawn() instead of inlining ~20 lines of duplicated plumbing each. Also drops the unused rollupAddress parameter from tmp() (it was dead since tmp() bypasses DatabaseVersionManager) and the unused fromIpc() static factory (no remaining callers).
…/finally NativeWorldStateService now implements [Symbol.asyncDispose] so callers can write 'await using ws = await NativeWorldStateService.tmp()' and let the runtime invoke close() on scope exit. Replaces five try/finally wrappers in native_world_state.test.ts, one in integration.test.ts, and one in rerun-epoch-proving-job.ts. Reverting the indentation removes ~540 lines of pure-format churn from the PR diff without changing behavior.
…ions flake The multiple_validators_sentinel test was flaking on slot 7 with 'collected 3/5 attestations' even after the +1 slot warmup, because the p2p mesh isn't always fully populated by then. Bumped to +2 slots and added the same warmup to the second test variant (which previously had no warmup at all).
…ntime layout WsdbIpcMerkleDB moved to vm2/simulation/dbs/ in the first stack; the generated wsdb client lives at wsdb/generated/wsdb_ipc_client.hpp. Drop the world_state link from nodejs_module — the cutover deleted that dependency.
charlielye
force-pushed
the
cl/ipc-bb-js-migrate
branch
from
3cddf51 to
c8b3cfa
Compare
charlielye
force-pushed
the
cl/ipc-3-avm-wsdb-cutover
branch
from
f46984b to
0d5cc4a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cuts TypeScript world-state usage and the in-process C++ AVM NAPI path over to the standalone generated
@aztec/wsdbpackage andaztec-wsdbservice from the lower stack.This PR is now stacked after the bb.js migration, so it consumes the new generated package/runtime architecture instead of the old hand-rolled IPC pieces.
Stack
cl/ipc-foundationcl/ipc-wsdb-migratecl/ipc-bb-migratecl/ipc-bb-rs-migratecl/ipc-bb-js-migratecl/ipc-3-avm-wsdb-cutover— this PRcl/ipc-4-avm-binarycl/ipc-5-avm-cutoverWhat changes
C++ / NAPI AVM path
AvmSimAPI::simulatetakes aLowLevelMerkleDBInterface&instead of an in-processWorldState&.WorldState.ipc_runtimepath from the lower stack.TypeScript world-state path
NativeWorldStateServicespawnsaztec-wsdbthrough the generated@aztec/wsdbpackage.WorldStateRevisionWithHandleshape is removed; callers use ordinary revisions and, where needed, the wsdb socket path.End state after this PR
The TS world-state path and the C++ AVM path both talk to world state over IPC. The in-process NAPI WSDB module is gone.
Validation
wsdb/bootstrap.shyarn-project/yarn install --immutableon this branch after generated package lock updates../bootstrap.shpassed on this branch during stack validation.