fix: ntps test remove low=2 scenario#19920
Merged
Merged
Conversation
PhilWindle
approved these changes
Jan 25, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
alexghr
approved these changes
Jan 25, 2026
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
mrzeszutko
added a commit
that referenced
this pull request
Mar 23, 2026
## Summary - **Restore the `LOW_VALUE_TPS=2` benchmark scenario** that was removed in PR #19920 after repeated setup timeouts - **Parallelize wallet creation** in `n_tps.test.ts` by switching from `timesAsync` (sequential) to `timesParallel`, cutting setup time proportionally to the number of wallets - **Add retry logic with backoff** to `deployAccountWithDiagnostics` that tracks pending transactions across retries, avoiding nullifier conflicts when a previous tx is still in the mempool under chaos mesh conditions - **Set `SEQ_BUILD_CHECKPOINT_IF_EMPTY=true`** in the TPS scenario environment so the chain keeps advancing during quiet periods (setup/teardown) ## Details ### Root cause With `LOW_VALUE_TPS=2`, the test needs 3 wallets (2 low + 1 high). Each wallet deployment is an on-chain transaction that must be mined. Under chaos mesh conditions (20% packet drop + latency), sequential deployment of accounts frequently exceeds the test timeout. ### Changes **`yarn-project/end-to-end/src/spartan/n_tps.test.ts`** - Replace `timesAsync` with `timesParallel` for wallet creation — each wallet gets an independent PXE/node client, so no shared state - Add try/catch around `getBlockNumber` polling to avoid crashing on transient RPC errors during setup **`yarn-project/end-to-end/src/spartan/setup_test_wallets.ts`** - Wrap the deploy-and-wait cycle in `retry()` with exponential backoff (5 attempts, [1,2,4,8,16]s delays) - Use a 600s per-attempt `waitForTx` timeout, with retries providing cumulative wait of ~3000s - Track `sentTxHash` across retries: on retry, check if the previous tx was dropped before deciding to re-send. If the tx is still pending, wait for it again instead of sending a duplicate (which would cause `NULLIFIER_CONFLICT` under chaos mesh) - Pre-check `getContract()` before each retry to skip re-deploy if a previous attempt succeeded but `waitForTx` timed out - Check `receipt.isDropped()` explicitly to clear the tracked tx and trigger a fresh send **`spartan/environments/tps-scenario.env`** - Add `SEQ_BUILD_CHECKPOINT_IF_EMPTY=true` to match `prove-n-tps-fake.env` — ensures empty checkpoint blocks are produced in quiet slots, keeping the chain live **`spartan/bootstrap.sh`** - Re-add `2` to `low_value_tps_list` to restore the benchmark scenario ## Test plan - CI should run the restored `low_0_1__high_2` TPS benchmark scenario end-to-end - Existing lower TPS scenarios (0.1, 0.2, 0.5, 1) should be unaffected by the retry/parallel changes Fixes A-492
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CI failed again for the same reason as described in the #19917
This PR just removes the failing scenario altogether until I figure out what's happening, becaue I want to see some numbers in benchmarks