fix(ci): bump AVM check-circuit per-test timeout to 120s for large circuits#23929
Closed
AztecBot wants to merge 1 commit into
Closed
fix(ci): bump AVM check-circuit per-test timeout to 120s for large circuits#23929AztecBot wants to merge 1 commit into
AztecBot wants to merge 1 commit into
Conversation
Collaborator
Author
|
Automatically closing this stale claudebox draft PR (no updates for 5+ days). Re-open if still needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Raise the per-test budget for the
avm_check_circuitruns inyarn-project/end-to-end/bootstrap.shfromTIMEOUT=30stoTIMEOUT=120s.Why
The AVM check-circuit check failed on
next(run 27132247511, commitd9b6e71). Every dumped input passed in 3–5s except one, which was killed withcode: 124(timeout):The sub-log shows where the time went (container limited to
CPUS=2,TIMEOUT=30s):e2e_multiple_blobsproduces the largest AVM circuit we dump (~700k rows). Trace generation takes ~23s on its own, leaving under 7s for the relation check — so the run exceeds the 30s budget that was originally sized for "relatively small" e2e txs (exactly the case the existing WARNING comment anticipated).Root cause
This is a per-test time-budget issue, not a logic bug, and it is not caused by the barretenberg Pippenger/MSM refactor in the triggering merge-train commit:
avm_check_circuitperforms AVM trace generation and gate-relation checking, neither of which uses Pippenger MSM. The 30s timeout has been in place since the feature landed (#18747); the largest circuit simply tipped over the edge.Fix
Bump the timeout to
120s(≈3× the observed ~40s worst case, leaving headroom for CI contention) and update the WARNING comment to point at the largest known case and the lever (TIMEOUT/CPUS) to pull if a bigger tx starts timing out. Parallelism and per-job resources are unchanged, so the small txs are unaffected and overall wall-clock is essentially unchanged (only the one large run now runs to completion instead of being killed).Created by claudebox · group:
slackbot