Skip to content

fix(cheat-codes): wait for post-warp L2 block in warpL2TimeAtLeastTo#22796

Closed
AztecBot wants to merge 1 commit into
merge-train/spartanfrom
claudebox/fix-warp-race-spartan
Closed

fix(cheat-codes): wait for post-warp L2 block in warpL2TimeAtLeastTo#22796
AztecBot wants to merge 1 commit into
merge-train/spartanfrom
claudebox/fix-warp-race-spartan

Conversation

@AztecBot

Copy link
Copy Markdown
Collaborator

Summary

Fixes the merge-queue failure in e2e_blacklist_token_contract/shielding (CI run) where every test fails in applyMint with Invalid tx: Invalid expiration timestamp.

Root cause

warpL2TimeAtLeastTo (introduced in #22084) calls eth.warp followed by node.mineBlock(). The sequencer's polling loop captures nowSeconds/slot at the top of each work() cycle. An in-flight cycle that started just before the warp will mine a block at the pre-warp slot — L1 prunes that block, but it lingers in local world state and the PXE anchors subsequent txs against it. With MAX_TX_LIFETIME == CHANGE_ROLES_DELAY == 86400s, the resulting expiration_timestamp lands exactly on the post-warp slot boundary and the validator rejects the tx as soon as the wall-clock crosses to the next slot.

Concretely, from the failing log:

23:23:16.205  Warped L1 timestamp to 1777159965 (slot 1208)
23:23:16.252  Built block 8 at slot 8 (timestamp 1777073565)   ← stale, but world-state kept it
23:23:16.323  World state updated with L2 block 8
23:23:16.352  Tx simulated at slot 1208, anchored to block 8 → expiration = 1777159965 → invalid

Fix

After eth.warp, retry mineBlock until the latest L2 block's slot is at or past the slot corresponding to the warped timestamp. The first mineBlock may return a stale block produced by an in-flight cycle; the next mineBlock triggers a fresh sequencer cycle that reads the post-warp time and builds a block at the post-warp slot. Subsequent txs then anchor against a fresh block whose expiration_timestamp is well in the future.

Signature of warpL2TimeAtLeastTo/warpL2TimeAtLeastBy widened from AztecNodeDebug to AztecNode & AztecNodeDebug so we can read getBlockHeader(). The single composed test that passed only AztecNodeDebug is updated to merge the two clients.

Full investigation and log excerpts: https://gist.github.com/AztecBot/d0ddeadeb0c1fbe424ffabdf93ee1207

Test plan

  • e2e_blacklist_token_contract/shielding (the originally-failing suite)
  • e2e_blacklist_token_contract/access_control, transfer_*, unshielding, etc. — all share applyBaseSetup which calls crossTimestampOfChange
  • e2e_contract_updates
  • composed/e2e_cheat_codes (verifies the type-widening change still resolves the methods correctly)

ClaudeBox log: https://claudebox.work/s/b25160ea81bb1152?run=1

@AztecBot AztecBot added ci-draft Run CI on draft PRs. claudebox Owned by claudebox. it can push to this PR. labels Apr 27, 2026
@AztecBot

AztecBot commented May 3, 2026

Copy link
Copy Markdown
Collaborator Author

Automatically closing this stale claudebox draft PR (no updates for 5+ days). Re-open if still needed.

@AztecBot AztecBot closed this May 3, 2026
benesjan pushed a commit that referenced this pull request May 13, 2026
…23213)

## Summary

Fixes the merge-queue failure in
`e2e_blacklist_token_contract/shielding` ([CI
run](http://ci.aztec-labs.com/d5485e6652b3f32a)) where every test fails
in `applyMint` with `Invalid tx: Invalid expiration timestamp`.

## Root cause

`warpL2TimeAtLeastTo` (introduced in #22084) calls `eth.warp` followed
by `node.mineBlock()`. The sequencer's polling loop captures
`nowSeconds`/`slot` at the top of each `work()` cycle. An in-flight
cycle that started just before the warp will mine an L2 block at the
*pre-warp* slot — L1 sync prunes that block from the canonical chain,
but it lingers in local world state and the PXE anchors subsequent txs
against it. With `MAX_TX_LIFETIME == CHANGE_ROLES_DELAY == 86400s`, the
resulting `expiration_timestamp` lands exactly on the post-warp slot
boundary and the validator rejects the tx as soon as the wall-clock
crosses to the next slot.

## Fix

After `eth.warp`, retry `mineBlock` until the latest L2 block's slot is
at or past the slot corresponding to the warped timestamp. The first
`mineBlock` may return a stale block produced by an in-flight cycle; the
next triggers a fresh sequencer cycle that reads the post-warp time and
builds a block at the post-warp slot. Subsequent txs then anchor against
a fresh block whose `expiration_timestamp` is well in the future.

The signature of `warpL2TimeAtLeastTo`/`warpL2TimeAtLeastBy` widens from
`AztecNodeDebug` to `AztecNode & AztecNodeDebug` so we can read the
latest block via `getBlockData('latest')`. All current callers already
type their node as the intersection.

This re-applies the diagnosis from the prior #22796 (which never
merged), adapted to the current `getBlockData('latest')` API.

Full analysis:
https://gist.github.com/AztecBot/67815cbe3c3f853d97ec3345dfb0c985

## Test plan

- `e2e_blacklist_token_contract/shielding` (originally failing)
-
`e2e_blacklist_token_contract/{access_control,burn,minting,transfer_*,unshielding}`
— share `applyBaseSetup` → `crossTimestampOfChange`
- `e2e_contract_updates`
- `composed/e2e_cheat_codes` (verifies the type-widening change still
resolves the methods correctly)

ClaudeBox log: https://claudebox.work/s/28594b4dc64f1cd0?run=1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-draft Run CI on draft PRs. claudebox Owned by claudebox. it can push to this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant