test: migrate benchmarks to pipelining setup#23647
Merged
Merged
Conversation
spalladino
added
ci-full
PhilWindle
approved these changes
May 29, 2026
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
spalladino
added a commit
that referenced
this pull request
May 29, 2026
Proposer pipelining (build for slot+1, defer finalization to the next slot, build on the locally-proposed parent checkpoint) is now the only behavior of the production sequencer. Removes the SEQ_ENABLE_PROPOSER_PIPELINING env var / enableProposerPipelining config and every isProposerPipeliningEnabled() and pipeliningOffset() check, collapsing each to the pipelining path. The EpochCache always applies PROPOSER_PIPELINING_SLOT_OFFSET. The checkpoint timing model is consolidated to a single pipelined model (StandardCheckpointTimingModel deleted; the pipelining selector removed from createCheckpointTimingModel / calculateMaxBlocksPerSlot). Gossipsub maxBlocksPerSlot rises accordingly, now consistent with the always-pipelining sequencer. The test-only AutomineSequencer is preserved: it is selected by the separate useAutomineSequencer flag, publishes synchronously in-slot via sendRequests, and keeps passing pipeliningEnabled: false to getPreviousCheckpointOutHashes (the one retained parameter, with dedicated unit coverage). Checks for whether a proposed checkpoint exists (hasProposedCheckpoint, proposedCheckpointData) are kept. BREAKING CHANGE: SEQ_ENABLE_PROPOSER_PIPELINING removed; pipelining is always on. Also includes the benchmark pipelining-setup migration from #23647.
spalladino
added a commit
that referenced
this pull request
May 29, 2026
Proposer pipelining (build for slot+1, defer finalization to the next slot, build on the locally-proposed parent checkpoint) is now the only behavior of the production sequencer. Removes the SEQ_ENABLE_PROPOSER_PIPELINING env var / enableProposerPipelining config and every isProposerPipeliningEnabled() and pipeliningOffset() check, collapsing each to the pipelining path. The EpochCache always applies PROPOSER_PIPELINING_SLOT_OFFSET. The checkpoint timing model is consolidated to a single pipelined model (StandardCheckpointTimingModel deleted; the pipelining selector removed from createCheckpointTimingModel / calculateMaxBlocksPerSlot). Gossipsub maxBlocksPerSlot rises accordingly, now consistent with the always-pipelining sequencer. The test-only AutomineSequencer is preserved: it is selected by the separate useAutomineSequencer flag, publishes synchronously in-slot via sendRequests, and keeps passing pipeliningEnabled: false to getPreviousCheckpointOutHashes (the one retained parameter, with dedicated unit coverage). Checks for whether a proposed checkpoint exists (hasProposedCheckpoint, proposedCheckpointData) are kept. BREAKING CHANGE: SEQ_ENABLE_PROPOSER_PIPELINING removed; pipelining is always on. Also includes the benchmark pipelining-setup migration from #23647.
spalladino
added a commit
that referenced
this pull request
Jun 1, 2026
Proposer pipelining (build for slot+1, defer finalization to the next slot, build on the locally-proposed parent checkpoint) is now the only behavior of the production sequencer. Removes the SEQ_ENABLE_PROPOSER_PIPELINING env var / enableProposerPipelining config and every isProposerPipeliningEnabled() and pipeliningOffset() check, collapsing each to the pipelining path. The EpochCache always applies PROPOSER_PIPELINING_SLOT_OFFSET. The checkpoint timing model is consolidated to a single pipelined model (StandardCheckpointTimingModel deleted; the pipelining selector removed from createCheckpointTimingModel / calculateMaxBlocksPerSlot). Gossipsub maxBlocksPerSlot rises accordingly, now consistent with the always-pipelining sequencer. The test-only AutomineSequencer is preserved: it is selected by the separate useAutomineSequencer flag, publishes synchronously in-slot via sendRequests, and keeps passing pipeliningEnabled: false to getPreviousCheckpointOutHashes (the one retained parameter, with dedicated unit coverage). Checks for whether a proposed checkpoint exists (hasProposedCheckpoint, proposedCheckpointData) are kept. BREAKING CHANGE: SEQ_ENABLE_PROPOSER_PIPELINING removed; pipelining is always on. Also includes the benchmark pipelining-setup migration from #23647.
spalladino
added a commit
that referenced
this pull request
Jun 1, 2026
## Motivation Proposer pipelining (the proposer builds for `slot + 1` while in `slot`, defers checkpoint finalization to the next slot, and builds on the locally-gossiped proposed parent checkpoint) was gated behind `SEQ_ENABLE_PROPOSER_PIPELINING`. Production already runs with it on and every non-trivial e2e suite opts in, so the dual on/off code path was dead weight — a duplicated timing model and `if (isPipelining)` branches scattered across the sequencer, validator, and p2p stack. This makes pipelining the only behavior of the production sequencer and removes the toggle. ## Approach Removed the `enableProposerPipelining` config / `SEQ_ENABLE_PROPOSER_PIPELINING` env var everywhere and dropped every `isProposerPipeliningEnabled()` / `pipeliningOffset()` check, collapsing each to the pipelining branch (the `EpochCache` now always applies `PROPOSER_PIPELINING_SLOT_OFFSET`). The checkpoint timing model was consolidated to a single pipelined model. The test-only `AutomineSequencer` is preserved — it is selected by the separate `useAutomineSequencer` flag, publishes synchronously in-slot, and is the one remaining caller of the non-pipelined `getPreviousCheckpointOutHashes` branch. Checks for whether a proposed checkpoint exists (`hasProposedCheckpoint` / `proposedCheckpointData`) are kept. ## Changes - **stdlib**: deleted `pipelining-config.ts`; consolidated `timetable` to a single `CheckpointTimingModel` (removed `StandardCheckpointTimingModel` and the `pipelining` option from `createCheckpointTimingModel` / `calculateMaxBlocksPerSlot`); kept the `pipeliningEnabled` param on `getPreviousCheckpointOutHashes` for automine, with new dedicated unit coverage. - **foundation / archiver / sequencer-client / epoch-cache (config)**: removed the env-var registry entry and the `PipelineConfig` merges. - **epoch-cache**: removed `isProposerPipeliningEnabled()` / `pipeliningOffset()`; `getTargetSlot` / `getTargetEpochAndSlotInNextL1Slot` / `getTargetAndNextSlot` always apply the offset. - **sequencer-client**: collapsed the sequencer / checkpoint-proposal-job / publisher branches to the pipelining path; `SequencerTimetable` no longer takes a `pipelining` flag; `AutomineSequencer` untouched (boundary comment added). - **validator-client / p2p**: collapsed proposal-handler, p2p-client, and clock-tolerance branches; deleted the orphaned `waitForBlockSourceSync` and the now-dead `block_source_not_synced` reason. - **p2p gossipsub**: `maxBlocksPerSlot` now uses the pipelined timing model (a higher value), consistent with the always-pipelining sequencer. - **end-to-end (tests)**: dropped the flag from `PIPELINING_SETUP_OPTS` / `AUTOMINE_E2E_OPTS` and ~40 test sites; `setup.ts` allows empty checkpoints unconditionally. - **spartan / docs / aztec-up / docker-compose**: removed the env var from infra, and added a migration note. - Also includes the benchmark pipelining-setup migration cherry-picked from #23647. Note: `SEQ_ENABLE_PROPOSER_PIPELINING` is a breaking change for node operators — see the migration note. Labeled `ci-no-fail-fast` to survey the full suite.
danielntmd
pushed a commit
to danielntmd/aztec-packages
that referenced
this pull request
Jun 4, 2026
BEGIN_COMMIT_OVERRIDE test(e2e): unskip pipelining related e2e tests (AztecProtocol#23642) fix(archiver): prune blocks without proposed checkpoint by end of build slot (AztecProtocol#23606) test: migrate benchmarks to pipelining setup (AztecProtocol#23647) fix(p2p): fall back to archiver in BLOCK_TXS response validation (AztecProtocol#23624) docs(slashing): align operator and slasher docs with AZIP-7 (AztecProtocol#23494) fix(p2p): do not penalize peers that signal a missing block with Fr.ZERO (AztecProtocol#23672) chore: adjust metrics deployment (AztecProtocol#23676) fix(cheat-codes): warpL2TimeAtLeastBy advances relative to leading clock (AztecProtocol#23675) chore: tighten node pool sizes (AztecProtocol#23678) chore: remove archival nodes (AztecProtocol#23630) chore: merge blob sink duties into RPC node (AztecProtocol#23631) fix: sync avm-transpiler Cargo.lock with noir submodule (AztecProtocol#23683) fix(spartan): set validator lag env vars in tps-scenario (AztecProtocol#23684) fix: make world-state hash queries reorg-aware to close getWorldState race (AztecProtocol#23677) fix: pin noir submodule to next's version on merge-train/spartan (AztecProtocol#23690) fix: ensure image ref is used by bench runner (AztecProtocol#23682) fix(ci): retry aztec-nr nargo dependency clone on transient network flake (AztecProtocol#23653) chore: run one-off jobs on network nodes (AztecProtocol#23701) fix: simulate proposals inside target slot (AztecProtocol#23692) chore: smaller eth-devnet (AztecProtocol#23704) chore: enable testnet autoscaling (AztecProtocol#23705) feat(api)!: redesign node log retrieval API around tag-based queries (AztecProtocol#23625) fix(sequencer): set own proposed checkpoint locally instead of via p2p loopback (AztecProtocol#23659) END_COMMIT_OVERRIDE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PIPELINING_SETUP_OPTS.ClientFlowsBenchmark.mineBlock()when available so Automine can progress empty blocks.Testing
yarn format end-to-endyarn buildyarn lint end-to-end