feat(archiver): decouple calldata from blob fetching in L1 synchronizer#22716
Merged
Conversation
spalladino
added
the
ci-no-fail-fast
Maddiaa0
approved these changes
Apr 21, 2026
Base automatically changed from
spl/fix-genesis-world-state-again
to
merge-train/spartan
April 22, 2026 08:43
Splits checkpoint retrieval into two phases: first fetch calldata only (header, attestations, archive root), then check if a proposed checkpoint with matching header already exists in the store. If so, skip blob fetching and promote the proposed checkpoint to confirmed. Otherwise, fetch blobs in parallel as before. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move promote to after attestation validation to avoid persisting invalid checkpoints (split into build + persist steps) - Add validateCheckpoint call in promote path - Add archive root comparison to proposed checkpoint match condition - Remove dead retrieveCheckpointsFromRollup and processCheckpointProposedLogs - Pass pendingChainValidationStatus to promote path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
spalladino
force-pushed
the
spl/decouple-calldata-blobs
branch
from
40bf06d to
42da07a
Compare
chrismarino
pushed a commit
to chrismarino/aztec-packages
that referenced
this pull request
May 5, 2026
BEGIN_COMMIT_OVERRIDE fix(kv-store): ensure LMDB cursor is closed on iteration abort (AztecProtocol#22509) fix(telemetry-client): use appropriate histogram buckets for L1 gas prices (AztecProtocol#22512) fix(telemetry-client): log warning when BatchSpanProcessor drops spans (AztecProtocol#22511) fix(stdlib): wrap HA signer databaseUrl in SecretValue (AztecProtocol#22510) fix(prover-client): don't mark in-progress epoch N jobs as stale when epoch N+1 starts (AztecProtocol#22508) chore: (A-730) graceful shutdown for services in node startup failure path (AztecProtocol#22112) fix(prover-client): reject stale job promises and count timeouts toward retry limit (AztecProtocol#21842) feat(archiver): validate historical L1 log availability at startup (AztecProtocol#22644) fix(archiver): do not query MessageSent events by blockhash (AztecProtocol#22641) refactor(e2e): skip initial sequencer in p2p and epochs tests (AztecProtocol#22535) fix: handle missing L1 finalized block on devnets (AztecProtocol#22663) fix(world-state): treat historical block 0 queries as historical, not latest (AztecProtocol#22679) fix(sequencer): re-check parent checkpoint validity before pipelined L1 submission (AztecProtocol#22586) fix(world-state): make block 0 a first-class historical block (AztecProtocol#22711) chore: show all running versions (AztecProtocol#22376) chore: fix prettier inside worktrees (AztecProtocol#22557) feat: use optimized verifier for rollup (AztecProtocol#21840) fix(kv-store): skip pool creation on ephemeral deleteDb to unstick browser tests (AztecProtocol#22693) chore: rm claude lockfile (AztecProtocol#22718) fix(e2e): wait for first checkpoint in fee_asset_price_oracle_gossip test (AztecProtocol#22719) chore(prover-node): track estimated L1 fee when proof publishing is disabled (AztecProtocol#22691) fix(ci): rerun squashed PR check on base branch change (AztecProtocol#22713) feat(archiver): decouple calldata from blob fetching in L1 synchronizer (AztecProtocol#22716) refactor(e2e): enable pipelining in e2e_epochs tests (AztecProtocol#22544) feat(p2p): reject and evict txs with insufficient max fee per gas (AztecProtocol#22118) refactor(world-state): always index block 0 regardless of initial tree size (AztecProtocol#22724) fix(e2e): fix redistribution test (AztecProtocol#22729) END_COMMIT_OVERRIDE
alexghr
added a commit
that referenced
this pull request
May 7, 2026
…22994) ## Motivation The `aztec.archiver.block_height` series with no status attribute (rendered as the "Pending chain" line on the network, prover, and fisherman Grafana dashboards) stopped being published a couple of weeks ago. With pipelining enabled every checkpoint arriving from L1 already has its blocks in the proposed store, so the L1 synchronizer always took the new promotion fast path introduced in #22716, leaving `checkpointsToAdd` empty and skipping the metric call. ## Approach Record the checkpointed block-height metrics across all valid checkpoints in the batch instead of only the ones routed through `addCheckpoints`, so the promoted checkpoint contributes too. The duration is averaged over the full batch since `addCheckpoints` performs the work for both paths in a single transaction. ## Changes - **archiver (`l1_synchronizer.ts`)**: Move the `processNewCheckpointedBlocks` call to use `validCheckpoints` rather than `checkpointsToAdd`, restoring the empty-status `block_height`, `checkpoint_height`, `sync_block_count`, and `sync_per_checkpoint` series under pipelining. --------- Co-authored-by: Alex Gherghisan <alexghr@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reopening #22472 which was accidentally merged
Motivation
When the node is a validator that already built a checkpoint locally (via
addProposedBlock+setProposedCheckpoint), the blocks are already in the archiver store. Fetching blobs from the beacon chain is redundant and expensive, especially during sync. This decouples calldata and blob retrieval so we can skip blob fetching when the proposed checkpoint matches.Approach
Split
retrieveCheckpointsFromRollupinto two phases: (1) fetch calldata only (header, attestations, archive root), (2) check the archiver store for a proposed checkpoint with matching header. If found, promote it to confirmed via a fast path. Otherwise, fetch blobs in parallel (sameasyncPool(10, ...)concurrency as before) and store as normal.Changes
CalldataOnlyCheckpointtype,retrieveCheckpointCalldataFromRollup(calldata-only fetch), andfetchBlobsAndBuildPublishedCheckpoint(deferred blob fetch). Existing functions left intact.promoteProposedToCheckpointedmethod that reads existing blocks from store, writes a confirmed checkpoint entry with L1 metadata + attestations, and clears the proposed singleton.promoteProposedToCheckpointed.promoteProposedCheckpointwrapper that handles validation status and tips cache refresh.handleCheckpointsnow partitions calldata checkpoints into promote-vs-fetch-blobs, fetches blobs in parallel for non-matching ones, promotes the matched one, then merges results for validation.Fixes A-877