Revert "feat(archiver): decouple calldata from blob fetching in L1 synchronizer"#22715
Closed
spalladino wants to merge 18 commits into
Closed
Revert "feat(archiver): decouple calldata from blob fetching in L1 synchronizer"#22715spalladino wants to merge 18 commits into
spalladino wants to merge 18 commits into
Conversation
## Summary
### Motivation
Previously, ZK masking used tail masking: random values were stored in
separate `MaskingTailData` polynomials at the end of the trace. This
forced all PCS code to be `dyadic_size` dependent — the batched
polynomial `A₀`, Gemini folds, and Shplonk quotient all had to be
allocated and operated at `dyadic_size` because the masking tails
extended to the last rows. The row-disabling polynomial was also
circuit-size dependent (`1 - L_{n-1} - ... - L_{n-4}`), requiring a
`padding_indicator_array` to handle padded sumcheck rounds.
### What Changed
Top-of-trace masking: Random masking values are written directly into
witness polynomials at rows `{1, 2, 3}` during allocation. The
row-disabling polynomial disables the first 4 rows (`0–3`) instead of
the last 4. `lagrange_first` moves from row `0` to row `TRACE_OFFSET (=
4)`.
### Key Benefits
* PCS decoupled from `dyadic_size`: With masking at low indices,
committed polynomials only extend to `max_end_index()` (the actual trace
extent), not `dyadic_size`. Gemini fold polynomials already track
`actual_size` from `A_0.end_index()`. This enables future non-dyadic PCS
proving where sparse circuits avoid paying for the full power-of-2
padding.
* Circuit-size-independent row-disabling: The polynomial `1 -
∏_{i≥2}(1-u_i)` depends only on sumcheck challenges, not circuit size.
The `padding_indicator_array` is eliminated.
* Simpler code: Removed `MaskingTailData` (218 lines),
`padding_indicator_array` (93+134 lines), and all tail-batching logic.
Net `-707` lines across 103 files.
* Unified verifier paths: Sumcheck and Shplemini verifiers no longer
take `padding_indicator_array` parameters. This simplifies native,
recursive, and AVM verifier flows alike.
### Merge Protocol and ECC Op Wire Changes
Moving `lagrange_first` from row `0` to row `TRACE_OFFSET (= 4)` shifts
the entire execution trace down by 4 rows. In Mega, the `ecc_op` block
now starts at `trace_offset() = TRACE_OFFSET + 1 = 5`, and `ecc_op_wire`
data sits one row before at row `TRACE_OFFSET = 4` (to satisfy the
`ecc_op_wire[row] == w_shift[row]` constraint).
The merge protocol must match this layout: it prepends `TRACE_OFFSET`
zeros to its table polynomials (`L`, `R`, and `M` in PREPEND mode) so
that the prover's Shplonk quotient is consistent with the `ecc_op_wire`
commitments held by the verifier. Previously, these polynomials started
at row `0`; now they start at row `TRACE_OFFSET` to align with the
shifted circuit layout.
The Translator receives its op queue data from Mega's `ecc_op_wires`.
Since the merge protocol handles the offset alignment in chonk, standard
Mega → Translator flows work correctly. However, the two-layer AVM flow
(AVM recursive verifier inside a Mega circuit) required adding
`TRACE_OFFSET = 0` to the standalone AVM flavor. The `MegaAVM` flavor
inherits Mega's `TRACE_OFFSET = 4`, so `ecc_op_wires` in the outer Mega
circuit are offset. The inner AVM has no disabled region (`TRACE_OFFSET
= 0`), but since it runs inside a Mega wrapper that handles the merge
protocol, the op queue data alignment stays consistent across the
boundary.
### Files Changed (103 files, +1484/-2191)
#### Deleted
* `sumcheck/masking_tail_data.hpp` — tail masking infrastructure
* `stdlib/primitives/padding_indicator_array/` — virtual-round indicator
for old row-disabling
#### Added
* `ultra_honk/zk_boundary.test.cpp` — tests for masking layout, `ecc_op`
alignment, row-disabling
#### Core changes
* Prover instance (`prover_instance.cpp`): Polynomials allocated with
`add_masking()` at rows `{1,2,3}`; `lagrange_first` at `TRACE_OFFSET`;
trace blocks start at `TRACE_OFFSET + 1`
* Oink prover (`oink_prover.cpp`): Removed all tail references;
commitments use base polynomials directly
* Ultra prover (`ultra_prover.cpp`): CRS sized to `max_end_index()`
instead of `dyadic_size()` for ZK
* ECCVM flavor (`eccvm_flavor.hpp`): Polynomial allocation with trace
offset; Lagrange polys shifted
* ECCVM prover (`eccvm_prover.cpp`): Removed tail batching and
`extend_with_tail` copies for translation polys
* Sumcheck (`sumcheck.hpp`): Unified ZK/non-ZK virtual round paths;
removed `padding_indicator_array` from verifier
* Row-disabling (`row_disabling_polynomial.hpp`): Simplified to `1 -
∏_{i≥2}(1-u_i)`
* Gemini (`gemini_impl.hpp`): Fold polynomials track `actual_size` from
`A_0.end_index()`
* Shplemini (`shplemini.hpp`): Removed `padding_indicator_array` from
`compute_batch_opening_claim`
* AVM (`vm2/constraining/`): Removed `padding_indicator_array` from
native and recursive AVM verifiers; added `TRACE_OFFSET = 0` to AVM
flavor
* Merge protocol (`merge_prover.cpp/hpp`, `merge_verifier.cpp/hpp`):
Prepends `TRACE_OFFSET` zeros to table polynomials; verifier accounts
for offset in commitment checks
* Grand product / log-derivative: Start after disabled region via
`gp_start` / `start_index` parameters
* Databus (`databus.hpp`): `DEFAULT_VALUE = 0` (point-at-infinity
commitment matches default)
### Known Issues / Follow-ups
* The zk ultra prover efficiency needs to be investigated.
---
---------
Co-authored-by: notnotraju <raju@aztec-labs.com>
Fix the static assert in Translator: the size for the translator is determined by the `MINI_CIRCUIT_SIZE * CONCATENATION_GROUP_SIZE`. This number must be bigger than `SORTED_STEPS` (which is the max size of the microlimbs divided by 3) times `NUM_ORDERED_RANGE` (1 + the number of interleaved polynomials) plus the masking rows in the overflow column (which are equal to `4 * 64 / 5` (see the README). The old static assert compared `SORTED_STEPS` times `1 + NUM_CONCATENATED_POLYS` with the size of the translator.
This fixes some links in the docsite that point to gh files to instead point to apiref pages. I also removed hundred of stale include code markers that are not being included anywhere - remnants of deleted tutorials. I applied the link fixes to the 4.2.0 docs too. --------- Co-authored-by: AztecBot <tech@aztec-labs.com>
Persist a BlockPayload for block 0 at tree genesis so historical queries pinned to the initial state resolve against the block-0 snapshot instead of throwing or silently returning the latest committed tip. - cached_content_addressed_tree_store.hpp: write a block-0 payload (root=initialRoot, size=initialSize) in commit_genesis_state so get_block_data(0) succeeds. - append_only_tree / indexed_tree: drop the blockNumber==0 throw guards on read paths (get_sibling_path, get_leaf, find_leaf_indices_from, find_leaf_sibling_paths, find_low_leaf) now that genesis reads have a valid payload to resolve against. - aztec-node server.getWorldState: stop short-circuiting to getCommitted when the anchor matches the genesis header. Resolve to BlockNumber.ZERO and fall through to the standard snapshot + archive-root double-check so proving against the genesis anchor returns genesis state even after the node tip has advanced. - e2e_genesis_timestamp: unskip the regression test exercising the bug (prove a genesis-anchored tx after the chain has advanced with public data tree changes). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…er (#22472) ## Motivation When the node is a validator that already built a checkpoint locally (via `addProposedBlock` + `setProposedCheckpoint`), the blocks are already in the archiver store. Fetching blobs from the beacon chain is redundant and expensive, especially during sync. This decouples calldata and blob retrieval so we can skip blob fetching when the proposed checkpoint matches. ## Approach Split `retrieveCheckpointsFromRollup` into two phases: (1) fetch calldata only (header, attestations, archive root), (2) check the archiver store for a proposed checkpoint with matching header. If found, promote it to confirmed via a fast path. Otherwise, fetch blobs in parallel (same `asyncPool(10, ...)` concurrency as before) and store as normal. ## Changes - **archiver (l1/data_retrieval.ts)**: New `CalldataOnlyCheckpoint` type, `retrieveCheckpointCalldataFromRollup` (calldata-only fetch), and `fetchBlobsAndBuildPublishedCheckpoint` (deferred blob fetch). Existing functions left intact. - **archiver (store/block_store.ts)**: New `promoteProposedToCheckpointed` method that reads existing blocks from store, writes a confirmed checkpoint entry with L1 metadata + attestations, and clears the proposed singleton. - **archiver (store/kv_archiver_store.ts)**: Pass-through for `promoteProposedToCheckpointed`. - **archiver (modules/data_store_updater.ts)**: New `promoteProposedCheckpoint` wrapper that handles validation status and tips cache refresh. - **archiver (modules/l1_synchronizer.ts)**: `handleCheckpoints` now partitions calldata checkpoints into promote-vs-fetch-blobs, fetches blobs in parallel for non-matching ones, promotes the matched one, then merges results for validation. Fixes A-877 --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nchroniz…" This reverts commit 48236ce.
spalladino
force-pushed
the
spl/fix-genesis-world-state-again
branch
from
48236ce to
bb08d96
Compare
spalladino
requested review from
a team,
IlyasRidhuan,
LeilaWang,
fcarreiro,
jeanmon and
nventuro
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts #22472