Skip to content

feat: Min and max block times for sequencer#7630

Merged
spalladino merged 2 commits into
masterfrom
palla/seq-empty-blocks
Jul 26, 2024
Merged

feat: Min and max block times for sequencer#7630
spalladino merged 2 commits into
masterfrom
palla/seq-empty-blocks

Conversation

@spalladino

@spalladino spalladino commented Jul 26, 2024

Copy link
Copy Markdown
Contributor

Adds two new config variables for the sequencer: min and max block time in seconds. The sequencer now checks the current time against the previous block. If it's less than min, it doesn't build a block. If it's more than max, it builds the block even with no txs.

@spalladino
feat: Min and max block times for sequencer
d7690ec 
Adds two new config variables for the sequencer: min and max block time
in seconds. The sequencer now checks the current time against the
previous block. If it's less than min, it doesn't build a block. If it's
more than max, it builds the block even with no txs.
@AztecBot

AztecBot commented Jul 26, 2024 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Docs Preview

Hey there! 👋 You can check your preview at https://66a3cd3a4a43fc138d0e7414--aztec-docs-dev.netlify.app

@AztecBot

AztecBot commented Jul 26, 2024 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Benchmark results

Metrics with a significant change:

  • protocol_circuit_simulation_time_in_ms (private-kernel-tail-to-public): 5,303 (+18%)
  • avm_simulation_time_ms (Token:mint_public): 272 (+205%)
  • avm_simulation_time_ms (Token:assert_minter_and_mint): 49.2 (-51%)
  • avm_simulation_time_ms (Token:_increase_public_balance): 18.5 (-78%)
  • avm_simulation_time_ms (FPC:pay_refund_with_shielded_rebate): 135 (+106%)
Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric 1 threads 4 threads 16 threads 32 threads 64 threads
proof_construction_time_sha256_ms 5,714 (-1%) 1,543 705 750 (-1%) 769 (-1%)
proof_construction_time_sha256_30_ms 11,735 (-1%) 3,153 (-1%) 1,404 (-1%) 1,425 1,453 (-1%)
proof_construction_time_sha256_100_ms 43,810 (-1%) 11,971 5,415 (-1%) 5,393 (-1%) 5,361
proof_construction_time_poseidon_hash_ms 78.0 34.0 34.0 (-3%) 59.0 (+2%) 87.0
proof_construction_time_poseidon_hash_30_ms 1,517 (-1%) 419 (-1%) 202 228 (+2%) 270 (+2%)
proof_construction_time_poseidon_hash_100_ms 5,622 (-1%) 1,505 (-1%) 674 723 (-1%) 752 (+1%)

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 4 txs 8 txs 16 txs
l1_rollup_calldata_size_in_bytes 708 708 708
l1_rollup_calldata_gas 6,600 6,586 6,576
l1_rollup_execution_gas 584,280 584,417 584,558
l2_block_processing_time_in_ms 752 (-1%) 1,421 (+1%) 2,721
l2_block_building_time_in_ms 10,830 (-1%) 21,076 42,070
l2_block_rollup_simulation_time_in_ms 10,830 (-1%) 21,075 42,070
l2_block_public_tx_process_time_in_ms 9,039 (-1%) 19,105 39,918

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric 3 blocks 5 blocks
node_history_sync_time_in_ms 7,101 (+1%) 9,889
node_database_size_in_bytes 12,705,872 16,900,176
pxe_database_size_in_bytes 16,254 26,813

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit simulation_time_in_ms witness_generation_time_in_ms input_size_in_bytes output_size_in_bytes proving_time_in_ms proof_size_in_bytes num_public_inputs size_in_gates
private-kernel-init 104 (-6%) 412 (-1%) 21,310 53,089 N/A N/A N/A N/A
private-kernel-inner 188 (+1%) 722 79,930 53,265 N/A N/A N/A N/A
private-kernel-tail 451 905 (+1%) 60,700 57,446 N/A N/A N/A N/A
base-parity 6.44 709 160 96.0 1,994 (-4%) 13,188 19.0 131,072
root-parity 109 (+1%) 119 (-2%) 66,908 96.0 32,416 (+4%) 13,188 19.0 4,194,304
base-rollup 3,402 4,935 (-1%) 181,595 632 44,589 13,988 44.0 4,194,304
root-rollup 154 (+1%) 118 (-6%) 52,797 652 28,209 13,924 42.0 4,194,304
public-kernel-setup 116 (+1%) 2,707 (-1%) 112,376 80,246 17,747 (-2%) 141,444 4,027 2,097,152
public-kernel-app-logic 121 3,924 (-3%) 112,376 80,246 11,002 (-3%) 141,444 4,027 1,048,576
public-kernel-tail 639 23,038 (-7%) 410,117 10,814 66,913 (-1%) 26,372 431 8,388,608
private-kernel-reset-tiny 233 (-3%) 900 76,064 52,960 N/A N/A N/A N/A
private-kernel-tail-to-public ⚠️ 5,303 (+18%) 1,606 900,567 1,632 N/A N/A N/A N/A
public-kernel-teardown 107 (+6%) 3,882 (-2%) 112,376 80,246 19,454 (-1%) 141,444 4,027 2,097,152
merge-rollup 59.0 (+1%) N/A 34,590 632 N/A N/A N/A N/A
undefined N/A N/A N/A N/A 160,426 N/A N/A N/A

Stats on running time collected for app circuits

Function input_size_in_bytes output_size_in_bytes witness_generation_time_in_ms proof_size_in_bytes proving_time_in_ms
ContractClassRegisterer:register 1,312 11,699 344 N/A N/A
ContractInstanceDeployer:deploy 1,376 11,699 27.5 N/A N/A
MultiCallEntrypoint:entrypoint 1,888 11,699 604 N/A N/A
GasToken:deploy 1,344 11,699 436 N/A N/A
SchnorrAccount:constructor 1,280 11,699 430 N/A N/A
SchnorrAccount:entrypoint 2,272 11,699 820 N/A N/A
Token:privately_mint_private_note 1,248 11,699 502 (-1%) N/A N/A
FPC:fee_entrypoint_public 1,312 11,699 89.8 (+4%) N/A N/A
Token:transfer 1,280 11,699 1,319 N/A N/A
AuthRegistry:set_authorized (avm) 21,702 N/A N/A 102,048 2,234 (+1%)
FPC:prepare_fee (avm) 25,150 N/A N/A 102,112 2,244 (-2%)
Token:transfer_public (avm) 58,188 N/A N/A 102,112 6,795 (+1%)
AuthRegistry:consume (avm) 35,580 N/A N/A 102,080 2,666 (-2%)
FPC:pay_refund (avm) 30,010 N/A N/A 102,080 3,571 (-2%)
Benchmarking:create_note 1,312 11,699 421 N/A N/A
SchnorrAccount:verify_private_authwit 1,248 11,699 44.2 N/A N/A
Token:unshield 1,344 11,699 1,145 (+1%) N/A N/A
FPC:fee_entrypoint_private 1,344 11,699 1,429 (+1%) N/A N/A

AVM Simulation

Time to simulate various public functions in the AVM.

Function time_ms bytecode_size_in_bytes
GasToken:_increase_public_balance 98.6 13,790
GasToken:set_portal 14.7 (-7%) 3,339
Token:constructor 122 (+1%) 23,692
FPC:constructor 100 (+9%) 13,592
GasToken:mint_public 84.3 (+4%) 10,158
Token:mint_public ⚠️ 272 (+205%) 19,034
Token:assert_minter_and_mint ⚠️ 49.2 (-51%) 12,925
AuthRegistry:set_authorized 50.9 (+6%) 7,812
FPC:prepare_fee 251 (+3%) 11,068
Token:transfer_public 52.6 (-5%) 44,012
FPC:pay_refund 74.9 (+1%) 15,961
Benchmarking:increment_balance 1,063 11,281
Token:_increase_public_balance ⚠️ 18.5 (-78%) 15,006
FPC:pay_refund_with_shielded_rebate ⚠️ 135 (+106%) 17,056

Public DB Access

Time to access various public DBs.

Function time_ms
get-nullifier-index 0.157

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric 1 leaves 16 leaves 64 leaves 128 leaves 256 leaves 512 leaves 1024 leaves
batch_insert_into_append_only_tree_16_depth_ms 10.4 16.7 N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_count 16.8 31.7 N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_ms 0.601 0.514 N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_32_depth_ms N/A N/A 48.0 75.9 131 (-1%) 245 473
batch_insert_into_append_only_tree_32_depth_hash_count N/A N/A 95.9 159 287 543 1,055
batch_insert_into_append_only_tree_32_depth_hash_ms N/A N/A 0.491 0.467 0.450 (-1%) 0.444 0.441
batch_insert_into_indexed_tree_20_depth_ms N/A N/A 59.2 (-1%) 111 180 (-2%) 352 694 (-1%)
batch_insert_into_indexed_tree_20_depth_hash_count N/A N/A 109 207 355 691 1,363
batch_insert_into_indexed_tree_20_depth_hash_ms N/A N/A 0.501 (-1%) 0.498 0.477 (-2%) 0.477 0.476
batch_insert_into_indexed_tree_40_depth_ms N/A N/A 72.8 N/A N/A N/A N/A
batch_insert_into_indexed_tree_40_depth_hash_count N/A N/A 133 N/A N/A N/A N/A
batch_insert_into_indexed_tree_40_depth_hash_ms N/A N/A 0.518 N/A N/A N/A N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric 0 registered classes 1 registered classes
tx_size_in_bytes 72,014 666,910

Transaction size based on fee payment method

| Metric | |
| - | |

@spalladino spalladino enabled auto-merge (squash) July 26, 2024 16:08
@spalladino spalladino merged commit 2a013b8 into master Jul 26, 2024
@spalladino spalladino deleted the palla/seq-empty-blocks branch July 26, 2024 16:51
@AztecBot AztecBot mentioned this pull request Jul 26, 2024
Merged
vezenovm pushed a commit that referenced this pull request Jul 30, 2024
@AztecBot
chore(master): Release 0.47.1 (#7605)
bc3689f 
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.47.1</summary>

##
[0.47.1](aztec-package-v0.47.0...aztec-package-v0.47.1)
(2024-07-30)


### Features

* Do not evict tx objects from p2p tx pool immediately
([#7652](#7652))
([96492dc](96492dc))
* Min and max block times for sequencer
([#7630](#7630))
([2a013b8](2a013b8))
* Transaction sender bot
([#7586](#7586))
([176fd08](176fd08)),
closes
[#7562](#7562)


### Bug Fixes

* Aztec start --prover
([#7617](#7617))
([ac206a6](ac206a6))


### Miscellaneous

* Add tx bot to terraform
([#7609](#7609))
([6caa914](6caa914))
* Merge Devnet back to Master
([#7611](#7611))
([112961c](112961c))
</details>

<details><summary>barretenberg.js: 0.47.1</summary>

##
[0.47.1](barretenberg.js-v0.47.0...barretenberg.js-v0.47.1)
(2024-07-30)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.47.1</summary>

##
[0.47.1](aztec-packages-v0.47.0...aztec-packages-v0.47.1)
(2024-07-30)


### Features

* Add recursive aggregation object to proving/verification keys
([#6770](#6770))
([f48b069](f48b069))
* Adding aggregation to honk and rollup
([#7466](#7466))
([2633aa9](2633aa9))
* Allow get_notes to return zero notes
([#7621](#7621))
([e16452a](e16452a))
* **avm:** Pedersen commit in avm
([#7634](#7634))
([45e7867](45e7867))
* **avm:** Pedersen commitment sim
([#7632](#7632))
([cc420a0](cc420a0))
* Do not evict tx objects from p2p tx pool immediately
([#7652](#7652))
([96492dc](96492dc))
* Handle L1toL2 msgs in prover-node
([#7654](#7654))
([f35bac5](f35bac5))
* Leonidas
([#7597](#7597))
([eda498a](eda498a))
* Min and max block times for sequencer
([#7630](#7630))
([2a013b8](2a013b8))
* **sol-honk:** Integrate solidity honk verifier with bb and acir tests
([#7573](#7573))
([344ca6f](344ca6f))
* **sol-honk:** Test verifying recursive proof
([#7576](#7576))
([26408c1](26408c1))
* Static.aztec.network
([#7649](#7649))
([9dffef5](9dffef5))
* Transaction sender bot
([#7586](#7586))
([176fd08](176fd08)),
closes
[#7562](#7562)


### Bug Fixes

* Aztec start --prover
([#7617](#7617))
([ac206a6](ac206a6))
* Bb mac build
([#7619](#7619))
([e3c5602](e3c5602))
* Faucet correctly derives its account
([#7645](#7645))
([921773f](921773f))
* Uniswap e2e test "nonce too low"
([#7633](#7633))
([e9e2318](e9e2318))


### Miscellaneous

* Add tx bot to terraform
([#7609](#7609))
([6caa914](6caa914))
* **bb readme:** Add installation instructions and TODOs
([#7601](#7601))
([1a97698](1a97698))
* **bb readme:** Update versioning instructions and add Honk Solidity
verifier commands
([#7608](#7608))
([9dd9195](9dd9195))
* Bench client ivc proof verification
([#7553](#7553))
([b060309](b060309))
* Call requests
([#7483](#7483))
([ffedf39](ffedf39))
* Constant inputs for most blackboxes
([#7613](#7613))
([3247058](3247058))
* **docs:** Fix links
([#7602](#7602))
([f2029be](f2029be))
* Merge Devnet back to Master
([#7611](#7611))
([112961c](112961c))
* **p2p:** Tx serialization cleanup
([#7620](#7620))
([022a899](022a899))
* Replace relative paths to noir-protocol-circuits
([ebaafc7](ebaafc7))
* Replace relative paths to noir-protocol-circuits
([d3b818d](d3b818d))
* Replace relative paths to noir-protocol-circuits
([4a26a19](4a26a19))
</details>

<details><summary>barretenberg: 0.47.1</summary>

##
[0.47.1](barretenberg-v0.47.0...barretenberg-v0.47.1)
(2024-07-30)


### Features

* Add recursive aggregation object to proving/verification keys
([#6770](#6770))
([f48b069](f48b069))
* Adding aggregation to honk and rollup
([#7466](#7466))
([2633aa9](2633aa9))
* **avm:** Pedersen commit in avm
([#7634](#7634))
([45e7867](45e7867))
* **avm:** Pedersen commitment sim
([#7632](#7632))
([cc420a0](cc420a0))
* **sol-honk:** Integrate solidity honk verifier with bb and acir tests
([#7573](#7573))
([344ca6f](344ca6f))
* **sol-honk:** Test verifying recursive proof
([#7576](#7576))
([26408c1](26408c1))


### Bug Fixes

* Bb mac build
([#7619](#7619))
([e3c5602](e3c5602))


### Miscellaneous

* **bb readme:** Add installation instructions and TODOs
([#7601](#7601))
([1a97698](1a97698))
* **bb readme:** Update versioning instructions and add Honk Solidity
verifier commands
([#7608](#7608))
([9dd9195](9dd9195))
* Call requests
([#7483](#7483))
([ffedf39](ffedf39))
* Constant inputs for most blackboxes
([#7613](#7613))
([3247058](3247058))
* Merge Devnet back to Master
([#7611](#7611))
([112961c](112961c))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Jul 31, 2024
@AztecBot
chore(master): Release 0.47.1 (#7605)
adc05c3 
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.47.1</summary>

##
[0.47.1](AztecProtocol/aztec-packages@aztec-package-v0.47.0...aztec-package-v0.47.1)
(2024-07-30)


### Features

* Do not evict tx objects from p2p tx pool immediately
([#7652](AztecProtocol/aztec-packages#7652))
([96492dc](AztecProtocol/aztec-packages@96492dc))
* Min and max block times for sequencer
([#7630](AztecProtocol/aztec-packages#7630))
([2a013b8](AztecProtocol/aztec-packages@2a013b8))
* Transaction sender bot
([#7586](AztecProtocol/aztec-packages#7586))
([176fd08](AztecProtocol/aztec-packages@176fd08)),
closes
[#7562](AztecProtocol/aztec-packages#7562)


### Bug Fixes

* Aztec start --prover
([#7617](AztecProtocol/aztec-packages#7617))
([ac206a6](AztecProtocol/aztec-packages@ac206a6))


### Miscellaneous

* Add tx bot to terraform
([#7609](AztecProtocol/aztec-packages#7609))
([6caa914](AztecProtocol/aztec-packages@6caa914))
* Merge Devnet back to Master
([#7611](AztecProtocol/aztec-packages#7611))
([112961c](AztecProtocol/aztec-packages@112961c))
</details>

<details><summary>barretenberg.js: 0.47.1</summary>

##
[0.47.1](AztecProtocol/aztec-packages@barretenberg.js-v0.47.0...barretenberg.js-v0.47.1)
(2024-07-30)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.47.1</summary>

##
[0.47.1](AztecProtocol/aztec-packages@aztec-packages-v0.47.0...aztec-packages-v0.47.1)
(2024-07-30)


### Features

* Add recursive aggregation object to proving/verification keys
([#6770](AztecProtocol/aztec-packages#6770))
([f48b069](AztecProtocol/aztec-packages@f48b069))
* Adding aggregation to honk and rollup
([#7466](AztecProtocol/aztec-packages#7466))
([2633aa9](AztecProtocol/aztec-packages@2633aa9))
* Allow get_notes to return zero notes
([#7621](AztecProtocol/aztec-packages#7621))
([e16452a](AztecProtocol/aztec-packages@e16452a))
* **avm:** Pedersen commit in avm
([#7634](AztecProtocol/aztec-packages#7634))
([45e7867](AztecProtocol/aztec-packages@45e7867))
* **avm:** Pedersen commitment sim
([#7632](AztecProtocol/aztec-packages#7632))
([cc420a0](AztecProtocol/aztec-packages@cc420a0))
* Do not evict tx objects from p2p tx pool immediately
([#7652](AztecProtocol/aztec-packages#7652))
([96492dc](AztecProtocol/aztec-packages@96492dc))
* Handle L1toL2 msgs in prover-node
([#7654](AztecProtocol/aztec-packages#7654))
([f35bac5](AztecProtocol/aztec-packages@f35bac5))
* Leonidas
([#7597](AztecProtocol/aztec-packages#7597))
([eda498a](AztecProtocol/aztec-packages@eda498a))
* Min and max block times for sequencer
([#7630](AztecProtocol/aztec-packages#7630))
([2a013b8](AztecProtocol/aztec-packages@2a013b8))
* **sol-honk:** Integrate solidity honk verifier with bb and acir tests
([#7573](AztecProtocol/aztec-packages#7573))
([344ca6f](AztecProtocol/aztec-packages@344ca6f))
* **sol-honk:** Test verifying recursive proof
([#7576](AztecProtocol/aztec-packages#7576))
([26408c1](AztecProtocol/aztec-packages@26408c1))
* Static.aztec.network
([#7649](AztecProtocol/aztec-packages#7649))
([9dffef5](AztecProtocol/aztec-packages@9dffef5))
* Transaction sender bot
([#7586](AztecProtocol/aztec-packages#7586))
([176fd08](AztecProtocol/aztec-packages@176fd08)),
closes
[#7562](AztecProtocol/aztec-packages#7562)


### Bug Fixes

* Aztec start --prover
([#7617](AztecProtocol/aztec-packages#7617))
([ac206a6](AztecProtocol/aztec-packages@ac206a6))
* Bb mac build
([#7619](AztecProtocol/aztec-packages#7619))
([e3c5602](AztecProtocol/aztec-packages@e3c5602))
* Faucet correctly derives its account
([#7645](AztecProtocol/aztec-packages#7645))
([921773f](AztecProtocol/aztec-packages@921773f))
* Uniswap e2e test "nonce too low"
([#7633](AztecProtocol/aztec-packages#7633))
([e9e2318](AztecProtocol/aztec-packages@e9e2318))


### Miscellaneous

* Add tx bot to terraform
([#7609](AztecProtocol/aztec-packages#7609))
([6caa914](AztecProtocol/aztec-packages@6caa914))
* **bb readme:** Add installation instructions and TODOs
([#7601](AztecProtocol/aztec-packages#7601))
([1a97698](AztecProtocol/aztec-packages@1a97698))
* **bb readme:** Update versioning instructions and add Honk Solidity
verifier commands
([#7608](AztecProtocol/aztec-packages#7608))
([9dd9195](AztecProtocol/aztec-packages@9dd9195))
* Bench client ivc proof verification
([#7553](AztecProtocol/aztec-packages#7553))
([b060309](AztecProtocol/aztec-packages@b060309))
* Call requests
([#7483](AztecProtocol/aztec-packages#7483))
([ffedf39](AztecProtocol/aztec-packages@ffedf39))
* Constant inputs for most blackboxes
([#7613](AztecProtocol/aztec-packages#7613))
([3247058](AztecProtocol/aztec-packages@3247058))
* **docs:** Fix links
([#7602](AztecProtocol/aztec-packages#7602))
([f2029be](AztecProtocol/aztec-packages@f2029be))
* Merge Devnet back to Master
([#7611](AztecProtocol/aztec-packages#7611))
([112961c](AztecProtocol/aztec-packages@112961c))
* **p2p:** Tx serialization cleanup
([#7620](AztecProtocol/aztec-packages#7620))
([022a899](AztecProtocol/aztec-packages@022a899))
* Replace relative paths to noir-protocol-circuits
([ebaafc7](AztecProtocol/aztec-packages@ebaafc7))
* Replace relative paths to noir-protocol-circuits
([d3b818d](AztecProtocol/aztec-packages@d3b818d))
* Replace relative paths to noir-protocol-circuits
([4a26a19](AztecProtocol/aztec-packages@4a26a19))
</details>

<details><summary>barretenberg: 0.47.1</summary>

##
[0.47.1](AztecProtocol/aztec-packages@barretenberg-v0.47.0...barretenberg-v0.47.1)
(2024-07-30)


### Features

* Add recursive aggregation object to proving/verification keys
([#6770](AztecProtocol/aztec-packages#6770))
([f48b069](AztecProtocol/aztec-packages@f48b069))
* Adding aggregation to honk and rollup
([#7466](AztecProtocol/aztec-packages#7466))
([2633aa9](AztecProtocol/aztec-packages@2633aa9))
* **avm:** Pedersen commit in avm
([#7634](AztecProtocol/aztec-packages#7634))
([45e7867](AztecProtocol/aztec-packages@45e7867))
* **avm:** Pedersen commitment sim
([#7632](AztecProtocol/aztec-packages#7632))
([cc420a0](AztecProtocol/aztec-packages@cc420a0))
* **sol-honk:** Integrate solidity honk verifier with bb and acir tests
([#7573](AztecProtocol/aztec-packages#7573))
([344ca6f](AztecProtocol/aztec-packages@344ca6f))
* **sol-honk:** Test verifying recursive proof
([#7576](AztecProtocol/aztec-packages#7576))
([26408c1](AztecProtocol/aztec-packages@26408c1))


### Bug Fixes

* Bb mac build
([#7619](AztecProtocol/aztec-packages#7619))
([e3c5602](AztecProtocol/aztec-packages@e3c5602))


### Miscellaneous

* **bb readme:** Add installation instructions and TODOs
([#7601](AztecProtocol/aztec-packages#7601))
([1a97698](AztecProtocol/aztec-packages@1a97698))
* **bb readme:** Update versioning instructions and add Honk Solidity
verifier commands
([#7608](AztecProtocol/aztec-packages#7608))
([9dd9195](AztecProtocol/aztec-packages@9dd9195))
* Call requests
([#7483](AztecProtocol/aztec-packages#7483))
([ffedf39](AztecProtocol/aztec-packages@ffedf39))
* Constant inputs for most blackboxes
([#7613](AztecProtocol/aztec-packages#7613))
([3247058](AztecProtocol/aztec-packages@3247058))
* Merge Devnet back to Master
([#7611](AztecProtocol/aztec-packages#7611))
([112961c](AztecProtocol/aztec-packages@112961c))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
aakoshh added a commit that referenced this pull request Mar 14, 2025
@aakoshh @ludamad @TomAFrench
feat: Noir sync rework (#12574)
2b555ce 
# Check out Noir during build

Changes the way we sync the code from Noir away from having a
periodically synced copy of the codebase, to checking out the code
during the build based on a tag/branch/commit indicated by a file or env
var.

The following new files were added:
* `noir/noir-repo-ref` contains the reference in Noir that should be
checked out
*
[noir/scripts/sync.sh](https://github.com/AztecProtocol/aztec-packages/blob/noir-sync-rework/noir/scripts/sync.sh)
performs the checkouts; the `init` command does the initial checkout and
the `update` command switches to new branches
* `noir/bootstrap.sh` performs an `init` and `update` before every
command except `clean`

TODO:
- [x] Sync during the build
- [x] Detect the need to switch branches
- [x] Create a patch file from local commits
- [x] Apply the patch file during checkouts 
- [x] Fix patch creation to only include non-empty commits because `git
am` stops at them, and with `git am --allow-empty` it just fails
- [x] Add to the git hooks something to prevent the user from switching
branches without creating and committing a patch file, if there are
commits in `noir-repo`. (Currently an error is returned by `sync.sh` but
if we created a patch file, it should be committed on the aztec-packages
branch we switched away from).\
- [x] Update the daily workflow to update `noir-repo-ref` to the next
release.
- [x] Test the `pull-noir` workflow:
* Action that create the PR:
https://github.com/AztecProtocol/aztec-packages/actions/runs/13768222542
* PR that bumps the ref:
#12608


## Rationale

Currently there is a two-way sync between Noir and aztec-packages, with
automated workflows running daily to push changes between the `master`
branches by opening (or appending to already open) sync PRs. The two-way
sync was required because for example frequent changes in the `bb`
interface were easier to update in this repo and pushed back to Noir,
than to have to wait for a release cycle.

The problem we frequently encounter with the sync from Noir is that some
of the integration tests in aztec-packages break due to subtle changes
in the compiler, which we don't know about until we open the sync PR. At
that point we're looking at a potentially long list of changes (every
commit on `master` since the last sync), and it's not clear which one is
the culprit. We bisect the build to find a failure, fix it in the PR,
but this often just reveals the next bug to be fixed.

This process can take days, and during that time the sync process has to
be paused - if it runs it just makes the PR bigger, with potentially
more bugs. Regardless of whether we pause it or let it append to the PR,
no new features or fixes get merged into aztec-packages until all the
bugs are squashed.

Instead what we wish to achieve here is that we can:
1. Point aztec-packages at an integration/release branch, or nightly
tags to be synced from, and if there is a problem then we can cherry
pick changes from `master` onto the release branch so that the good
stuff can go in.
2. Initiate CI runs against arbitrary branches on Noir, so that we can
test smaller change sets before they are synced to aztec-packages, to
see their effect in isolation.

## Example

The following example shows:
1. Initialising `noir-repo` to a tag
2. Creating a file and commit
3. Trying to switch branch is rejected because the commit is not in the
patch
4. Creating a patch
5. Switching to a branch
6. Creating another commit and adding it to the patch
7. Switching to a commit
8. See both commits are re-applied

```console
% rm -rf noir-repo                                                                                                                                                                                 
% cat noir-repo-ref                                                                                                                                                                                
nightly-2025-03-07
% scripts/sync.sh init                                                                                                                                                                             
Cloning into 'noir-repo'...
...
Note: switching to '37be49fd081f33dc7256d23cee8ccc0719c50a82'.
...
HEAD is now at 37be49fd08 chore!: convert `TraitMethodNotInScope` to error (#7427)
➤ YN0000: ┌ Resolution step
➤ YN0002: │ @algolia/autocomplete-core@npm:1.9.3 doesn't provide @algolia/client-search (pbc410), requested by @algolia/autocomplete-shared
...
➤ YN0000: Done with warnings in 20s 227ms
[detached HEAD 6cee7cd4d6] Noir local fixup commit.
 4 files changed, 8 insertions(+), 9 deletions(-)
[detached HEAD 94cffa4cb1] Noir local patch commit.
% ./scripts/sync.sh info                                                         
Fixup commit:             30d08b921c71edb1c75491e459df6d54917c5e51
Checkout commit:          37be49fd081f33dc7256d23cee8ccc0719c50a82
Wanted:                   nightly-2025-03-07
Needs switch:             no
Needs patch:              no
Detached:                 yes
On branch:                no
Branch name:              n/a
Has wanted tag:           yes
Has tag commit:           yes
Has patch commit:         yes
Last commit is patch:     yes
Has fixup and patch:      yes
% cd noir-repo                                                                                                                                                                                  
% git status                                                                                                                                                                                    
Not currently on any branch.
nothing to commit, working tree clean
% echo "Hello" > hello.txt && git add . && git commit -m "Hello"                                                                                                                                
[detached HEAD 6053b5e04c] Hello
 1 file changed, 1 insertion(+)
 create mode 100644 hello.txt                                                                                                                                                                           
% cd ..                                                                                                                                                                                         
% NOIR_REPO_REF=master scripts/sync.sh update                                                                                                                                                      
noir-repo is on a detached HEAD and the last commit is not the patch marker commit;
switching from nightly-2025-03-07 to master could meand losing those commits.
Please use the 'make-patch' command to create a noir-repo.patch file and commit it in aztec-packages, 
so that it is applied after each checkout; make sure to commit the patch on the branch where it should be.
% scripts/sync.sh make-patch                                                                                                                                                                       
../patches/0001-Noir-local-patch-commit.patch
../patches/0002-Hello.patch
[detached HEAD 28f53e097b] Noir local patch commit.
% NOIR_REPO_REF=master scripts/sync.sh update                                                                                                                                                      
...
From https://github.com/noir-lang/noir
 * branch                  master     -> FETCH_HEAD
Warning: you are leaving 4 commits behind, not connected to
any of your branches:

  28f53e097b Noir local patch commit.
  6053b5e04c Hello
  94cffa4cb1 Noir local patch commit.
  6cee7cd4d6 Noir local fixup commit.

If you want to keep them by creating a new branch, this may be a good time
to do so with:

 git branch <new-branch-name> 28f53e097b

branch 'master' set up to track 'origin/master'.
Switched to a new branch 'master'
Already up to date.
➤ YN0000: ┌ Resolution step
➤ YN0002: │ @algolia/autocomplete-core@npm:1.9.3 doesn't provide @algolia/client-search (pbc410), requested by @algolia/autocomplete-shared
...
➤ YN0000: Done with warnings in 2s 686ms
[master 7cee0c05bb] Noir local fixup commit.
 4 files changed, 8 insertions(+), 9 deletions(-)
Applying: Hello
[master ad6812fae7] Noir local patch commit.
% cd noir-repo                                                                                                                                                                                  
% echo "World" > world.txt && git add . && git commit -m "World"                                                                                                                                    
[master 16f0054dc9] World
 1 file changed, 1 insertion(+)
 create mode 100644 world.txt
% cd ..                                                                                                                                                                                                                                                                                                                                                                  
% cat .noir-repo-last-ref                                                                                                                                                                          
master                                                                                                                                                          
% scripts/sync.sh make-patch                                                                                                                                                                       
../patches/0001-Hello.patch
../patches/0002-Noir-local-patch-commit.patch
../patches/0003-World.patch
[master 2cc18fd434] Noir local patch commit.
% cat noir-repo.patch                                                                                                                                                                              
From 6a8c19cd6fd70e4b982ec8ca10b4b70363636f97 Mon Sep 17 00:00:00 2001
From: aakoshh <akosh@aztecprotocol.com>
Date: Fri, 7 Mar 2025 23:46:36 +0000
Subject: [PATCH 1/3] Hello

---
 hello.txt | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 hello.txt

diff --git a/hello.txt b/hello.txt
new file mode 100644
index 0000000000..e965047ad7
--- /dev/null
+++ b/hello.txt
@@ -0,0 +1 @@
+Hello
-- 
9.43.0

From 16f0054dc961105cdb264d6ace10ff9061b7b96d Mon Sep 17 00:00:00 2001
From: aakoshh <akosh@aztecprotocol.com>
Date: Fri, 7 Mar 2025 23:49:19 +0000
Subject: [PATCH 3/3] World

---
 world.txt | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 world.txt

diff --git a/world.txt b/world.txt
new file mode 100644
index 0000000000..216e97ce08
--- /dev/null
+++ b/world.txt
@@ -0,0 +1 @@
+World
-- 
2.43.0

% export NOIR_REPO_REF=a9de769f3743733ac754a5d6c4e147ba8283a336                                                                                                                                    
% scripts/sync.sh update                                                                                                                                                                           
...
From https://github.com/noir-lang/noir
 * branch                  a9de769f3743733ac754a5d6c4e147ba8283a336 -> FETCH_HEAD
Note: switching to 'a9de769f3743733ac754a5d6c4e147ba8283a336'.
...

HEAD is now at a9de769f37 feat: add optional oracle resolver url in `acvm_cli` (#7630)
➤ YN0000: ┌ Resolution step
...
➤ YN0000: └ Completed in 1s 219ms
➤ YN0000: Done with warnings in 2s 643ms
[detached HEAD 53cbe7bae4] Noir local fixup commit.
 4 files changed, 8 insertions(+), 9 deletions(-)
Applying: Hello
Applying: World
[detached HEAD c4921bc4a8] Noir local patch commit.
% ls noir-repo/*.txt                                                                                                                                                                           16s 
noir-repo/hello.txt  noir-repo/world.txt
%                                                                                                                                                                                                  

```

---------

Co-authored-by: ludamad <adam.domurad@gmail.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PhilWindle PhilWindle PhilWindle approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@spalladino @AztecBot @PhilWindle