feat: fix contract issues with key rotation, introduce get npk_m_hash_at by sklppy88 · Pull Request #6541 · AztecProtocol/aztec-packages

sklppy88 · 2024-05-20T17:30:46Z

Resolves #6312

sklppy88 · 2024-05-20T17:31:05Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sklppy88 and the rest of your teammates on Graphite

AztecBot · 2024-05-20T21:59:27Z

Benchmark results

No base data found for comparison.

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric	1 threads	4 threads	16 threads	32 threads	64 threads
proof_construction_time_sha256	5,643	1,531	709	762	776

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	772	772	772
l1_rollup_calldata_gas	6,856	6,868	6,868
l1_rollup_execution_gas	587,396	587,408	587,408
l2_block_processing_time_in_ms	1,403	5,161	10,238
l2_block_building_time_in_ms	33,697	133,278	265,539
l2_block_rollup_simulation_time_in_ms	33,514	132,623	264,263
l2_block_public_tx_process_time_in_ms	20,664	83,720	167,184

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	15,351	28,545
node_database_size_in_bytes	21,282,896	38,273,104
pxe_database_size_in_bytes	29,868	59,425

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit	protocol_circuit_simulation_time_in_ms	protocol_circuit_witness_generation_time_in_ms	protocol_circuit_proving_time_in_ms	protocol_circuit_input_size_in_bytes	protocol_circuit_output_size_in_bytes	protocol_circuit_proof_size_in_bytes	protocol_circuit_num_public_inputs	protocol_circuit_size_in_gates
private-kernel-init	162	3,760	21,879	19,985	61,999	86,720	2,643	1,048,576
private-kernel-inner	610	4,973	40,419	89,053	61,999	86,720	2,643	2,097,152
private-kernel-reset-small	578	2,403	23,139	117,961	61,999	86,720	2,643	1,048,576
private-kernel-tail	535	3,823	38,017	86,849	79,454	10,688	267	2,097,152
base-parity	7.64	897	3,482	128	64.0	2,208	2.00	131,072
root-parity	50.2	508	41,051	27,064	64.0	2,720	18.0	2,097,152
base-rollup	812	2,821	43,344	111,158	957	3,136	31.0	2,097,152
root-rollup	94.2	110	8,332	11,518	821	3,456	41.0	524,288
public-kernel-app-logic	248	212	746	96,978	85,095	116,448	3,572	4,096
public-kernel-tail	878	798	1,134	388,207	7,755	10,176	251	512
public-kernel-setup	229	260	884	138,309	85,095	116,448	3,572	4,096
public-kernel-teardown	229	270	1,049	143,320	85,095	116,448	3,572	4,096
merge-rollup	6.53	82.7	1,616	2,760	957	3,136	31.0	65,536
private-kernel-tail-to-public	N/A	9,871	72,830	N/A	N/A	116,960	3,588	4,194,304

Stats on running time collected for app circuits

Function	app_circuit_proof_size_in_bytes	app_circuit_proving_time_in_ms	app_circuit_size_in_gates	app_circuit_num_public_inputs
SchnorrAccount:entrypoint	16,128	46,860	2,097,152	437
Test:emit_nullifier	16,128	2,789	65,536	437
FPC:fee_entrypoint_public	16,128	16,704	524,288	437
FPC:fee_entrypoint_private	16,128	8,871	524,288	437
Token:unshield	16,128	49,839	2,097,152	437
SchnorrAccount:spend_private_authwit	16,128	2,671	131,072	437
Token:transfer	16,128	34,932	2,097,152	437

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves	32 leaves
batch_insert_into_append_only_tree_16_depth_ms	11.5	18.6	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.7	31.8	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.668	0.570	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	52.9	83.8	271	507	1,003	1,983	N/A
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	543	1,055	2,079	4,127	N/A
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.541	0.517	0.491	0.473	0.475	0.473	N/A
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	63.3	120	385	745	1,499	2,980	N/A
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	106	208	692	1,363	2,707	5,395	N/A
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.550	0.541	0.522	0.512	0.518	0.518	N/A
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	68.4
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	108
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.599

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	84,741	673,219

Transaction size based on fee payment method

| Metric | |
| - | |

Transaction processing duration by data writes.

Metric	0 new note hashes	1 new note hashes	2 new note hashes
tx_pxe_processing_time_ms	29,701	4,283	100,599

Metric	0 public data writes	1 public data writes	2 public data writes	4 public data writes	8 public data writes
tx_sequencer_processing_time_ms	1,416	2,687	2,103	3,302	2,007

sklppy88 · 2024-05-21T02:17:22Z

 use crate::{context::PrivateContext, oracle::get_public_data_witness::get_public_data_witness};

-fn _public_storage_historical_read(storage_slot: Field, contract_address: AztecAddress, header: Header) -> Field {
+pub fn _public_storage_historical_read(storage_slot: Field, contract_address: AztecAddress, header: Header) -> Field {


I've exposed this to access this by passing in a header directly. One can make an argument that to keep it consistent with all the other changes I've made, that actually the public_storage_historical_read_at function below should change and take a header as well, and not only a block number; but I wanted to keep changes minimal. Open to opinions though !

I think exposing this is totally fine but would rename it to something without the underscore prefix since that evokes private/internal function.

sklppy88 · 2024-05-21T02:23:10Z

    key_index: Field
 ) -> GrumpkinPoint {
-    let key = fetch_key_from_registry(context, key_index, address);
+    let (x_coordinate_registry, y_coordinate_registry) = get_registry_with_key_index(context, key_index, address);


Reorganized some things here

The split now made here such that you can use the _at, right?

See comment above, I think we can make some simplifications to get rid of some of all the _at case we have which make the code harder to read at the moment.

sklppy88 · 2024-05-21T02:32:35Z

+    y_coordinate_registry: SharedMutablePrivateGetter<Field, DELAY>,
+    header: Header
+) -> GrumpkinPoint {
+    let key_x_coordinate = x_coordinate_registry.get_current_value_in_private_at(header.global_variables.block_number as u32, header);


Using get_current_value_in_private_at as opposed to get_value_in_private_at just for consistency, also to disambiguate between scheduled / current.

sklppy88 · 2024-05-21T02:35:32Z

        for i in 0..3 {
-            raw_value_change_fields[i] = public_storage_historical_read(
-                    context,
+            raw_value_change_fields[i] = _public_storage_historical_read(


Made this change because otherwise every time we wanted to read a historical value, we'd have to refetch the header :/. See this comment.

I'm not head over heals about the _at we got, suggested an idea earlier in this and in the team channel, we might want to deal with that first to not mess with these more than necessary.

sklppy88 · 2024-05-21T02:39:35Z

        }

        let delay_change_slot = self.get_delay_change_storage_slot();
-        let raw_delay_change_fields = [public_storage_historical_read(context, delay_change_slot, context.this_address())];


Fixing context.this_address() as it seems to be a mistake.

It's weird that this was not caught by a test. I see that it's used in the test contract to read from key registry so it's not the case that it was not caught because it was used to read from the same contract in a test.

Given that it's reading delay I would say we never test the case when a delay is non-default. Could you create a separate PR based on master in which you:

Create a test which fails because of this issue and submit it in 1 commit (this way we verify that the test is good),

fix the issue in another commit (this way we verify that it was addressed).

Thanks

sklppy88 · 2024-05-21T02:50:32Z

    ) {
-        let owner_npk_m_hash = get_npk_m_hash(&mut context, owner);
-
-        // TODO (#6312): This will break with key rotation. Fix this. Will not be able to find any notes after rotating key.


Not needed as private_values is scoped.

sklppy88 · 2024-05-21T02:51:13Z

    ({ teardown, wallet, logger } = await setup(1));
    owner = wallet.getAddress();

+    testContract = await TestContract.deploy(wallet).send().deployed();


Adding this for access to cross delay. Not the best, but just reusing the pattern.

We should probably get the cheatcode going #3168

sklppy88 · 2024-05-21T03:04:11Z

    fn private_get_value(amount: Field, owner: AztecAddress) -> Field {
-        let owner_npk_m_hash = get_npk_m_hash(&mut context, owner);
-
-        // TODO (#6312): This will break with key rotation. Fix this. Will not be able to find any notes after rotating key.


No longer needed because the set is scoped to owner.

sklppy88 · 2024-05-21T03:05:09Z

@@ -7,17 +7,12 @@ use dep::aztec::note::note_getter_options::{Sort, SortOrder};
 // Shows how to use NoteGetterOptions and query for notes.


The code in this file does not seem to be used anywhere and is for demonstration purposes only. Nothing actually was broken because account_npk_m_hash was passed in directly. I've made changes though because I think in general this pattern of filtering with npk_m_hash is not great.

sklppy88 · 2024-05-21T03:06:53Z

 ) -> NoteGetterOptions<CardNote, CARD_NOTE_LEN, Field> {
-    // TODO (#6312): This will break with key rotation. Fix this. Will not be able to find any notes after rotating key.
    let mut options = NoteGetterOptions::new();
    options.select(CardNote::properties().points, points as Field, Option::none()).select(CardNote::properties().randomness, secret, Option::none()).select(


I kept this one as is to show that all fields can be simultaneously filtered on (name of test).

benesjan · 2024-05-22T06:58:03Z

 use crate::{context::PrivateContext, oracle::get_public_data_witness::get_public_data_witness};

-fn _public_storage_historical_read(storage_slot: Field, contract_address: AztecAddress, header: Header) -> Field {
+pub fn _public_storage_historical_read(storage_slot: Field, contract_address: AztecAddress, header: Header) -> Field {


I think exposing this is totally fine but would rename it to something without the underscore prefix since that evokes private/internal function.

benesjan · 2024-05-22T07:02:33Z

 }

+// For historical access, we move the fetching of the header to the top level to minimize redundant fetching / proving validity of the header.
+pub fn get_npk_m_at(


This function is only called in get_npk_m_hash_at so would merge it with that to not have a huge amount of variations of functions.

benesjan · 2024-05-22T08:39:11Z

        }

        let delay_change_slot = self.get_delay_change_storage_slot();
-        let raw_delay_change_fields = [public_storage_historical_read(context, delay_change_slot, context.this_address())];


It's weird that this was not caught by a test. I see that it's used in the test contract to read from key registry so it's not the case that it was not caught because it was used to read from the same contract in a test.

Given that it's reading delay I would say we never test the case when a delay is non-default. Could you create a separate PR based on master in which you:

Create a test which fails because of this issue and submit it in 1 commit (this way we verify that the test is good),

fix the issue in another commit (this way we verify that it was addressed).

Thanks

benesjan · 2024-05-22T08:40:25Z

        if opt_notes[i].is_some() {
            let note = opt_notes[i].unwrap_unchecked();

-            // This is similar to destroy_note, except we only compute the owner_npk_m_hash once instead of doing it in


benesjan · 2024-05-22T08:41:22Z

-    // TODO (#6312): This will break with key rotation. Fix this. Will not be able to pass this after rotating keys.
-    assert(note.npk_m_hash.eq(owner_npk_m_hash));
-
+pub fn destroy_note(balance: PrivateSet<ValueNote, &mut PrivateContext>, note: ValueNote) -> Field {


This is cute

benesjan · 2024-05-22T08:48:49Z

            if maybe_notes[i].is_some() {
                let note = maybe_notes[i].unwrap_unchecked();

-                // Ensure the notes are actually owned by the owner (to prevent user from generating a valid proof while


But we don't have a guarantee that EasyPrivateUInt will only ever be used in maps, no?

I feel like the correct justification for removing this in here is that the notes inside the EasyPrivateUInt is protected by nullifier key and hence can't be nullified by "non-owner".

benesjan · 2024-05-22T08:51:48Z

                let card_note = CardNote::from_note(maybe_notes[i].unwrap_unchecked());
-                // Ensure the notes are actually owned by the owner (to prevent user from generating a valid proof while
-                // spending someone else's notes).
-                // TODO (#6312): This will break with key rotation. Fix this. Will not be able to pass this assert after rotating keys.


Once again I feel like the main protection here is the nullifier key as relying on Maps for protection just seems scary to me. @LHerskind or WDYT about using Maps as a guardrail?

benesjan · 2024-05-22T09:01:07Z

 }

-fn fetch_key_from_registry(
+fn get_registry_with_key_index(


I don't like much that we need to have this getter and then split the flows into fetch_key_from_registry and fetch_key_from_registry_at but I don't really now how to refactor it so will shut up for now.

But I think a better name would help here because when I saw the function I though it fetches some value from registry. I think something like instantiate_registry_getter would be a better name.

Update: Boss came up with how to refactor it here.

benesjan · 2024-05-22T09:07:29Z

    #[aztec(private)] // annotation to mark function as private and expose private context
    fn cast_vote(candidate: Field) {
-        let msg_sender_npk_m_hash = get_npk_m_hash(&mut context, context.msg_sender());
-        // TODO (#6312): This will break with key rotation. Fix this. Can vote multiple times by rotating keys.


Please document how the protection against nullifier key rotation double spend works here. The contracts are used as an example so it's very valuable to share this info.

LHerskind

Generally I think the pr is fine, but realised that we can likely handle the _at cases more cleanly than we are now which seems like something that could be beneficial to deal with before this one goes in since we don't need to add to then instantly remove it afterwards (you can add a new parent to this in the stack for example).

For that reason, I think we need to address #6589 and then merge this pr.

LHerskind · 2024-05-22T09:04:55Z

+    block_number: u32,
+    header: Header
+) -> GrumpkinPoint {
+    assert_eq(block_number, header.global_variables.block_number as u32);


This feels strange, why do you want to pass the block_number if it is the same value as we have in the header?

Just a small thing that popped up in my mind. As we seem to generally want to supply multiple ways to execute these functions on the header, should we take a look at having them as a trait on the header?

In that way, you would get the header that you are interested in for data, and then you can use the "getters".

let header = context.get_header_at(block_number); let npk_m = header.get_npk_m(context, address); // and for historical access to public state let storage = header.public_storage_historical_read(address, storage_slot);

@nventuro what are you thoughts? Should allow us to get rid of some code and think it is similar or better ux, but your call 🫡

If to change like this, might be better to handle in separate PR since we have the flow a few places that was otherwise untouched here. Think this could essentially get rid of almost all of the _at function we got for time, just keep the get_header_at.

LHerskind · 2024-05-22T09:08:36Z

    key_index: Field
 ) -> GrumpkinPoint {
-    let key = fetch_key_from_registry(context, key_index, address);
+    let (x_coordinate_registry, y_coordinate_registry) = get_registry_with_key_index(context, key_index, address);


The split now made here such that you can use the _at, right?

See comment above, I think we can make some simplifications to get rid of some of all the _at case we have which make the code harder to read at the moment.

LHerskind · 2024-05-22T09:12:23Z

-    let y_coordinate = y_coordinate_registry.get_current_value_in_private();

-    GrumpkinPoint::new(x_coordinate, y_coordinate)
+    (x_coordinate_registry, y_coordinate_registry)


The wording here makes it sound like it is separate registries, but it is just different coordinates so think I would still keep the naming similar to not make misunderstandings that it is a registry 🤷

LHerskind · 2024-05-22T09:23:55Z

        for i in 0..3 {
-            raw_value_change_fields[i] = public_storage_historical_read(
-                    context,
+            raw_value_change_fields[i] = _public_storage_historical_read(


I'm not head over heals about the _at we got, suggested an idea earlier in this and in the team channel, we might want to deal with that first to not mess with these more than necessary.

LHerskind · 2024-05-22T09:42:06Z

-        let msg_sender_npk_m_hash = get_npk_m_hash(&mut context, context.msg_sender());
-        // TODO (#6312): This will break with key rotation. Fix this. Can vote multiple times by rotating keys.
+        let active_at_block = storage.active_at_block.read_private();
+        assert(


I'm not sure that you would could end up in this one failing? Namely, to call this function you have that the contract must be deployed and since it must be initialized before letting you call here think we cannot hit it? 🤔

Regardless, I think you will already have this handled from the get_header_at? 🤔

LHerskind · 2024-05-22T09:44:15Z

    const anotherDonationNote = await processExtendedNote(notes![0]);

-    // 3) We claim the reward token via the Claim contract
+    // We create an unrelated pxe and wallet without access to the nsk_app that correlates to the npk_m specified in the proof note.


This is neat, I'm gonna steal this. 👀

LHerskind · 2024-05-22T09:46:02Z

    ({ teardown, wallet, logger } = await setup(1));
    owner = wallet.getAddress();

+    testContract = await TestContract.deploy(wallet).send().deployed();


We should probably get the cheatcode going #3168

…rotation

sklppy88 · 2024-05-24T13:03:59Z

Have tried a different approach, and have addressed feedback in this stack

sklppy88 · 2024-05-24T17:23:42Z

Closed in favor of #6656

sklppy88 changed the title ~~get npk_m_hash_at~~ feat: fix contract issues with key rotation get npk_m_hash_at May 20, 2024

sklppy88 changed the title ~~feat: fix contract issues with key rotation get npk_m_hash_at~~ feat: fix contract issues with key rotation, introduce get npk_m_hash_at May 20, 2024

sklppy88 changed the base branch from master to ek/refactor/refactor-key-rotate-call-and-address-comments-from-6405 May 20, 2024 17:40

sklppy88 force-pushed the ek/fix/logic-issues-in-contracts-with-key-rotation branch from 06517a5 to 3d81929 Compare May 20, 2024 20:32

sklppy88 commented May 21, 2024

View reviewed changes

sklppy88 mentioned this pull request May 21, 2024

refactor: refactor key rotate and address comments from 6405 #6450

Merged

sklppy88 linked an issue May 21, 2024 that may be closed by this pull request

refactor(Keys_Nullifier): Fix security issues / broken contract logic from key rotation #6312

Closed

sklppy88 requested review from LHerskind, benesjan and nventuro May 21, 2024 13:55

sklppy88 marked this pull request as ready for review May 21, 2024 13:56

Base automatically changed from ek/refactor/refactor-key-rotate-call-and-address-comments-from-6405 to master May 21, 2024 14:12

sklppy88 force-pushed the ek/fix/logic-issues-in-contracts-with-key-rotation branch from c90cba0 to 385ef25 Compare May 21, 2024 20:08

benesjan reviewed May 22, 2024

View reviewed changes

LHerskind mentioned this pull request May 22, 2024

refactor(HistoricalData): Align usage flows to get rid of _at #6589

Closed

LHerskind reviewed May 22, 2024

View reviewed changes

Initial

276b08c

sklppy88 force-pushed the ek/fix/logic-issues-in-contracts-with-key-rotation branch from 40053c2 to 276b08c Compare May 22, 2024 19:08

sklppy88 requested review from IlyasRidhuan, Maddiaa0, dbanks12, fcarreiro and jeanmon as code owners May 22, 2024 19:08

sklppy88 removed request for IlyasRidhuan, Maddiaa0, dbanks12, fcarreiro and jeanmon May 22, 2024 19:11

Merge branch 'master' into ek/fix/logic-issues-in-contracts-with-key-…

6e5ec03

…rotation

sklppy88 marked this pull request as draft May 24, 2024 13:04

sklppy88 closed this May 24, 2024

sklppy88 deleted the ek/fix/logic-issues-in-contracts-with-key-rotation branch June 3, 2024 16:05

		@@ -7,17 +7,12 @@ use dep::aztec::note::note_getter_options::{Sort, SortOrder};
		// Shows how to use NoteGetterOptions and query for notes.

Conversation

sklppy88 commented May 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sklppy88 commented May 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AztecBot commented May 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark results

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

Uh oh!

sklppy88 May 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LHerskind left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sklppy88 commented May 20, 2024 •

edited

Loading

sklppy88 commented May 20, 2024 •

edited

Loading

AztecBot commented May 20, 2024 •

edited

Loading

sklppy88 May 21, 2024 •

edited

Loading