From 0f5fe134c49bf486a58da6d951267ad547eeaafb Mon Sep 17 00:00:00 2001 From: Jonathan Hao Date: Fri, 4 Jul 2025 09:14:08 +0000 Subject: [PATCH 1/3] chore: readme for benchmarking remotely --- barretenberg/cpp/scripts/README.md | 33 ++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 barretenberg/cpp/scripts/README.md diff --git a/barretenberg/cpp/scripts/README.md b/barretenberg/cpp/scripts/README.md new file mode 100644 index 000000000000..c44d9c9a0bae --- /dev/null +++ b/barretenberg/cpp/scripts/README.md @@ -0,0 +1,33 @@ +# Benchmark Barretenberg Remotely + +## Why +Aztec engineers usually share computing resources on a mainframe. Its CPU load varies which adds noise to benchmark results. + +## Who +To eliminate such noise, we have a machine dedicated to run benchmarks for Aztec engineers. + +## What +There are scripts that: +1. Build your target locally, so you do not need to switch branches remotely. +2. `scp` your target to the remote machine. +3. Run your benchmark command remotely. +4. `scp` the benchmark results, usually in json, back to your local machine. +5. Maybe analyze the results. + +## Setup & Troubleshooting +1. Ask a crypto eng team member for the private key. Put the key in your mainframe home like `/mnt/user-data//remote-bb-worker.pem`. Set the permission by `chmod 600 /mnt/user-data//remote-bb-worker.pem`. + - Putting the key in your home named like this might allow it to be automatically updated when the machine is reset. +2. Add these to your `~/.zshrc` and restart `zsh`. + - `export BB_SSH_KEY=-i/mnt/user-data//remote-bb-worker.pem` + - `export BB_SSH_INSTANCE=` + - `export BB_SSH_CPP_PATH=/home/ubuntu/aztec-packages/barretenberg/cpp` +3. Try `ssh $BB_SSH_KEY $BB_SSH_INSTANCE`. + - If it times out, double check your `$BB_SSH_INSTANCE` with another team member. + - If you get a permission denied, double check your private key with another team member. + - If it says the private key is too public, make sure you did `chmod 600 /mnt/user-data//remote-bb-worker.pem`. You should see `-rw-------` if you `ls -al` the key. +4. If `ssh` worked, the setup is complete. + +## How +- `./scripts/benchmark_client_ivc.sh` lets you run `client_ivc_bench` remotely and analyze the results. +- `./scripts/benchmark_example_ivc_flow_remote.sh` copies the example flow input you'd like to run to the remote machine, runs `bb_cli_bench`, and analyze the results. +- If you have other special needs, look inside the above scripts and see what parameters you can give, or use `./scripts/benchmark_remote.sh` From 17850b375db0dbb78d12c871fe7ceba17f2ed8bc Mon Sep 17 00:00:00 2001 From: Jonathan Hao Date: Fri, 4 Jul 2025 12:27:13 +0000 Subject: [PATCH 2/3] replace all aztec engineers with crypto eng --- barretenberg/cpp/scripts/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/barretenberg/cpp/scripts/README.md b/barretenberg/cpp/scripts/README.md index c44d9c9a0bae..16b8a2d77394 100644 --- a/barretenberg/cpp/scripts/README.md +++ b/barretenberg/cpp/scripts/README.md @@ -4,7 +4,7 @@ Aztec engineers usually share computing resources on a mainframe. Its CPU load varies which adds noise to benchmark results. ## Who -To eliminate such noise, we have a machine dedicated to run benchmarks for Aztec engineers. +To eliminate such noise, we have a machine dedicated to run benchmarks for the Aztec cryptography engineering team. ## What There are scripts that: From 58f33e3cdeed7128ab7dd2a45df73b1ac2bcd45f Mon Sep 17 00:00:00 2001 From: Jonathan Hao Date: Tue, 8 Jul 2025 16:22:21 +0000 Subject: [PATCH 3/3] address review comments --- barretenberg/cpp/scripts/README.md | 3 ++- barretenberg/cpp/scripts/benchmark_remote.sh | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/barretenberg/cpp/scripts/README.md b/barretenberg/cpp/scripts/README.md index 16b8a2d77394..b70be89e0e6a 100644 --- a/barretenberg/cpp/scripts/README.md +++ b/barretenberg/cpp/scripts/README.md @@ -30,4 +30,5 @@ There are scripts that: ## How - `./scripts/benchmark_client_ivc.sh` lets you run `client_ivc_bench` remotely and analyze the results. - `./scripts/benchmark_example_ivc_flow_remote.sh` copies the example flow input you'd like to run to the remote machine, runs `bb_cli_bench`, and analyze the results. -- If you have other special needs, look inside the above scripts and see what parameters you can give, or use `./scripts/benchmark_remote.sh` + - For the script to work you need to have the example flows downloaded locally, by `AZTEC_CACHE_COMMIT=origin/next~3 DOWNLOAD_ONLY=1 yarn-project/end-to-end/bootstrap.sh build_bench` +- If you have other special needs, look inside the above scripts and see what parameters you can give, or use `./scripts/benchmark_remote.sh`. diff --git a/barretenberg/cpp/scripts/benchmark_remote.sh b/barretenberg/cpp/scripts/benchmark_remote.sh index c2240f07f4fa..3c32180f1608 100755 --- a/barretenberg/cpp/scripts/benchmark_remote.sh +++ b/barretenberg/cpp/scripts/benchmark_remote.sh @@ -9,7 +9,7 @@ set -eu BENCHMARK=${1:-client_ivc_bench} COMMAND=${2:-./$BENCHMARK} -PRESET=${3:-clang16} +PRESET=${3:-clang16-no-avm} BUILD_DIR=${4:-build} HARDWARE_CONCURRENCY=${HARDWARE_CONCURRENCY:-16}