fix(ci): run contract-snapshots tests in CI#23895
Merged
Merged
Conversation
The contract-snapshots suite (added in #23061) was only wired into the noir-projects/bootstrap.sh test_cmds aggregate, which nothing in the Makefile-driven CI invokes, so the suite has never run in CI. This let the noir 1.0.0-beta.22 bump (#23870) land green with stale snapshots, and aztec-nr changes since #23464 have drifted the snapshots unnoticed: 14 of 61 tests currently fail on this branch with its own pinned nargo. Wire the suite in as its own Makefile target (mirroring noir-protocol-circuits-tests), add it to the fast target, and mark the bootstrap aggregate as local-dev only.
nchamo
approved these changes
Jun 5, 2026
Green half of the demo for #23895. The committed snapshots had drifted from aztec-nr changes landed since the last regen in #23464, unnoticed because CI never ran the suite. Regenerated with the branch's pinned nargo (1.0.0-beta.21+f1a4575a); all 61 tests pass locally. With the CI wiring from #23895 included in this branch, the contract-snapshots suite runs in CI here and should be green, in contrast to the red run on #23895 itself. Merge order: this PR into `mv/run-contract-snapshots-in-ci`, then #23895 into `merge-train/fairies-v5`. Note: #23878 (regen for the beta.22 bump) stays parked as draft, since the bump was reverted on v5-next in 9e53e1f. --- Update: the beta.22 bump is back on the train via #23886, so this branch now regenerates against 1.0.0-beta.22 with two more commits: - chore: scrub macro-source line numbers in contract failure snapshots. Locations inside the aztec-nr macro sources (call stack `at` lines and code frame gutter numbers under `<repo>` headers) are scrubbed to `<line>`/`<col>` markers, like the existing `<repo>` path marker. Macro edits previously churned every compile_failure snapshot with pure line number diffs (see `authorize_once_from_wrong_type` in #23878 for an example). Locations in the test program itself and in the stdlib are kept. Resolves #23878 (comment) - chore: regenerate expand snapshots for noir 1.0.0-beta.22. `nargo expand` output changed with the bump. All 61 tests pass locally with the branch's pinned nargo (1.0.0-beta.22). --------- Co-authored-by: AztecBot <tech@aztec-labs.com>
- invalid_event stderr trace: messages/message_delivery.nr -> messages/delivery/mod.nr (leftover from #23875) - 5 expand snaps: insta header format bump (legacy -> current), no content change
vezenovm
removed
ci-draft
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
This draft exists just so display the first half of a red-green flow. We should merge the child PR #23896 into this one before merging.
The contract-snapshots suite (added in #23061) has never run in CI. It was wired only into the
noir-projects/bootstrap.sh test_cmdsaggregate, but the Makefile-driven CI never invokes parent aggregates: it calls each subproject'stest_cmdsdirectly (noir-protocol-circuits-testsinfast,noir-projects-txe-testspost-build). The aggregate is a local-dev entry point only.Consequences so far:
The fix
contract-snapshots-testsMakefile target (mirrorsnoir-protocol-circuits-tests, depends only onnoirfor the nargo binary), included infast.Expected CI result
This PR is intentionally red: the newly wired suite fails with the 14 drifted snapshots, demonstrating the gap. A follow-up PR targeting this branch regenerates the snapshots and turns it green. Merge order: regen PR into this branch, then this branch into the train.
Draft with
ci-draftlabel so CI runs while merge automation stays off until the regen lands.