fix!: AVM was missing range check on remainder for div in ALU

[dbanks12]

Cherry-picked from v4 PR: #21066

The Bug

The AVM's ALU division circuit (alu.pil) constrains integer division a / b = c with remainder r (stored in helper1) via:

DIV_OPS_NON_U128 * (ib * ic - ia + sel_op_div * helper1) = 0;

This enforces b * c + r = a (i.e., a - r = b * c). A separate greater-than check ensures r < b.

However, the remainder helper1 was never range-checked to fit within the operand's bit-width (max_bits). A malicious prover could set helper1 to an arbitrarily large field element (e.g., close to the field modulus) that satisfies b * c + r = a (mod p) and r < b in the field, but does not represent a valid unsigned integer remainder. This would allow forging incorrect division results.

The Exploit

For non-u128 integer division (u8, u16, u32, u64), a prover can choose a remainder r that is a large field element (not a valid unsigned integer) but still satisfies:

1. b * c + r ≡ a (mod p) — the main division relation
2. r < b — the greater-than check (which operates over the full field)

Without a range check on r, the prover can manipulate the quotient c to be any value they want, completely breaking the soundness of integer division.

The Fix

Added a lookup-based range check on the remainder:

#[RANGE_CHECK_DIV_REMAINDER]
sel_div_no_err { helper1, max_bits } in range_check.sel_alu { range_check.value, range_check.rng_chk_bits };

This ensures helper1 (the remainder) fits within max_bits (e.g., 32 bits for u32), preventing the prover from using oversized field elements as the remainder.

The simulation (alu.cpp) was also updated to emit the corresponding assert_range event, and unit tests were updated to expect the new range check call.

[dbanks12]

• fix!: AVM was missing range check on remainder for div in ALU #21074 👈 (View in Graphite)
• merge-train/avm

This stack of pull requests is managed by Graphite. Learn more about stacking.