intrinsic-test: sve support

[davidtwco]

This needs rust-lang/rust#158088 to be in nightly for CI to pass - it is already approved and in the bors queue. Parts of this have been split out in #2169, #2170, #2171, #2172, #2172 and #2173.

This patch contains all the changes necessary for intrinsic-test to generate tests for the SVE intrinsics:

• Some of the commits are just general refactors to the tool, motivated by the commits that follow and that I felt were hard to justify on their own
  • All of the other refactorings have already been landed (intrinsic-test: sve type parsing and constraints #2121, intrinsic-test: run rustfmt on generated sources #2122, intrinsic-test: add lots of documentation #2124, intrinsic-test: refactor populate_random and related code #2126, intrinsic-test: remove arm::argument module #2127)
• Some of the commits are small changes I made along the way (e.g. forwarding args from intrinsic-test.sh to cargo test)
  • I could split these out if we wanted
  • Merged in intrinsic-test: fwd args in intrinsic-test.sh #2164
• Some of the commits modify the definitions of intrinsics so that they pass tests - this only happened for a handful of intrinsics and the changes required to their definitions were very minor
  • I could split these out if we wanted
  • Merged in core_arch: redefine svrev, svzip and svuzp #2163
• Some of the commits update arm_intrinsics.json to add some additional constraints necessary to generate the right tests (we're making sure that these addl. constraints are reflected in the source of truth we use to generate that file internally)
  • I could split these out if we wanted
  • Merged in update arm_intrinsics.json for svset and svget #2162
• Any remaining commits are changes to make the tool generate the correct tests, primarily:
  • Adding the arm_sve.h headers and appropriate target feature flags for C and Rust
  • Using the svld1 intrinsic when loading an argument of scalable vector type
    • There is a special case for svbool_t arguments which do not have an svld1 intrinsic, we load a svint8_t and use svcmpne_n_s8 against zero to produce a svbool_t from it
  • Generating test arrays for intrinsics that take boolean values and svbool_t values (though the earlier commits use the same svptrue_$ty predicate for these arguments initially)
  • Supporting calls to enum-typed const generic arguments (such as svpattern), using simple const functions that map integers from intrinsic-test constraints to the appropriate variant
  • Generating a local containing a predicate that is used by all intrinsics which require a predicate (just svptrue_$ty() to enable every lane)
  • Using svcmpeq_$ty and svptest_any to compare the Rust and C results instead of assert_eq!
    • There is a special-case for tuples of vectors, where multiple svcmpeq_$ty and svptest_any blocks are generated for each vector in the tuple, with svgetN calls to extract the appropriate vector from the result tuples
    • There is also a special-case for intrinsics that return svbool_t, where there is no equivalent svcmpeq_$ty call, so sveor_b_z (exclusive OR) and !svptest_any is used
    • There is yet another special case for scalable vectors of floats, where the pre-existing NanEqF* type cannot be used, so a (rust == c) || (isnan(rust) && isnan(c)) check is performed instead, via a handful of SVE intrinsics
• I haven't been been able to test this on x86 to check that I haven't broken the test generation, but I don't think I've made any changes to the generated output that weren't in the Arm-specific parts

[sayantn]

Is there some way to print the diff for SVE? I would imagine it is difficult due to SVE being non-const sized, but is there some way? That would massively improve debuggability. One approach I can suggest is have some small functions that convert from SVE vectors to &[T], e.g.

#[target_feature(enable = "sve")]
pub fn svfloat32_to_slice(a: &svfloat32_t) -> &[NanEqF32] {
    unsafe {
        core::slice::from_raw_parts(core::ptr::from_ref(a).cast(), svcntw() as usize)
    }
}

this might work, with significantly less complexity

View changes since the review

[davidtwco]

When I've needed to debug them, I've disabled the invocation of intrinsic-test in intrinsic-test.sh and added these snippets to the generated tests that I wanted to debug.

For non-bool vectors, just replace the x0_val and x1_val with the vectors that you want to inspect, e.g. __rust_return_value or __c_return_value:

{
    let num_elems = svcnth() as usize;
    let mut x0_buf = Vec::with_capacity(num_elems);
    let mut x1_buf = Vec::with_capacity(num_elems);
    svst1_u16(__pred, x0_buf.as_mut_ptr(), x0_val);
    x0_buf.set_len(num_elems);
    svst1_u16(__pred, x1_buf.as_mut_ptr(), x1_val);
    x1_buf.set_len(num_elems);
    for i in 0..num_elems {
        let x0_val = x0_buf[i];
        dbg!(i, x0_val);
    }
    for i in 0..num_elems {
        let x1_val = x1_buf[i];
        dbg!(i, x1_val);
    }
}

Similarly for bool vectors:

{
    let num_elems = svcntb() as usize;
    let mut _op_val = Vec::with_capacity(num_elems);
    svst1_u8(op_val, _op_val.as_mut_ptr(), svdup_n_u8(1));
    _op_val.set_len(num_elems);
    for i in 0..num_elems {
        let _op_val_el = if _op_val[i] == 1 { true } else { false };
        dbg!(i, _op_val_el);
    }
}

I'm not sure if what you've suggested will work, happy to try. I'm more than happy to make some improvements here for debuggability of the tests when they do fail, but could that be left to a follow-up?

[sayantn]

very minor nit, but you can use something like intrinsic.specializations().for_each(|imm_values| fmt(...))

View changes since the review

[sayantn]

also, it would be nice if this can be split into a few focused PRs (I can help if you want)

[davidtwco]

Submitted rust-lang/rust#158088 to fix the failure in debug mode - should pass CI after that

[davidtwco]

also, it would be nice if this can be split into a few focused PRs (I can help if you want)

I'll try and split some out tomorrow

[davidtwco]

Split out the first batch of changes in #2169

[davidtwco]

Split out the second batch of changes in #2170

[davidtwco]

Split out the third batch of changes in #2171

[davidtwco]

Split out fourth batch of changes in #2172

[davidtwco]

Split out final batch of changes in #2173. Shouldn't be much less other than the actual SVE testing implementation here after those are merged, but there will be a few merge conflicts between them as they land.