Add AVX-512 support#231
Open
Shnatsel wants to merge 61 commits into
Open
Conversation
…edicated AVX-512 implementations for complex int/float vector operations that benefit the most. LLM summary of the changes: Implemented: - Added `X86::Avx512` in the generator with Ice Lake feature set, `native_width = 512`, `max_block_size = 512`. - Generated new `fearless_simd/src/generated/avx512.rs`. - Wired public API: `Avx512`, `x86::Avx512`, `Level::Avx512`, `Level::as_avx512`, dispatch, and `kernel!` support. - Updated runtime/static detection so Ice Lake AVX-512 is selected before AVX2, while `as_avx2()` and `as_sse4_2()` downgrade correctly. - Bumped MSRV/docs/CI/check-target metadata to Rust 1.89. Generator/backend behavior: - 512-bit vectors use native `__m512`, `__m512d`, and `__m512i`. - AVX-512 masks now use raw compact `__mmask8/16/32/64` storage, with no aligned wrapper. - Generic `SimdFrom<__mmask*, S>` / `From<mask*, __mmask*>` now route through `from_bitmask` / `to_bitmask`, so they are correct for non-AVX-512 `S` too. - Added AVX-512 compare/select paths using mask-returning compares and mask blends. - Added direct conversion paths, including `f32 <-> i32/u32` and `u8 <-> u16`. - Added AVX-512 vector slides for vectors only; masks intentionally have no slide support. - Added dedicated AVX-512 zip/unzip/interleave/deinterleave using `permutex2var`, especially for 256/512-bit widths. Tests/coverage: - Extended `#[simd_test]` to include AVX-512. - Added AVX-512 detection/dispatch coverage. - Updated mask bitwise tests for canonical boolean mask lanes. - Added a regression test that AVX-512 mask public types are compact and match `__mmask*` sizes.
…nt the spooky bug I almost introduced
…rage for these ops.
…calar, now we use the dedicated intrinsics.
…ackend, and specialize it for AVX-512. Add test coverage that sets every single bit and verifies it was set correctly.
… test to exercise it. i8/u8 test is still bad because of rust-lang/rust#156891
…rage. Only for 8-bit left shift LLVM autovectorizes the scalar fallback into GFNI instructions on 256-bit halves which emits more instructions but schedules better and ends up being slightly faster according to llvm-mca on sapphire rapids; but the difference isn't huge and I don't want to rely on autovectorization because of its fragility.
…it vectors on AVX-512; expand test coverage
… no cost to throughput
… so they didn't show up earlier when I removed those methods.
Closed
…e get dead code warnings
…ppy --tests` without a reported location, I've failed to isolate it to a specific crate and suppress it there
Collaborator
|
I think it would indeed be great to have a custom PR for 3. |
Contributor
Author
|
It will cause a lot of conflicts if I try to split it, but I have it isolated to its own commit at least: f08f7e6 |
Merged origin/main commit 13dd530. The merge applied cleanly.
Replace AVX512 interleaved load intrinsics emitted by the branch with checked_transmute_copy, then regenerate the generated AVX512 module.
Merged origin/main commit fbc97da. The merge applied cleanly.
Regenerate the branch-added AVX512 module so by-value transmutes use checked_transmute_copy, matching PR linebender#234. Validation: cargo test
Merged origin/main commit 0d13b0a. The merge applied cleanly.
Regenerate the branch-added AVX512 module so reference casts use checked_cast_ref and checked_cast_mut. Also apply the float bit-pattern assertion style from PR linebender#235 to the branch-added f32x16 interleaved-load test. Validation: cargo test
Merged origin/main commit 650815d. The merge applied cleanly.
PR linebender#237 only updates NEON load construction. The AVX512 branch-specific unsafe load sites were already adapted in the PR linebender#233 follow-up, and a search found no remaining load intrinsics needing the linebender#237 pattern.
Includes the regenerated AVX-512 output from the same generator update.
Includes regenerated AVX-512 slide helpers for the same safety cleanup.
Contributor
Author
|
I've researched whether the instruction set we chose is forward-compatible with Intel's upcoming AVX10. It is: according to the Intel AVX10 architecture specification revision 7.0, all AVX10 CPUs include the AVX-512 features from Ice Lake (our target) as well as Sapphire Rapids (higher than our target but doesn't add anything particularly useful). |
Includes regenerated AVX-512 interleaved load/store output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Yes, really. It's all here. In one humongous PR. Sorry 😅
This is probably best reviewed commit-by-commit. The first commit is still big because the history was getting really messy with changes and rollbacks, and squashing it made it less of a mess.
This also touches other backends in three ways:
set_mask()is now a backend method so it could be specialized per-leveltransmute_copy()is wrapped intochecked_transmute_copy()and the raw version disallowed after I almost had a horrible accident with it.This could be its own PR but I wanted the insurance right away.This was split and shipped in v0.5.0Everything changed here should be covered by tests. I've expanded test coverage where it was lacking.
Closes #179