From b29f8603d440645c28e5817097422f321bb8d4f5 Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Sun, 19 Jul 2026 12:37:49 -0700 Subject: [PATCH 1/6] Rewrite SIMD article as SIMD and Hardware Intrinsics guide Modernize docs/standard/simd.md into a source-of-truth guide covering the layered SIMD support in .NET 11: System.Numerics fixed-shape types, Vector, cross-platform Vector64/128/256/512, platform-specific hardware intrinsics, and TensorPrimitives. Adds a full common-operations reference, code-structure/remainder-handling guidance, an instruction-set configuration-knob reference, and testing/benchmarking sections, backed by a verified C# snippet project. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/fundamentals/toc.yml | 2 +- docs/standard/simd.md | 325 +++++++++++++----- .../simd/csharp/AdvancedVectorization.cs | 271 +++++++++++++++ .../simd/csharp/HardwareIntrinsicsExample.cs | 38 ++ .../snippets/simd/csharp/NumericsExamples.cs | 74 ++++ docs/standard/snippets/simd/csharp/Program.cs | 48 +++ .../snippets/simd/csharp/SimdSnippets.csproj | 15 + .../simd/csharp/SimpleVectorization.cs | 76 ++++ .../simd/csharp/TensorPrimitivesExample.cs | 23 ++ 9 files changed, 789 insertions(+), 83 deletions(-) create mode 100644 docs/standard/snippets/simd/csharp/AdvancedVectorization.cs create mode 100644 docs/standard/snippets/simd/csharp/HardwareIntrinsicsExample.cs create mode 100644 docs/standard/snippets/simd/csharp/NumericsExamples.cs create mode 100644 docs/standard/snippets/simd/csharp/Program.cs create mode 100644 docs/standard/snippets/simd/csharp/SimdSnippets.csproj create mode 100644 docs/standard/snippets/simd/csharp/SimpleVectorization.cs create mode 100644 docs/standard/snippets/simd/csharp/TensorPrimitivesExample.cs diff --git a/docs/fundamentals/toc.yml b/docs/fundamentals/toc.yml index 57efc41124d8b..e85be24e5a53e 100644 --- a/docs/fundamentals/toc.yml +++ b/docs/fundamentals/toc.yml @@ -411,7 +411,7 @@ items: href: ../standard/memory-and-spans/index.md - name: Memory and Span usage guidelines href: ../standard/memory-and-spans/memory-t-usage-guidelines.md - - name: SIMD-enabled types + - name: SIMD and Hardware Intrinsics href: ../standard/simd.md - name: Value tuples href: ../standard/value-tuples.md diff --git a/docs/standard/simd.md b/docs/standard/simd.md index c275fc3c1fa2a..2e5c45b2d4c43 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -1,121 +1,282 @@ --- -title: SIMD-accelerated types in .NET -description: This article describes SIMD-enabled types in .NET and demonstrates how to use hardware SIMD operations in C# and .NET. +title: Use SIMD and Hardware Intrinsics in .NET +description: Learn about the layered SIMD support in .NET, from System.Numerics vector types to cross-platform Vector64/128/256/512 APIs, hardware intrinsics, and TensorPrimitives. author: FIVIL ms.author: tagoo -ms.date: 03/26/2026 +ms.date: 07/18/2026 ai-usage: ai-assisted --- -# Use SIMD-accelerated numeric types +# Use SIMD and Hardware Intrinsics in .NET -SIMD (Single instruction, multiple data) provides hardware support for performing an operation on multiple pieces of data, in parallel, using a single instruction. In .NET, there's set of SIMD-accelerated types under the namespace. SIMD operations can be parallelized at the hardware level. That increases the throughput of the vectorized computations, which are common in mathematical, scientific, and graphics apps. +SIMD (single instruction, multiple data) is hardware support for applying one operation to multiple pieces of data in parallel with a single instruction. Vectorized code processes several values per iteration instead of one, which can greatly increase throughput for the kind of numeric, scientific, graphics, text-processing, and data-parallel work where the same operation repeats over a buffer. The trade-off is added complexity, so it pays off most when the input is large enough and the win is confirmed with measurements. -## .NET SIMD-accelerated types +.NET provides several kinds of SIMD support. Pick the one that matches how much control you need and how much complexity you're willing to take on. -The .NET SIMD-accelerated types include the following types: +## Types of SIMD support in .NET -- The , , and types, which represent vectors with 2, 3, and 4 values. +| API | Namespace | When to use it | +| --- | --- | --- | +| Fixed-purpose vector and matrix types | | Graphics and geometry math with 2-4 element vectors, matrices, quaternions, and planes. | +| `Vector` | | Portable, variable-width vectorization when you don't need per-platform control. | +| `Vector64`, `Vector128`, `Vector256`, `Vector512` | | Cross-platform, fixed-width vectorization with fine-grained control. This is the recommended starting point for new vectorized algorithms. | +| Hardware intrinsics | , , | Specific processor instructions that the higher-level APIs don't expose, for the last bit of performance on a hot path. | +| `TensorPrimitives` | | Ready-made, vectorized math over spans. It does the vectorization for you. | -- Two matrix types, , which represents a 3x2 matrix, and , which represents a 4x4 matrix of values. +Where these APIs overlap, they relate through layers of abstraction. The generic vector types are the foundational interchange types that the other layers pass around, so they're technically the lowest level: the variable-width `Vector`, which grows to whatever width the running hardware supports, and the fixed-width `Vector64` through `Vector512`. The platform-specific hardware intrinsics in , , and operate on those types, each mapping directly to an individual processor instruction. The cross-platform operations exposed on the generic types sit a step above the platform-specific intrinsics, lowering to them for each target. Higher still are the managed APIs that operate over entire buffers—vectorized methods on `Span` and `string`, and —which build on the layers below, so you get SIMD acceleration without hand-writing any of it. The fixed-shape types are domain-specific convenience types for graphics and geometry rather than part of this interchange stack. -- The type, which represents a plane in three-dimensional space using values. +The rest of this article works through these APIs from the highest level to the lowest, then covers testing, benchmarking, and best practices. -- The type, which represents a vector that is used to encode three-dimensional physical rotations using values. +## System.Numerics vector and matrix types -- The type, which represents a vector of a specified numeric type and provides a broad set of operators that benefit from SIMD support. The count of a instance is fixed for the lifetime of an application, but its value depends on the CPU of the machine running the code. +The namespace provides SIMD-accelerated types with a fixed shape: - > [!NOTE] - > The type is not included in .NET Framework. You must install the [System.Numerics.Vectors](https://www.nuget.org/packages/System.Numerics.Vectors) NuGet package to get access to this type. - -The SIMD-accelerated types are implemented in such a way that they can be used with non-SIMD-accelerated hardware or JIT compilers. To determine whether SIMD acceleration is available at runtime, use . If that property returns `true`, at least some APIs use hardware-accelerated SIMD operations. If it returns `false`, no APIs are hardware accelerated. +- , , and represent vectors of 2, 3, and 4 values. +- and represent 3x2 and 4x4 matrices of values. +- represents a plane in three-dimensional space. +- represents a vector used to encode three-dimensional rotations. -## How to use SIMD? +These types map naturally to graphics and geometry, and the runtime accelerates their operations with SIMD instructions where the hardware supports them. The following example adds two vectors: -Before executing custom SIMD algorithms, it's possible to check if the host machine supports SIMD by using , which returns a . This doesn't guarantee that SIMD-acceleration is enabled for a specific type, but is an indicator that it's supported by some types. +:::code language="csharp" source="./snippets/simd/csharp/NumericsExamples.cs" id="SimpleVectorAdd"::: -## Simple vectors +They also expose the common vector math you'd expect, such as dot product, distance, and clamping: -The most primitive SIMD-accelerated types in .NET are , , and types, which represent vectors with 2, 3, and 4 values. The example below uses to add two vectors. +:::code language="csharp" source="./snippets/simd/csharp/NumericsExamples.cs" id="SimpleVectorOps"::: -```csharp -var v1 = new Vector2(0.1f, 0.2f); -var v2 = new Vector2(1.1f, 2.2f); -var vResult = v1 + v2; -``` +The matrix types support matrix math such as transpose and multiplication: -It's also possible to use .NET vectors to calculate other mathematical properties of vectors such as `Dot product`, `Transform`, `Clamp` and so on. +:::code language="csharp" source="./snippets/simd/csharp/NumericsExamples.cs" id="MatrixMultiply"::: -```csharp -var v1 = new Vector2(0.1f, 0.2f); -var v2 = new Vector2(1.1f, 2.2f); -var vResult1 = Vector2.Dot(v1, v2); -var vResult2 = Vector2.Distance(v1, v2); -var vResult3 = Vector2.Clamp(v1, Vector2.Zero, Vector2.One); -``` +## Vector -## Matrix +`Vector` represents a variable-width vector of a primitive numeric type. Its length is fixed for the lifetime of the process, but the value of `Vector.Count` depends on the CPU that runs the code. The Just-In-Time (JIT) compiler treats `Count` as a constant, so loops written against it optimize well. -, which represents a 3x2 matrix, and , which represents a 4x4 matrix. Can be used for matrix-related calculations. The example below demonstrates multiplication of a matrix to its correspondent transpose matrix using SIMD. +`Vector` gives you portable vectorization without per-platform code, at the cost of not knowing the vector width at compile time. The following example computes the element-wise add of two arrays: -```csharp -var m1 = new Matrix4x4( - 1.1f, 1.2f, 1.3f, 1.4f, - 2.1f, 2.2f, 3.3f, 4.4f, - 3.1f, 3.2f, 3.3f, 3.4f, - 4.1f, 4.2f, 4.3f, 4.4f); +:::code language="csharp" source="./snippets/simd/csharp/NumericsExamples.cs" id="VectorTAdd"::: -var m2 = Matrix4x4.Transpose(m1); -var mResult = Matrix4x4.Multiply(m1, m2); -``` +> [!NOTE] +> This example is illustrative. You rarely need to write a loop like this by hand, because [`TensorPrimitives`](#higher-level-math-with-tensorprimitives) already provides accelerated span-based math. This element-wise add is , and reductions such as are provided too. These operations are hardware-accelerated for the element types that `Vector` supports (`Vector.IsSupported`). -## Vector\ +## Check for hardware acceleration -The gives the ability to use longer vectors. The count of a instance is fixed, but its value depends on the CPU of the machine running the code. +SIMD-accelerated types work even on hardware or JIT configurations that don't support SIMD, because they fall back to non-accelerated software implementations. To branch on whether acceleration is actually available, check the relevant `IsHardwareAccelerated` property: -The following example demonstrates how to calculate the element-wise sum of two arrays using . +- reports whether `Vector` operations are accelerated. +- , and the `Vector64`/`Vector256`/`Vector512` equivalents, report acceleration for each fixed width. -```csharp -double[] Sum(double[] left, double[] right) -{ - if (left is null) - { - throw new ArgumentNullException(nameof(left)); - } +These properties are turned into constants by the JIT, so the branches you don't take are eliminated and there's no runtime cost to checking them. Don't cache the values; read them directly where you need them. The same applies to the `Count` properties (for example, `Vector128.Count`), which are also JIT-time constants. - if (right is null) - { - throw new ArgumentNullException(nameof(right)); - } +Most operations on an accelerated width are themselves accelerated, but it isn't guaranteed for every operation. For example, floating-point division may be accelerated where integer division isn't. When `Vector256` is accelerated, `Vector128` usually is too, but there's no guarantee, so check each width you use. - if (left.Length != right.Length) - { - throw new ArgumentException($"{nameof(left)} and {nameof(right)} are not the same length"); - } +> [!TIP] +> If an operation you need isn't accelerated on a platform you care about, or you'd like a new cross-platform API, file an issue on [dotnet/runtime](https://github.com/dotnet/runtime/issues). The same applies to codegen improvements. - int length = left.Length; - double[] result = new double[length]; +Not every element type is valid for every vector. `Vector128` and its siblings support the primitive numeric types (`byte`, `sbyte`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `float`, `double`, `nint`, and `nuint`) today, and that set may grow to include other types in the future. Use `Vector128.IsSupported` to determine whether a given `T` is valid, which is especially useful from generic code. - // Get the number of elements that can't be processed in the vector - // NOTE: Vector.Count is a JIT time constant and will get optimized accordingly - int remaining = length % Vector.Count; +Types that aren't supported, such as `char` and `bool`, can still be vectorized by reinterpreting the buffer as a supported type of the same size. Use to reinterpret a span—for example, `char` to `ushort`—or the vector's `As` method to reinterpret a vector you already hold. Reinterpretation only changes the type, not the underlying bits, so it's your responsibility to keep the data well-formed: a `bool` must stay `0` or `1`, and a `char` must remain a valid UTF-16 code unit. If a vectorized operation could produce an out-of-range value, take care to normalize the result before writing it back. - for (int i = 0; i < length - remaining; i += Vector.Count) - { - var v1 = new Vector(left, i); - var v2 = new Vector(right, i); - (v1 + v2).CopyTo(result, i); - } +## Cross-platform vectorization with Vector128 - for (int i = length - remaining; i < length; i++) - { - result[i] = left[i] + right[i]; - } +`Vector128` is the common denominator across every platform that supports vectorization, so it's the best place to start. It holds a 128-bit vector: 16 bytes, 8 shorts, 4 ints/floats, or 2 longs/doubles. `Vector256` is twice as wide, and `Vector512` twice again. Not all hardware supports the larger widths, so the examples that follow use `Vector128` for portability. - return result; -} -``` +Each width has a generic type (`Vector128`) for the data and a non-generic static class () that holds most of the operations, including static factory methods like `Create` and `Load`. Operators such as `+`, `&`, and `<<` are the idiomatic way to express arithmetic and bit operations; prefer them over the named method equivalents to avoid operator-precedence bugs and improve readability. For algorithms that depend on byte order, branch on , which the JIT also folds to a constant. -## Remarks +> [!NOTE] +> On x86/x64, `Vector256` operations are generally treated as two independent 128-bit "lanes". For most element-wise operations this is transparent, but operations that cross lanes (such as shuffles or pairwise/horizontal operations) may behave differently or cost more than the `Vector128` equivalent. Confirm with benchmarks before assuming a wider vector is faster. -SIMD is more likely to remove one bottleneck and expose the next, for example memory throughput. In general the performance benefit of using SIMD varies depending on the specific scenario, and in some cases it can even perform worse than simpler non-SIMD equivalent code. +### Common operations + +`Vector128` and its wider siblings expose a large API surface. You don't need to memorize it—know the categories and look up the details when you need them. Every operation has a software fallback for platforms that can't accelerate it. The table below covers essentially the whole surface. + +| Category | What it does | Representative APIs | +| --- | --- | --- | +| Constants | Predefined constant vectors | `Zero`, `One`, `NegativeOne`, `AllBitsSet`, `Indices`, `SignSequence`, `E`, `Pi`, `Tau`, `Epsilon`, `NaN`, `PositiveInfinity`, `NegativeInfinity`, `NegativeZero` | +| Creation | Broadcast a scalar, set elements, or generate a sequence | `Create`, `CreateScalar`, `CreateScalarUnsafe`, `Create(ReadOnlySpan)`, `CreateSequence`, `CreateGeometricSequence`, `CreateHarmonicSequence`, `CreateAlternatingSequence` | +| Load and store | Move data between memory and a vector | `Load`, `LoadUnsafe`, `LoadAligned`, `LoadAlignedNonTemporal`, `Store`, `StoreUnsafe`, `StoreAligned`, `StoreAlignedNonTemporal`, `CopyTo`, `TryCopyTo` | +| Arithmetic | Element-wise math and reductions | `Add (x + y)`, `Subtract (x - y)`, `Multiply (x * y)`, `Divide (x / y)`, `Negate (-x)`, `AddSaturate`, `SubtractSaturate`, `Abs`, `Sqrt`, `FusedMultiplyAdd`, `Dot`, `Sum` | +| Bit operations | Bitwise logic and shifts | `BitwiseAnd (x & y)`, `BitwiseOr (x \| y)`, `Xor (x ^ y)`, `AndNot (x & ~y)`, `OnesComplement (~x)`, `ShiftLeft (x << n)`, `ShiftRightArithmetic (x >> n)`, `ShiftRightLogical (x >>> n)` | +| Min, max, and clamp | Element-wise minimum, maximum, and range clamping | `Min`, `Max`, `Clamp`, `MinMagnitude`, `MaxMagnitude`, `MinNumber`, `MaxNumber`, `MinMagnitudeNumber`, `MaxMagnitudeNumber` | +| Rounding | Round each element to an integral value | `Ceiling`, `Floor`, `Round`, `Truncate` | +| Math functions | Sign, interpolation, angle, and transcendental helpers | `CopySign`, `Lerp`, `DegreesToRadians`, `RadiansToDegrees`, `Hypot`, `Sin`, `Cos`, `SinCos`, `Asin`, `Exp`, `Log`, `Log2` | +| Comparison | Compare each element; the result is a vector mask, not the `bool` an operator gives | `Equals`, `GreaterThan`, `GreaterThanOrEqual`, `LessThan`, `LessThanOrEqual` | +| Classification | Per-element predicates over the number line, each returning a vector mask | `IsNaN`, `IsFinite`, `IsInfinity`, `IsPositiveInfinity`, `IsNegativeInfinity`, `IsInteger`, `IsEvenInteger`, `IsOddInteger`, `IsNegative`, `IsPositive`, `IsNormal`, `IsSubnormal`, `IsZero` | +| Comparison reductions | Collapse a per-element comparison to a single `bool` | `EqualsAll (x == y)`, `EqualsAny`, `GreaterThanAll`, `GreaterThanAny`, `GreaterThanOrEqualAll`, `GreaterThanOrEqualAny`, `LessThanAll`, `LessThanAny`, `LessThanOrEqualAll`, `LessThanOrEqualAny` | +| Whole-vector predicates | Reduce a vector to a `bool`: whether all, any, or no elements equal a value, or (the `WhereAllBitsSet` forms) have all bits set. Prefer these over converting a mask to an index | `All`, `Any`, `None`, `AllWhereAllBitsSet`, `AnyWhereAllBitsSet`, `NoneWhereAllBitsSet` | +| Search | Count or locate elements by value, or (the `WhereAllBitsSet` forms) set lanes in a mask | `Count`, `IndexOf`, `LastIndexOf`, `CountWhereAllBitsSet`, `IndexOfWhereAllBitsSet`, `LastIndexOfWhereAllBitsSet` | +| Mask to index | Turn a comparison mask into a scalar bitmask and scan it | `ExtractMostSignificantBits` with or | +| Selection | Blend two vectors according to a mask, bit by bit | `ConditionalSelect(x, y, z)`, equivalent to `(y & x) \| (z & ~x)` | +| Conversion | Change numeric type, computing new values (for example, `int` to `float`) | `ConvertToInt32`, `ConvertToInt64`, `ConvertToUInt32`, `ConvertToUInt64`, `ConvertToSingle`, `ConvertToDouble` | +| Widening and narrowing | Split elements into a wider type or pack them into a narrower one | `Widen`, `WidenLower`, `WidenUpper`, `Narrow`, `NarrowWithSaturation` | +| Reinterpretation | Reinterpret the bits as another element type without changing them | `As`, `AsByte`, `AsInt32`, `AsSingle`, and the other `As*` element forms | +| System.Numerics interop | Reinterpret between `Vector128` and the fixed-shape numerics types | `AsVector`, `AsVector2`, `AsVector3`, `AsVector4`, `AsPlane`, `AsQuaternion`, `AsVector128`, `AsVector128Unsafe` | +| Reorder | Rearrange elements by index or interleave two vectors | `Shuffle`, `Reverse`, `Zip`, `ZipLower`, `ZipUpper`, `Unzip`, `UnzipEven`, `UnzipOdd`, `ConcatLowerLower`, `ConcatLowerUpper`, `ConcatUpperLower`, `ConcatUpperUpper` | +| Lane access | Read or replace individual elements and halves, or resize the vector | `GetElement`, `WithElement`, `ToScalar`, `GetLower`, `GetUpper`, `WithLower`, `WithUpper`, `ToVector256` | + +> [!TIP] +> Several operations have `Estimate` and `Native` variants—for example, `MultiplyAddEstimate`, `ClampNative`, `MinNative`, `MaxNative`, `ShuffleNative`, and `ConvertToInt32Native`. They map to a faster hardware instruction that trades some precision or gives up an IEEE edge-case guarantee (such as NaN handling), so reach for them only when a benchmark shows the exact form is the bottleneck and the looser semantics are acceptable. + +> [!NOTE] +> `Vector256.Shuffle` treats its input as a single 256-bit vector, where-as the platform-specific `Avx2.Shuffle` operates as two independent 128-bit lanes. The cross-platform API is the more portable choice, but confirm the behavior you need when porting hand-written intrinsics. + +### Structure the code path + +A vectorized method typically fans out into a path per vector width plus a scalar fallback for small inputs and non-accelerated hardware. To use the largest vector the hardware supports, check the widest vector first and work down: + +:::code language="csharp" source="./snippets/simd/csharp/AdvancedVectorization.cs" id="CodeStructure"::: + +Each width's outer guard combines `Vector128.IsHardwareAccelerated` (a JIT-time constant for whether the platform accelerates that width) with `Vector128.IsSupported` (whether the element type `T` is valid for that width). Inside a supported block, compare the input length against `Count` to choose between the vectorized path and a small-input fallback. The method is generic over `T`, and the `Vector256` and `Vector512` blocks—identical to the `Vector128` block but using the wider type—are shown commented out for brevity. + +There are two distinct fallbacks. A buffer that's too small for even the narrowest vector, but on accelerated hardware, goes to `SumVectorSmall`—an explicit `switch` jump table that handles each possible sub-vector length without a loop: + +:::code language="csharp" source="./snippets/simd/csharp/AdvancedVectorization.cs" id="VectorSmall"::: + +`sizeof(T)` is also a JIT-time constant (and, with the `unmanaged` constraint, doesn't require an `unsafe` context in C# 15 / .NET 11), so `SumVectorSmall` dispatches on the element width to a table sized for the widest vector's worth of elements—the same approach uses. Only the 4-byte table is shown; the 1-, 2-, and 8-byte tables share its shape. Its larger cases fold the leftover with a `Vector256` or `Vector128` using two overlapping loads—one from the start, one from the end—so the wider-remainder handling for the omitted `Vector512`/`Vector256` paths lives right in the jump table. The two loads overlap whenever the length isn't an exact multiple of the width, so the tail is masked to the additive identity with `ConditionalSelect` before it's summed. That mask is only needed because addition is non-idempotent; an idempotent operation such as a search could fold the overlapping tail in directly. A buffer on hardware with no vectorization at all falls through to `SumScalar`, an ordinary scalar loop. + +### Loop over the input and handle the remainder + +To process a buffer larger than a single vector, loop over it a vector at a time, then handle the leftover elements that don't fill a full vector. The robust way to handle that tail is to reprocess the last full vector's worth of elements, overlapping some the loop already handled—which avoids a separate scalar epilogue. Whether that overlap needs correcting depends on the operation. + +A **non-idempotent** operation such as a sum would count the overlapping elements twice, so mask them down to the operation's identity before folding them in. Use this when each element must contribute exactly once: + +:::code language="csharp" source="./snippets/simd/csharp/AdvancedVectorization.cs" id="MaskedRemainder"::: + +This version guards the unrolled loop with an `if`, so small payloads skip the four accumulators entirely and fall straight to the remainder. When there's enough data, a `do`/`while` accumulates four vectors per iteration into independent accumulators—which lets the processor pipeline the additions—and combines them pairwise. A `switch` jump table then folds in the remaining zero-to-three full vectors and, in `case 0`, the sub-vector tail: it reuses a full vector preloaded from the end of the buffer, overlapping the elements already processed, and masks that overlap down to the additive identity with `ConditionalSelect` so the tail stays vectorized instead of falling out to a scalar loop. As before, `Vector128.Create` reads `Vector128.Count` elements from the span. The JIT elides the span's bounds check for typical access patterns, so `Create` is a fine default even in a hot loop; `LoadUnsafe` (covered next) is the lower-level alternative for when you walk the buffer by managed reference. For very large inputs, a complete implementation would also align the buffer and use non-temporal loads and stores to avoid evicting useful cache lines—both omitted here and covered fully by `TensorPrimitives`. + +An **idempotent** operation such as searching for a value can reprocess the overlap harmlessly, so it folds the last vector in directly with no mask: + +:::code language="csharp" source="./snippets/simd/csharp/SimpleVectorization.cs" id="VectorizedRemainder"::: + +> [!WARNING] +> Mishandling the remainder is a common source of bugs. A loop that reads past the end of the buffer produces non-deterministic results and can crash. The runtime's test suite uses a `BoundedMemory` helper that places a no-access page immediately after the buffer, so any out-of-bounds read throws an during testing. Always cover the remainder logic, including buffers whose length isn't a multiple of the vector width. + +### Load and store vectors safely + +For most code, `Vector128.Create(span)` and are the simplest way to move data between a span and a vector, and the JIT keeps them efficient. When you need the lower-level load and store—for example, to walk a buffer by managed reference—prefer the and overloads that take a managed reference and an `nuint` element offset. Unlike the pointer-based `Load`/`Store` overloads, they don't require pinning the buffer, and unlike raw reference arithmetic they don't need you to manually advance a `ref`. Both alternatives are easy to get wrong in ways that introduce garbage-collector holes or access violations. + +So that empty buffers don't throw, get the starting reference from (or for arrays) rather than `ref span[0]`. + +> [!IMPORTANT] +> Offset arithmetic uses unsigned `nuint`. Always check the buffer length before computing an offset such as `buffer.Length - Vector128.Count`. If the buffer is smaller than one vector, that subtraction underflows to a huge value and the loop reads invalid memory. + +## Platform-specific hardware intrinsics + +When a specific processor instruction gives you an edge that the portable APIs don't expose, reach for the hardware intrinsics in , , and . Each intrinsic class has an `IsSupported` property (also a JIT constant) so you can guard the specialized path and fall back to portable code elsewhere: + +:::code language="csharp" source="./snippets/simd/csharp/HardwareIntrinsicsExample.cs" id="HardwareIntrinsics"::: + +The preceding method shows how to light up per-architecture code paths when you want them, but it's deliberately a simple example: you don't actually need it here. The portable expression `(vector & mask) == Vector128.Zero` already lowers to the optimal instruction on each platform (for example, `ptest` on x86/x64), so it's doing the same work as the hand-written branches, just without the complexity. Reach for explicit intrinsics only when a specific instruction measurably beats what the portable APIs generate. + +Hardware intrinsics require a separate implementation per instruction set, so treat them as an optimization for measured hot paths rather than a default. The `Vector128`/`Vector256` APIs already lower to efficient instructions on each platform, and in practice sophisticated per-instruction code doesn't always win. Confirm the difference with a benchmark before committing to the extra maintenance. + +## Higher-level math with TensorPrimitives + +If you need vectorized math over spans and don't want to write the loops yourself, provides a large set of numeric operations—element-wise arithmetic, exponentials, and reductions such as dot product and cosine similarity—that are already vectorized internally. It's available in the [System.Numerics.Tensors](https://www.nuget.org/packages/System.Numerics.Tensors) NuGet package. + +:::code language="csharp" source="./snippets/simd/csharp/TensorPrimitivesExample.cs" id="TensorPrimitives"::: + +For AI and numeric workloads, `TensorPrimitives` often delivers most of the benefit of hand-written SIMD with none of the complexity. + +## Test all code paths + +Because a vectorized method has several code paths, tests need to cover each one: the `Vector256` path, the `Vector128` path, and the scalar path, each with inputs both large enough and too small to benefit. You can vary the input size in tests, but you can't toggle hardware acceleration at the test level. Instead, control it with environment variables before the process starts: + +- Set `DOTNET_EnableAVX2=0` to make return `false`. +- Set `DOTNET_EnableHWIntrinsic=0` to disable intrinsics entirely, so `Vector128`, `Vector64`, and `Vector` all report no acceleration. + +To exercise every path on a single machine, run the test suite once with no overrides, once with `DOTNET_EnableAVX2=0`, and once with `DOTNET_EnableHWIntrinsic=0`. The alternative is running across enough varied hardware to cover them. + +### Instruction-set configuration knobs + +Beyond those two, the runtime recognizes a knob per logical grouping of instruction sets, each prefixed with `DOTNET_`. A single knob can cover several related instruction sets—`EnableAVX2`, for instance, gates AVX2 along with BMI1, BMI2, F16C, FMA, LZCNT, and MOVBE. Setting a knob to `0` disables its whole group and everything layered on top of it. Setting it to `1` (the default for most) allows the group, but the hardware must still actually support it—enabling a knob the current CPU lacks is ignored, so you can only ever narrow what's used, never force an unsupported instruction on and break yourself. `DOTNET_EnableHWIntrinsic=0` is the big hammer—it turns off everything down to the base, so `Vector128`, `Vector64`, and `Vector` all report no acceleration and the code falls to its software path. + +> [!IMPORTANT] +> These are diagnostic tools, meant primarily for testing and validation—exercising each code path, reproducing a hardware-specific issue, or confirming a fallback. They aren't designed for general or production use, and they aren't a stability contract: the set below is what .NET 11 recognizes; earlier releases exposed a different set—the baseline and AVX-512 knobs in particular were reconfigured—so confirm the names against the runtime version you target. +> +> They also have limits on what they reach. Because they gate JIT decisions, they don't affect code that was already compiled ahead of time through ReadyToRun or Native AOT, and they don't necessarily affect internal routines the runtime and core libraries use themselves. Treat them as a way to steer your own JIT-compiled code, not a global off switch for an instruction set. + +The base switch and the width caps apply on every architecture: + +| Knob (`DOTNET_` prefix) | Default | Effect | +| --- | --- | --- | +| `EnableHWIntrinsic` | `1` | Master switch for all hardware intrinsics; `0` forces the fully software path. | +| `MaxVectorTBitWidth` | system default | Caps `Vector` to a maximum width in bits; a value below 128 means the system default. | +| `PreferredVectorBitWidth` | system default | Caps the maximum fixed-width vector that reports `IsHardwareAccelerated`, in bits; a value below 128 means the system default. | + +The system default for `MaxVectorTBitWidth` can be narrower than the hardware fully supports, so `Vector` doesn't automatically grow to the widest available vector. For example, `Vector512.IsHardwareAccelerated` can be `true` while `Vector` stays 256-bit; set `DOTNET_MaxVectorTBitWidth=512` to opt `Vector` into the wider width. + +`PreferredVectorBitWidth` caps the maximum vector width that reports `IsHardwareAccelerated`. Lowering it below what the hardware supports flips the wider widths off: on a machine that supports 512-bit vectors, `DOTNET_PreferredVectorBitWidth=256` makes `Vector512.IsHardwareAccelerated` report `false`. It's a general knob, but only x86/x64 offers widths above 128 today, so that's the only place it has an observable effect. + +Each logical grouping of x86/x64 instruction sets also has its own switch: + +| Knob (`DOTNET_` prefix) | Default | Gates | +| --- | --- | --- | +| `EnableAVX` | `1` | AVX and dependents | +| `EnableAVX2` | `1` | AVX2, BMI1, BMI2, F16C, FMA, LZCNT, MOVBE, and dependents | +| `EnableAVX512` | `1` | AVX-512 F+BW+CD+DQ+VL and dependents | +| `EnableAVX512BMM` | `1` | AVX-512 BMM | +| `EnableAVX512v2` | `1` | AVX-512 IFMA+VBMI | +| `EnableAVX512v3` | `1` | AVX-512 BITALG+VBMI2+VPOPCNTDQ+VNNI | +| `EnableAVX10v1` | `1` | AVX10.1 | +| `EnableAVX10v2` | `0` | AVX10.2 | +| `EnableAPX` | `0` | APX (extended general-purpose registers) | +| `EnableAES` | `1` | AES, PCLMULQDQ | +| `EnableAVX512VP2INTERSECT` | `1` | AVX-512 VP2INTERSECT | +| `EnableAVXIFMA` | `1` | AVX-IFMA | +| `EnableAVXVNNI` | `1` | AVX-VNNI | +| `EnableAVXVNNIINT` | `1` | VEX AVX-VNNI-INT8 and AVX-VNNI-INT16 | +| `EnableGFNI` | `1` | GFNI | +| `EnableSHA` | `1` | SHA | +| `EnableVAES` | `1` | VAES, VPCLMULQDQ | +| `EnableWAITPKG` | `1` | WAITPKG | +| `EnableX86Serialize` | `1` | X86 SERIALIZE | + +On Arm64, each logical grouping of instruction sets has its own switch: + +| Knob (`DOTNET_` prefix) | Default | Gates | +| --- | --- | --- | +| `EnableArm64Aes` | `1` | AES | +| `EnableArm64Atomics` | `1` | Large System Extensions (LSE) atomics | +| `EnableArm64Crc32` | `1` | CRC32 | +| `EnableArm64Dczva` | `1` | `DC ZVA` cache-zeroing | +| `EnableArm64Dp` | `1` | Dot Product | +| `EnableArm64Rdm` | `1` | Rounding Double Multiply (RDMA) | +| `EnableArm64Sha1` | `1` | SHA1 | +| `EnableArm64Sha256` | `1` | SHA256 | +| `EnableArm64Rcpc` | `1` | Release Consistent processor consistent (RCPC) | +| `EnableArm64Rcpc2` | `1` | RCPC2 | +| `EnableArm64Cssc` | `0` | Common Short Sequence Compression (CSSC) | +| `EnableArm64Sve` | `1` | Scalable Vector Extension (SVE) | +| `EnableArm64Sve2` | `1` | SVE2 | +| `EnableArm64Sha3` | `1` | SHA3 | +| `EnableArm64Sm4` | `1` | SM4 | +| `EnableArm64SveAes` | `1` | SVE AES | +| `EnableArm64SveSha3` | `1` | SVE SHA3 | +| `EnableArm64SveSm4` | `1` | SVE SM4 | + +A knob defaulting to `0` (for example, `EnableAVX10v2` or `EnableArm64Cssc`) gates an instruction set that's still coming online, so it stays off until you opt in. + +## Benchmark to confirm the win + +Vectorization adds complexity, so measure that it pays off before you keep it. Use [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet), and use the same environment variables above to compare the scalar, `Vector128`, and `Vector256` implementations in one run. BenchmarkDotNet's disassembly diagnoser can also emit the generated assembly, which is invaluable when tuning high-performance code. + +A few things to keep in mind: + +- **Larger inputs benefit more.** For small buffers, vectorized code can be slower than scalar code because of setup overhead. Benchmark the input sizes your callers actually use. +- **Speedups are rarely perfect.** A 256-bit vector operating on 32-bit elements won't reliably be 8x faster; memory throughput, alignment, and instruction latency all factor in. +- **Memory alignment affects stability.** Randomized allocation alignment adds noise between runs. You can allocate aligned memory with for stable results, or enable BenchmarkDotNet's memory randomization to observe the full distribution. + +## Best practices + +- Reach for the existing higher-level APIs first. `Span`, `string`, LINQ, `TensorPrimitives`, and the tensor types already accelerate many common operations for you—don't hand-roll what's already optimized and tested. +- Start with `Vector128`; it's accelerated on the broadest set of hardware, and you don't need `Vector256` to get a correct, portable implementation. Add wider widths and hardware intrinsics only for measured hot paths. +- Check `IsHardwareAccelerated` and `Count` directly instead of caching them; the JIT turns them into constants. +- Always handle the loop remainder, and account for overlapping source and destination buffers when storing. +- Write edge-case tests first, then a scalar solution, then express that scalar logic with the vector APIs. +- Test every code path (including access violations) and benchmark realistic input sizes before you commit to the added complexity. + +## See also + +- +- diff --git a/docs/standard/snippets/simd/csharp/AdvancedVectorization.cs b/docs/standard/snippets/simd/csharp/AdvancedVectorization.cs new file mode 100644 index 0000000000000..d20fca1999826 --- /dev/null +++ b/docs/standard/snippets/simd/csharp/AdvancedVectorization.cs @@ -0,0 +1,271 @@ +using System.Diagnostics; +using System.Numerics; +using System.Runtime.Intrinsics; + +namespace SimdSnippets; + +public static class AdvancedVectorization +{ + // + // Sums a buffer, choosing the widest vector the hardware and element type support. + public static T Sum(ReadOnlySpan buffer) + where T : unmanaged, INumberBase + { + // The widest-first order continues with the Vector512 and Vector256 paths, which belong + // here ahead of the Vector128 block below. They're identical to it aside from the wider + // type (for example, Vector512 with Vector512.Create and Vector512.Sum), so they're + // omitted for brevity: + // + // if (Vector512.IsHardwareAccelerated && Vector512.IsSupported) + // { + // if (buffer.Length >= Vector512.Count) + // { + // return SumVector512(buffer); + // } + // return SumVectorSmall(buffer); + // } + // + // if (Vector256.IsHardwareAccelerated && Vector256.IsSupported) + // { + // if (buffer.Length >= Vector256.Count) + // { + // return SumVector256(buffer); + // } + // return SumVectorSmall(buffer); + // } + + if (Vector128.IsHardwareAccelerated && Vector128.IsSupported) + { + if (buffer.Length >= Vector128.Count) + { + return SumVector128(buffer); + } + return SumVectorSmall(buffer); + } + + return SumScalar(buffer); + } + // + + private static T SumScalar(ReadOnlySpan buffer) + where T : unmanaged, INumberBase + { + T result = T.Zero; + foreach (T value in buffer) + { + result += value; + } + return result; + } + + // + // Sums a buffer smaller than the widest vector. The complete "optimal" shape dispatches on the + // element width so each width uses a switch jump table sized to the number of elements that fit + // in the widest vector (63 for byte, 31 for short, 15 for int/float, 7 for long/double). + private static T SumVectorSmall(ReadOnlySpan buffer) + where T : unmanaged, INumberBase + { + // sizeof(T) is a JIT constant, so only the matching branch survives for a given T. + if (sizeof(T) == 4) + { + return SumVectorSmall4(buffer); + } + + // The 1-, 2-, and 8-byte tables share the shape below, sized for their element width. + // They're omitted for brevity, so those widths fall back to a scalar loop here: + // + // if (sizeof(T) == 1) return SumVectorSmall1(buffer); // switch over lengths 0..63 + // if (sizeof(T) == 2) return SumVectorSmall2(buffer); // switch over lengths 0..31 + // if (sizeof(T) == 8) return SumVectorSmall8(buffer); // switch over lengths 0..7 + return SumScalar(buffer); + } + + private static T SumVectorSmall4(ReadOnlySpan buffer) + where T : unmanaged, INumberBase + { + Debug.Assert(sizeof(T) == 4); + Debug.Assert(buffer.Length < Vector512.Count); + + T result = T.Zero; + + // A 4-byte element gives Count == 4/8/16 for Vector128/256/512, so a remainder can be up to + // 15 elements. The larger cases fold the leftover with the widest vector that fits, using two + // overlapping loads (one from the start, one from the end) rather than recursing—the shape + // TensorPrimitives uses. The loads overlap for lengths that aren't an exact multiple of the + // width, so the tail is masked down to the additive identity before it's summed. That mask is + // only needed because addition is non-idempotent; an idempotent operation such as a search + // could fold the overlapping tail in directly. + switch (buffer.Length) + { + // One or two Vector256's worth of data. + case 15: + case 14: + case 13: + case 12: + case 11: + case 10: + case 9: + case 8: + { + Vector256 beg = Vector256.Create(buffer); + Vector256 end = Vector256.Create(buffer.Slice(buffer.Length - Vector256.Count)); + + Vector256 msk = CreateRemainderMask256(buffer.Length - Vector256.Count); + end = Vector256.ConditionalSelect(msk, end, Vector256.Zero); + + result = Vector256.Sum(beg + end); + break; + } + + // One or two Vector128's worth of data. + case 7: + case 6: + case 5: + case 4: + { + Vector128 beg = Vector128.Create(buffer); + Vector128 end = Vector128.Create(buffer.Slice(buffer.Length - Vector128.Count)); + + Vector128 msk = CreateRemainderMask128(buffer.Length - Vector128.Count); + end = Vector128.ConditionalSelect(msk, end, Vector128.Zero); + + result = Vector128.Sum(beg + end); + break; + } + + // Smaller than a single vector: each case falls through to the next, accumulating one + // element per label. + case 3: + { + result += buffer[2]; + goto case 2; + } + + case 2: + { + result += buffer[1]; + goto case 1; + } + + case 1: + { + result += buffer[0]; + goto case 0; + } + + case 0: + { + break; + } + } + + return result; + } + + // Builds a mask whose last `keepLast` lanes are all-bits-set and the rest zero, so an overlapping + // tail load can be folded in without double-counting the lanes the head already covered. + // TensorPrimitives uses an internal table-based helper. The mask is only a bit pattern keyed on + // lane width, so it's built with the same-width integer Indices ([0, 1, 2, ...]) and reinterpreted + // to T: integer comparisons are cheaper than floating-point ones, so a float/double table would + // still compare as int/long rather than in its own element type. + private static Vector256 CreateRemainderMask256(int keepLast) + where T : unmanaged, INumberBase + { + Debug.Assert(sizeof(T) == 4); + + Vector256 firstKept = Vector256.Create(Vector256.Count - keepLast); + return Vector256.GreaterThanOrEqual(Vector256.Indices, firstKept).As(); + } + + private static Vector128 CreateRemainderMask128(int keepLast) + where T : unmanaged, INumberBase + { + Debug.Assert(sizeof(T) == 4); + + Vector128 firstKept = Vector128.Create(Vector128.Count - keepLast); + return Vector128.GreaterThanOrEqual(Vector128.Indices, firstKept).As(); + } + // + + // + // Sums a buffer with an unrolled vector loop plus a masked, jump-table remainder. + private static T SumVector128(ReadOnlySpan buffer) + where T : unmanaged, INumberBase + { + Debug.Assert(Vector128.IsHardwareAccelerated && Vector128.IsSupported); + Debug.Assert(buffer.Length >= Vector128.Count); + + // Preload the last full vector, overlapping the tail. Any sub-vector remainder is folded in + // from here (masked) by case 0 of the switch below, so the loop never falls out to a separate + // scalar tail—the same shape TensorPrimitives uses. + Vector128 end = Vector128.Create(buffer.Slice(buffer.Length - Vector128.Count)); + + // A production implementation would also align the buffer to a vector boundary and, for + // very large inputs, use non-temporal loads/stores so the data doesn't evict useful + // cache lines. Both are omitted here; see TensorPrimitives for a complete treatment. + + Vector128 sum = Vector128.Zero; + + // Only pay for the four independent accumulators when there's enough data to unroll; + // smaller payloads skip straight to the remainder below. Four vectors per iteration lets + // the accumulators pipeline; Vector128.Create reads the first Vector128.Count elements. + if (buffer.Length >= Vector128.Count * 4) + { + Vector128 sum0 = Vector128.Zero; + Vector128 sum1 = Vector128.Zero; + Vector128 sum2 = Vector128.Zero; + Vector128 sum3 = Vector128.Zero; + + do + { + sum0 += Vector128.Create(buffer); + sum1 += Vector128.Create(buffer.Slice(Vector128.Count)); + sum2 += Vector128.Create(buffer.Slice(Vector128.Count * 2)); + sum3 += Vector128.Create(buffer.Slice(Vector128.Count * 3)); + + buffer = buffer.Slice(Vector128.Count * 4); + } + while (buffer.Length >= Vector128.Count * 4); + + // Combine pairwise so the two independent adds can pipeline. + sum = (sum0 + sum1) + (sum2 + sum3); + } + + // Split the remainder into its full vectors and a sub-vector tail. The full vectors fall + // through the jump table; the tail lands in case 0, where the preloaded end is masked so only + // the trailing elements the full vectors didn't already cover are added. + (int blocks, int trailing) = Math.DivRem(buffer.Length, Vector128.Count); + + switch (blocks) + { + case 3: + { + sum += Vector128.Create(buffer.Slice(Vector128.Count * 2)); + goto case 2; + } + + case 2: + { + sum += Vector128.Create(buffer.Slice(Vector128.Count)); + goto case 1; + } + + case 1: + { + sum += Vector128.Create(buffer); + goto case 0; + } + + case 0: + { + Vector128 msk = CreateRemainderMask128(trailing); + sum += Vector128.ConditionalSelect(msk, end, Vector128.Zero); + break; + } + } + + // Horizontally add the lanes into a single scalar. + return Vector128.Sum(sum); + } + // +} diff --git a/docs/standard/snippets/simd/csharp/HardwareIntrinsicsExample.cs b/docs/standard/snippets/simd/csharp/HardwareIntrinsicsExample.cs new file mode 100644 index 0000000000000..743c0bea5652f --- /dev/null +++ b/docs/standard/snippets/simd/csharp/HardwareIntrinsicsExample.cs @@ -0,0 +1,38 @@ +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.Arm; +using System.Runtime.Intrinsics.Wasm; +using System.Runtime.Intrinsics.X86; + +namespace SimdSnippets; + +public static class HardwareIntrinsicsExample +{ + // + // Illustrates per-platform lightup. The portable '(vector & mask) == Zero' below + // already lowers optimally, so prefer it unless a specific instruction measurably wins. + public static bool AllBitsClear(Vector128 vector, Vector128 mask) + { + if (Sse41.IsSupported) + { + // x86/x64: a single ptest instruction. + return Sse41.TestZ(vector, mask); + } + else if (AdvSimd.Arm64.IsSupported) + { + // Arm64: AND, then reduce the maximum byte across every lane. + Vector128 anded = AdvSimd.And(vector, mask); + return AdvSimd.Arm64.MaxAcross(anded).ToScalar() == 0; + } + else if (PackedSimd.IsSupported) + { + // WebAssembly: AND, then test whether any lane is non-zero. + return !PackedSimd.AnyTrue(PackedSimd.And(vector, mask)); + } + else + { + // Portable fallback for any other platform. + return (vector & mask) == Vector128.Zero; + } + } + // +} diff --git a/docs/standard/snippets/simd/csharp/NumericsExamples.cs b/docs/standard/snippets/simd/csharp/NumericsExamples.cs new file mode 100644 index 0000000000000..dbe67213b0af3 --- /dev/null +++ b/docs/standard/snippets/simd/csharp/NumericsExamples.cs @@ -0,0 +1,74 @@ +using System.Numerics; + +namespace SimdSnippets; + +public static class NumericsExamples +{ + public static void SimpleVectors() + { + // + Vector2 v1 = Vector2.Create(0.1f, 0.2f); + Vector2 v2 = Vector2.Create(1.1f, 2.2f); + Vector2 sum = v1 + v2; + // + + // + float dot = Vector2.Dot(v1, v2); + float distance = Vector2.Distance(v1, v2); + Vector2 clamped = Vector2.Clamp(v1, Vector2.Zero, Vector2.One); + // + + Console.WriteLine($"sum={sum}, dot={dot}, distance={distance}, clamped={clamped}"); + } + + public static void Matrices() + { + // + Matrix4x4 m1 = Matrix4x4.Create( + 1.1f, 1.2f, 1.3f, 1.4f, + 2.1f, 2.2f, 3.3f, 4.4f, + 3.1f, 3.2f, 3.3f, 3.4f, + 4.1f, 4.2f, 4.3f, 4.4f); + + Matrix4x4 m2 = Matrix4x4.Transpose(m1); + Matrix4x4 product = Matrix4x4.Multiply(m1, m2); + // + + Console.WriteLine($"product.M11={product.M11}"); + } + + // + // Illustrative: element-wise add with Vector. In practice, prefer the already-accelerated + // TensorPrimitives.Add, which is optimized for every Vector.IsSupported element type. + public static double[] Add(double[] left, double[] right) + { + ArgumentNullException.ThrowIfNull(left); + ArgumentNullException.ThrowIfNull(right); + ArgumentOutOfRangeException.ThrowIfNotEqual(right.Length, left.Length); + + double[] result = new double[left.Length]; + + int i = 0; + + // Vector.Count is a JIT-time constant, so the compiler optimizes the loop bound. + int lastVectorStart = left.Length - Vector.Count; + + for (; i <= lastVectorStart; i += Vector.Count) + { + Vector v1 = Vector.Create(left.AsSpan(i)); + Vector v2 = Vector.Create(right.AsSpan(i)); + (v1 + v2).CopyTo(result, i); + } + + // Process any remaining elements that don't fill a full vector. + // Simplified for illustration: a scalar tail isn't optimal. A vectorized + // remainder that reprocesses the last full vector avoids the per-element loop. + for (; i < left.Length; i++) + { + result[i] = left[i] + right[i]; + } + + return result; + } + // +} diff --git a/docs/standard/snippets/simd/csharp/Program.cs b/docs/standard/snippets/simd/csharp/Program.cs new file mode 100644 index 0000000000000..da03a283373c6 --- /dev/null +++ b/docs/standard/snippets/simd/csharp/Program.cs @@ -0,0 +1,48 @@ +using System.Numerics; +using System.Linq; +using System.Runtime.Intrinsics; +using SimdSnippets; + +// Verification runner. Not part of the published snippets. + +Console.WriteLine($"Vector.IsHardwareAccelerated: {Vector.IsHardwareAccelerated}"); +Console.WriteLine($"Vector128.IsHardwareAccelerated: {Vector128.IsHardwareAccelerated}"); +Console.WriteLine($"Vector256.IsHardwareAccelerated: {Vector256.IsHardwareAccelerated}"); +Console.WriteLine($"Vector512.IsHardwareAccelerated: {Vector512.IsHardwareAccelerated}"); +Console.WriteLine($"Vector.Count: {Vector.Count}"); +Console.WriteLine(); + +NumericsExamples.SimpleVectors(); +NumericsExamples.Matrices(); + +double[] left = [1, 2, 3, 4, 5, 6, 7]; +double[] right = [10, 20, 30, 40, 50, 60, 70]; +Console.WriteLine($"Vector Add: [{string.Join(", ", NumericsExamples.Add(left, right))}]"); +Console.WriteLine(); + +int[] numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]; +double[] reals = [1.5, 2.5, 3.0, 4.0, 5.0]; +Console.WriteLine($"Sum(1..10) = {AdvancedVectorization.Sum(numbers)}"); +Console.WriteLine($"Sum(reals) = {AdvancedVectorization.Sum(reals)}"); +Console.WriteLine($"Sum(small) = {AdvancedVectorization.Sum(new int[] { 1, 2, 3 })}"); +Console.WriteLine($"Sum(1..37) = {AdvancedVectorization.Sum(Enumerable.Range(1, 37).ToArray())}"); +Console.WriteLine($"Contains(7) = {SimpleVectorization.Contains(numbers, 7)}"); +Console.WriteLine($"Contains(42) = {SimpleVectorization.Contains(numbers, 42)}"); +Console.WriteLine(); + +byte[] ascii = "Hello, SIMD!"u8.ToArray(); +byte[] notAscii = [0x48, 0x65, 0xFF, 0x6C]; +Console.WriteLine($"IsAscii(ascii) = {SimpleVectorization.IsAscii(ascii)}"); +Console.WriteLine($"IsAscii(notAscii) = {SimpleVectorization.IsAscii(notAscii)}"); +Console.WriteLine(); + +Vector128 highBits = Vector128.Create(0b_1000_0000); +Console.WriteLine($"AllBitsClear(zero, highBits) = {HardwareIntrinsicsExample.AllBitsClear(Vector128.Zero, highBits)}"); +Console.WriteLine($"AllBitsClear(0xFF.., highBits) = {HardwareIntrinsicsExample.AllBitsClear(Vector128.AllBitsSet, highBits)}"); +Console.WriteLine(); + +float[] a = [1, 2, 3, 4]; +float[] b = [5, 6, 7, 8]; +float[] c = [1, 1, 1, 1]; +Console.WriteLine($"MultiplyAdd = [{string.Join(", ", TensorPrimitivesExample.MultiplyAdd(a, b, c))}]"); +Console.WriteLine($"CosineSimilarity(a, a) = {TensorPrimitivesExample.CosineSimilarity(a, a)}"); diff --git a/docs/standard/snippets/simd/csharp/SimdSnippets.csproj b/docs/standard/snippets/simd/csharp/SimdSnippets.csproj new file mode 100644 index 0000000000000..4df2fd22f5768 --- /dev/null +++ b/docs/standard/snippets/simd/csharp/SimdSnippets.csproj @@ -0,0 +1,15 @@ + + + + Exe + net11.0 + enable + enable + preview + + + + + + + diff --git a/docs/standard/snippets/simd/csharp/SimpleVectorization.cs b/docs/standard/snippets/simd/csharp/SimpleVectorization.cs new file mode 100644 index 0000000000000..e9d83d12f7bb7 --- /dev/null +++ b/docs/standard/snippets/simd/csharp/SimpleVectorization.cs @@ -0,0 +1,76 @@ +using System.Diagnostics; +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SimdSnippets; + +public static class SimpleVectorization +{ + // + // Idempotent search that re-processes the final vector instead of a scalar loop. + public static bool Contains(ReadOnlySpan buffer, int searched) + { + Debug.Assert(Vector128.IsHardwareAccelerated && buffer.Length >= Vector128.Count); + + Vector128 values = Vector128.Create(searched); + ReadOnlySpan remaining = buffer; + + while (remaining.Length >= Vector128.Count) + { + if (Vector128.EqualsAny(Vector128.Create(remaining), values)) + { + return true; + } + remaining = remaining.Slice(Vector128.Count); + } + + // If any elements remain, re-check the last full vector in the buffer. + if (!remaining.IsEmpty) + { + Vector128 tail = Vector128.Create(buffer.Slice(buffer.Length - Vector128.Count)); + if (Vector128.EqualsAny(tail, values)) + { + return true; + } + } + + return false; + } + // + + public static bool IsValidAscii(byte value) => (value & 0b1000_0000) == 0; + + // Returns true when every byte in the vector is a valid ASCII value (0-127). + // Anding with the high bit and comparing to zero avoids a movemask extraction. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static bool IsValidAscii(Vector128 vector) => + (vector & Vector128.Create(0b1000_0000)) == Vector128.Zero; + + public static bool IsAscii(ReadOnlySpan buffer) + { + ReadOnlySpan remaining = buffer; + + if (Vector128.IsHardwareAccelerated) + { + while (remaining.Length >= Vector128.Count) + { + if (!IsValidAscii(Vector128.Create(remaining))) + { + return false; + } + remaining = remaining.Slice(Vector128.Count); + } + } + + // Scalar path handles the remainder and short or non-accelerated inputs. + foreach (byte value in remaining) + { + if (!IsValidAscii(value)) + { + return false; + } + } + + return true; + } +} diff --git a/docs/standard/snippets/simd/csharp/TensorPrimitivesExample.cs b/docs/standard/snippets/simd/csharp/TensorPrimitivesExample.cs new file mode 100644 index 0000000000000..b3c3e34b45f17 --- /dev/null +++ b/docs/standard/snippets/simd/csharp/TensorPrimitivesExample.cs @@ -0,0 +1,23 @@ +using System.Numerics.Tensors; + +namespace SimdSnippets; + +public static class TensorPrimitivesExample +{ + // + // Computes result = (left * right) + addend over the whole span, vectorized internally. + public static float[] MultiplyAdd(float[] left, float[] right, float[] addend) + { + float[] result = new float[left.Length]; + + TensorPrimitives.Multiply(left, right, result); + TensorPrimitives.Add(result, addend, result); + + return result; + } + + // Higher-level reductions are available too. + public static float CosineSimilarity(float[] left, float[] right) => + TensorPrimitives.CosineSimilarity(left, right); + // +} From 853e720e483a0bd81a1339264f2363ef75bc6876 Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Sun, 19 Jul 2026 12:44:44 -0700 Subject: [PATCH 2/6] Fix markdownlint violations in SIMD article Escape the angle brackets in the Vector heading (MD033 / OPS disallowed-html-tag) and drop a trailing space in the config-knob callout (MD009). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/standard/simd.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/standard/simd.md b/docs/standard/simd.md index 2e5c45b2d4c43..fa1904bbc1bde 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -48,7 +48,7 @@ The matrix types support matrix math such as transpose and multiplication: :::code language="csharp" source="./snippets/simd/csharp/NumericsExamples.cs" id="MatrixMultiply"::: -## Vector +## Vector\ `Vector` represents a variable-width vector of a primitive numeric type. Its length is fixed for the lifetime of the process, but the value of `Vector.Count` depends on the CPU that runs the code. The Just-In-Time (JIT) compiler treats `Count` as a constant, so loops written against it optimize well. @@ -193,7 +193,7 @@ Beyond those two, the runtime recognizes a knob per logical grouping of instruct > [!IMPORTANT] > These are diagnostic tools, meant primarily for testing and validation—exercising each code path, reproducing a hardware-specific issue, or confirming a fallback. They aren't designed for general or production use, and they aren't a stability contract: the set below is what .NET 11 recognizes; earlier releases exposed a different set—the baseline and AVX-512 knobs in particular were reconfigured—so confirm the names against the runtime version you target. -> +> > They also have limits on what they reach. Because they gate JIT decisions, they don't affect code that was already compiled ahead of time through ReadyToRun or Native AOT, and they don't necessarily affect internal routines the runtime and core libraries use themselves. Treat them as a way to steer your own JIT-compiled code, not a global off switch for an instruction set. The base switch and the width caps apply on every architecture: From 5ffa72da1b58e6cab0dbeb817cfa6b383885b4cc Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Sun, 19 Jul 2026 12:51:34 -0700 Subject: [PATCH 3/6] Correct Arm64 RDM and RCpc knob descriptions Match the EnableArm64Rdm/Rcpc knob names: RDM (Rounding Doubling Multiply Accumulate), not RDMA; and RCpc casing without the duplicated word. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/standard/simd.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/standard/simd.md b/docs/standard/simd.md index fa1904bbc1bde..8b9530e15ff51 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -241,11 +241,11 @@ On Arm64, each logical grouping of instruction sets has its own switch: | `EnableArm64Crc32` | `1` | CRC32 | | `EnableArm64Dczva` | `1` | `DC ZVA` cache-zeroing | | `EnableArm64Dp` | `1` | Dot Product | -| `EnableArm64Rdm` | `1` | Rounding Double Multiply (RDMA) | +| `EnableArm64Rdm` | `1` | Rounding Doubling Multiply Accumulate (RDM) | | `EnableArm64Sha1` | `1` | SHA1 | | `EnableArm64Sha256` | `1` | SHA256 | -| `EnableArm64Rcpc` | `1` | Release Consistent processor consistent (RCPC) | -| `EnableArm64Rcpc2` | `1` | RCPC2 | +| `EnableArm64Rcpc` | `1` | Release-Consistent, processor-consistent ordering (RCpc) | +| `EnableArm64Rcpc2` | `1` | RCpc2 | | `EnableArm64Cssc` | `0` | Common Short Sequence Compression (CSSC) | | `EnableArm64Sve` | `1` | Scalable Vector Extension (SVE) | | `EnableArm64Sve2` | `1` | SVE2 | From b0344d809df1e1715067289c8c8343bb17261e3e Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Sun, 19 Jul 2026 12:53:27 -0700 Subject: [PATCH 4/6] Guard the Contains sample against sub-vector buffers The overlapping-tail load underflows for a non-empty buffer shorter than one vector; fall back to a scalar scan in that case so the sample is correct for any length. Verify small and empty buffers in the runner. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/standard/snippets/simd/csharp/Program.cs | 3 +++ .../simd/csharp/SimpleVectorization.cs | 19 +++++++++++++++---- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/docs/standard/snippets/simd/csharp/Program.cs b/docs/standard/snippets/simd/csharp/Program.cs index da03a283373c6..be7aeb320342f 100644 --- a/docs/standard/snippets/simd/csharp/Program.cs +++ b/docs/standard/snippets/simd/csharp/Program.cs @@ -28,6 +28,9 @@ Console.WriteLine($"Sum(1..37) = {AdvancedVectorization.Sum(Enumerable.Range(1, 37).ToArray())}"); Console.WriteLine($"Contains(7) = {SimpleVectorization.Contains(numbers, 7)}"); Console.WriteLine($"Contains(42) = {SimpleVectorization.Contains(numbers, 42)}"); +Console.WriteLine($"Contains(small,2) = {SimpleVectorization.Contains(new int[] { 1, 2, 3 }, 2)}"); +Console.WriteLine($"Contains(small,9) = {SimpleVectorization.Contains(new int[] { 1, 2, 3 }, 9)}"); +Console.WriteLine($"Contains(empty) = {SimpleVectorization.Contains(Array.Empty(), 1)}"); Console.WriteLine(); byte[] ascii = "Hello, SIMD!"u8.ToArray(); diff --git a/docs/standard/snippets/simd/csharp/SimpleVectorization.cs b/docs/standard/snippets/simd/csharp/SimpleVectorization.cs index e9d83d12f7bb7..8696c076d72c3 100644 --- a/docs/standard/snippets/simd/csharp/SimpleVectorization.cs +++ b/docs/standard/snippets/simd/csharp/SimpleVectorization.cs @@ -10,7 +10,7 @@ public static class SimpleVectorization // Idempotent search that re-processes the final vector instead of a scalar loop. public static bool Contains(ReadOnlySpan buffer, int searched) { - Debug.Assert(Vector128.IsHardwareAccelerated && buffer.Length >= Vector128.Count); + Debug.Assert(Vector128.IsHardwareAccelerated); Vector128 values = Vector128.Create(searched); ReadOnlySpan remaining = buffer; @@ -24,11 +24,22 @@ public static bool Contains(ReadOnlySpan buffer, int searched) remaining = remaining.Slice(Vector128.Count); } - // If any elements remain, re-check the last full vector in the buffer. - if (!remaining.IsEmpty) + if (remaining.IsEmpty) + { + return false; + } + + // A partial vector remains. When the buffer holds at least one full vector, + // re-check the last one (overlapping the tail); otherwise scan the few elements directly. + if (buffer.Length >= Vector128.Count) { Vector128 tail = Vector128.Create(buffer.Slice(buffer.Length - Vector128.Count)); - if (Vector128.EqualsAny(tail, values)) + return Vector128.EqualsAny(tail, values); + } + + foreach (int value in remaining) + { + if (value == searched) { return true; } From 03d8f40802a97493f5418620129e0891c334d4e9 Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Mon, 20 Jul 2026 09:51:45 -0700 Subject: [PATCH 5/6] Apply editorial review feedback to SIMD article Sentence-case the title/heading/toc, drop redundant displayProperty=fullName from namespace xrefs, and take the copyedits from review (may->might, wording, and acronym spacing). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/fundamentals/toc.yml | 2 +- docs/standard/simd.md | 36 ++++++++++++++++++------------------ 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/fundamentals/toc.yml b/docs/fundamentals/toc.yml index e85be24e5a53e..fe707ee890ac6 100644 --- a/docs/fundamentals/toc.yml +++ b/docs/fundamentals/toc.yml @@ -411,7 +411,7 @@ items: href: ../standard/memory-and-spans/index.md - name: Memory and Span usage guidelines href: ../standard/memory-and-spans/memory-t-usage-guidelines.md - - name: SIMD and Hardware Intrinsics + - name: SIMD and hardware intrinsics href: ../standard/simd.md - name: Value tuples href: ../standard/value-tuples.md diff --git a/docs/standard/simd.md b/docs/standard/simd.md index 8b9530e15ff51..dc36930d9f5e7 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -1,5 +1,5 @@ --- -title: Use SIMD and Hardware Intrinsics in .NET +title: Use SIMD and hardware intrinsics in .NET description: Learn about the layered SIMD support in .NET, from System.Numerics vector types to cross-platform Vector64/128/256/512 APIs, hardware intrinsics, and TensorPrimitives. author: FIVIL ms.author: tagoo @@ -7,7 +7,7 @@ ms.date: 07/18/2026 ai-usage: ai-assisted --- -# Use SIMD and Hardware Intrinsics in .NET +# Use SIMD and hardware intrinsics in .NET SIMD (single instruction, multiple data) is hardware support for applying one operation to multiple pieces of data in parallel with a single instruction. Vectorized code processes several values per iteration instead of one, which can greatly increase throughput for the kind of numeric, scientific, graphics, text-processing, and data-parallel work where the same operation repeats over a buffer. The trade-off is added complexity, so it pays off most when the input is large enough and the win is confirmed with measurements. @@ -17,13 +17,13 @@ SIMD (single instruction, multiple data) is hardware support for applying one op | API | Namespace | When to use it | | --- | --- | --- | -| Fixed-purpose vector and matrix types | | Graphics and geometry math with 2-4 element vectors, matrices, quaternions, and planes. | -| `Vector` | | Portable, variable-width vectorization when you don't need per-platform control. | -| `Vector64`, `Vector128`, `Vector256`, `Vector512` | | Cross-platform, fixed-width vectorization with fine-grained control. This is the recommended starting point for new vectorized algorithms. | -| Hardware intrinsics | , , | Specific processor instructions that the higher-level APIs don't expose, for the last bit of performance on a hot path. | -| `TensorPrimitives` | | Ready-made, vectorized math over spans. It does the vectorization for you. | +| Fixed-purpose vector and matrix types | | Graphics and geometry math with 2-4 element vectors, matrices, quaternions, and planes. | +| `Vector` | | Portable, variable-width vectorization when you don't need per-platform control. | +| `Vector64`, `Vector128`, `Vector256`, `Vector512` | | Cross-platform, fixed-width vectorization with fine-grained control. This is the recommended starting point for new vectorized algorithms. | +| Hardware intrinsics | , , | Specific processor instructions that the higher-level APIs don't expose, for the last bit of performance on a hot path. | +| `TensorPrimitives` | | Ready-made, vectorized math over spans. It does the vectorization for you. | -Where these APIs overlap, they relate through layers of abstraction. The generic vector types are the foundational interchange types that the other layers pass around, so they're technically the lowest level: the variable-width `Vector`, which grows to whatever width the running hardware supports, and the fixed-width `Vector64` through `Vector512`. The platform-specific hardware intrinsics in , , and operate on those types, each mapping directly to an individual processor instruction. The cross-platform operations exposed on the generic types sit a step above the platform-specific intrinsics, lowering to them for each target. Higher still are the managed APIs that operate over entire buffers—vectorized methods on `Span` and `string`, and —which build on the layers below, so you get SIMD acceleration without hand-writing any of it. The fixed-shape types are domain-specific convenience types for graphics and geometry rather than part of this interchange stack. +Where these APIs overlap, they relate through layers of abstraction. The generic vector types are the foundational interchange types that the other layers pass around, so they're technically the lowest level: the variable-width `Vector`, which grows to whatever width the running hardware supports, and the fixed-width `Vector64` through `Vector512`. The platform-specific hardware intrinsics in , , and operate on those types, each mapping directly to an individual processor instruction. The cross-platform operations exposed on the generic types sit a step above the platform-specific intrinsics, lowering to them for each target. Higher still are the managed APIs that operate over entire buffers—vectorized methods on `Span` and `string`, and —which build on the layers below, so you get SIMD acceleration without hand-writing any of it. The fixed-shape types are domain-specific convenience types for graphics and geometry rather than part of this interchange stack. The rest of this article works through these APIs from the highest level to the lowest, then covers testing, benchmarking, and best practices. @@ -68,12 +68,12 @@ SIMD-accelerated types work even on hardware or JIT configurations that don't su These properties are turned into constants by the JIT, so the branches you don't take are eliminated and there's no runtime cost to checking them. Don't cache the values; read them directly where you need them. The same applies to the `Count` properties (for example, `Vector128.Count`), which are also JIT-time constants. -Most operations on an accelerated width are themselves accelerated, but it isn't guaranteed for every operation. For example, floating-point division may be accelerated where integer division isn't. When `Vector256` is accelerated, `Vector128` usually is too, but there's no guarantee, so check each width you use. +Most operations on an accelerated width are themselves accelerated, but it isn't guaranteed for every operation. For example, floating-point division might be accelerated where integer division isn't. When `Vector256` is accelerated, `Vector128` usually is too, but there's no guarantee, so check each width you use. > [!TIP] > If an operation you need isn't accelerated on a platform you care about, or you'd like a new cross-platform API, file an issue on [dotnet/runtime](https://github.com/dotnet/runtime/issues). The same applies to codegen improvements. -Not every element type is valid for every vector. `Vector128` and its siblings support the primitive numeric types (`byte`, `sbyte`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `float`, `double`, `nint`, and `nuint`) today, and that set may grow to include other types in the future. Use `Vector128.IsSupported` to determine whether a given `T` is valid, which is especially useful from generic code. +Not every element type is valid for every vector. `Vector128` and its siblings support the primitive numeric types (`byte`, `sbyte`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `float`, `double`, `nint`, and `nuint`) today, and that set might grow to include other types in the future. Use `Vector128.IsSupported` to determine whether a given `T` is valid, which is especially useful from generic code. Types that aren't supported, such as `char` and `bool`, can still be vectorized by reinterpreting the buffer as a supported type of the same size. Use to reinterpret a span—for example, `char` to `ushort`—or the vector's `As` method to reinterpret a vector you already hold. Reinterpretation only changes the type, not the underlying bits, so it's your responsibility to keep the data well-formed: a `bool` must stay `0` or `1`, and a `char` must remain a valid UTF-16 code unit. If a vectorized operation could produce an out-of-range value, take care to normalize the result before writing it back. @@ -84,11 +84,11 @@ Types that aren't supported, such as `char` and `bool`, can still be vectorized Each width has a generic type (`Vector128`) for the data and a non-generic static class () that holds most of the operations, including static factory methods like `Create` and `Load`. Operators such as `+`, `&`, and `<<` are the idiomatic way to express arithmetic and bit operations; prefer them over the named method equivalents to avoid operator-precedence bugs and improve readability. For algorithms that depend on byte order, branch on , which the JIT also folds to a constant. > [!NOTE] -> On x86/x64, `Vector256` operations are generally treated as two independent 128-bit "lanes". For most element-wise operations this is transparent, but operations that cross lanes (such as shuffles or pairwise/horizontal operations) may behave differently or cost more than the `Vector128` equivalent. Confirm with benchmarks before assuming a wider vector is faster. +> On x86/x64, `Vector256` operations are generally treated as two independent 128-bit "lanes". For most element-wise operations this is transparent, but operations that cross lanes (such as shuffles or pairwise/horizontal operations) might behave differently or cost more than the `Vector128` equivalent. Confirm with benchmarks before assuming a wider vector is faster. ### Common operations -`Vector128` and its wider siblings expose a large API surface. You don't need to memorize it—know the categories and look up the details when you need them. Every operation has a software fallback for platforms that can't accelerate it. The table below covers essentially the whole surface. +`Vector128` and its wider siblings expose a large API surface. You don't need to memorize it—know the categories and look up the details when you need them. Every operation has a software fallback for platforms that can't accelerate it. The following table covers essentially the whole surface. | Category | What it does | Representative APIs | | --- | --- | --- | @@ -132,7 +132,7 @@ There are two distinct fallbacks. A buffer that's too small for even the narrowe :::code language="csharp" source="./snippets/simd/csharp/AdvancedVectorization.cs" id="VectorSmall"::: -`sizeof(T)` is also a JIT-time constant (and, with the `unmanaged` constraint, doesn't require an `unsafe` context in C# 15 / .NET 11), so `SumVectorSmall` dispatches on the element width to a table sized for the widest vector's worth of elements—the same approach uses. Only the 4-byte table is shown; the 1-, 2-, and 8-byte tables share its shape. Its larger cases fold the leftover with a `Vector256` or `Vector128` using two overlapping loads—one from the start, one from the end—so the wider-remainder handling for the omitted `Vector512`/`Vector256` paths lives right in the jump table. The two loads overlap whenever the length isn't an exact multiple of the width, so the tail is masked to the additive identity with `ConditionalSelect` before it's summed. That mask is only needed because addition is non-idempotent; an idempotent operation such as a search could fold the overlapping tail in directly. A buffer on hardware with no vectorization at all falls through to `SumScalar`, an ordinary scalar loop. +`sizeof(T)` is also a JIT-time constant (and, with the `unmanaged` constraint, doesn't require an `unsafe` context in C# 15 / .NET 11), so `SumVectorSmall` dispatches on the element width to a table sized for the widest vector's worth of elements—the same approach uses. Only the 4-byte table is shown; the 1-byte, 2-byte, and 8-byte tables share its shape. Its larger cases fold the leftover with a `Vector256` or `Vector128` using two overlapping loads—one from the start, one from the end—so the wider-remainder handling for the omitted `Vector512`/`Vector256` paths lives right in the jump table. The two loads overlap whenever the length isn't an exact multiple of the width, so the tail is masked to the additive identity with `ConditionalSelect` before it's summed. That mask is only needed because addition is non-idempotent; an idempotent operation such as a search could fold the overlapping tail in directly. A buffer on hardware with no vectorization at all falls through to `SumScalar`, an ordinary scalar loop. ### Loop over the input and handle the remainder @@ -153,7 +153,7 @@ An **idempotent** operation such as searching for a value can reprocess the over ### Load and store vectors safely -For most code, `Vector128.Create(span)` and are the simplest way to move data between a span and a vector, and the JIT keeps them efficient. When you need the lower-level load and store—for example, to walk a buffer by managed reference—prefer the and overloads that take a managed reference and an `nuint` element offset. Unlike the pointer-based `Load`/`Store` overloads, they don't require pinning the buffer, and unlike raw reference arithmetic they don't need you to manually advance a `ref`. Both alternatives are easy to get wrong in ways that introduce garbage-collector holes or access violations. +For most code, `Vector128.Create(span)` and are the simplest way to move data between a span and a vector, and the JIT keeps them efficient. When you need the lower-level load and store—for example, to walk a buffer by managed reference—prefer the and overloads that take a managed reference and an `nuint` element offset. Unlike the pointer-based `Load`/`Store` overloads, they don't require pinning the buffer, and unlike raw reference arithmetic, they don't need you to manually advance a `ref`. Both alternatives are easy to get wrong in ways that introduce garbage-collector holes or access violations. So that empty buffers don't throw, get the starting reference from (or for arrays) rather than `ref span[0]`. @@ -192,9 +192,9 @@ To exercise every path on a single machine, run the test suite once with no over Beyond those two, the runtime recognizes a knob per logical grouping of instruction sets, each prefixed with `DOTNET_`. A single knob can cover several related instruction sets—`EnableAVX2`, for instance, gates AVX2 along with BMI1, BMI2, F16C, FMA, LZCNT, and MOVBE. Setting a knob to `0` disables its whole group and everything layered on top of it. Setting it to `1` (the default for most) allows the group, but the hardware must still actually support it—enabling a knob the current CPU lacks is ignored, so you can only ever narrow what's used, never force an unsupported instruction on and break yourself. `DOTNET_EnableHWIntrinsic=0` is the big hammer—it turns off everything down to the base, so `Vector128`, `Vector64`, and `Vector` all report no acceleration and the code falls to its software path. > [!IMPORTANT] -> These are diagnostic tools, meant primarily for testing and validation—exercising each code path, reproducing a hardware-specific issue, or confirming a fallback. They aren't designed for general or production use, and they aren't a stability contract: the set below is what .NET 11 recognizes; earlier releases exposed a different set—the baseline and AVX-512 knobs in particular were reconfigured—so confirm the names against the runtime version you target. +> These are diagnostic tools, meant primarily for testing and validation—exercising each code path, reproducing a hardware-specific issue, or confirming a fallback. They aren't designed for general or production use, and they aren't a stability contract. The following set is what .NET 11 recognizes; earlier releases exposed a different set—the baseline and AVX-512 knobs in particular were reconfigured—so confirm the names against the runtime version you target. > -> They also have limits on what they reach. Because they gate JIT decisions, they don't affect code that was already compiled ahead of time through ReadyToRun or Native AOT, and they don't necessarily affect internal routines the runtime and core libraries use themselves. Treat them as a way to steer your own JIT-compiled code, not a global off switch for an instruction set. +> These tools also have limits on what they reach. Because they gate JIT decisions, they don't affect code that was already compiled ahead of time through ReadyToRun or Native AOT, and they don't necessarily affect internal routines the runtime and core libraries use themselves. Treat them as a way to steer your own JIT-compiled code, not a global off switch for an instruction set. The base switch and the width caps apply on every architecture: @@ -259,7 +259,7 @@ A knob defaulting to `0` (for example, `EnableAVX10v2` or `EnableArm64Cssc`) gat ## Benchmark to confirm the win -Vectorization adds complexity, so measure that it pays off before you keep it. Use [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet), and use the same environment variables above to compare the scalar, `Vector128`, and `Vector256` implementations in one run. BenchmarkDotNet's disassembly diagnoser can also emit the generated assembly, which is invaluable when tuning high-performance code. +Vectorization adds complexity, so measure that it pays off before you keep it. Use [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet), and use the same environment variables shown previously to compare the scalar, `Vector128`, and `Vector256` implementations in one run. BenchmarkDotNet's disassembly diagnoser can also emit the generated assembly, which is invaluable when tuning high-performance code. A few things to keep in mind: @@ -278,5 +278,5 @@ A few things to keep in mind: ## See also -- +- - From 5fc5938a54538c19911e2ad42ca658a070e4c9be Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Mon, 20 Jul 2026 12:35:21 -0700 Subject: [PATCH 6/6] Update docs/standard/simd.md Co-authored-by: Bill Wagner --- docs/standard/simd.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standard/simd.md b/docs/standard/simd.md index dc36930d9f5e7..43b23787c424c 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -132,7 +132,7 @@ There are two distinct fallbacks. A buffer that's too small for even the narrowe :::code language="csharp" source="./snippets/simd/csharp/AdvancedVectorization.cs" id="VectorSmall"::: -`sizeof(T)` is also a JIT-time constant (and, with the `unmanaged` constraint, doesn't require an `unsafe` context in C# 15 / .NET 11), so `SumVectorSmall` dispatches on the element width to a table sized for the widest vector's worth of elements—the same approach uses. Only the 4-byte table is shown; the 1-byte, 2-byte, and 8-byte tables share its shape. Its larger cases fold the leftover with a `Vector256` or `Vector128` using two overlapping loads—one from the start, one from the end—so the wider-remainder handling for the omitted `Vector512`/`Vector256` paths lives right in the jump table. The two loads overlap whenever the length isn't an exact multiple of the width, so the tail is masked to the additive identity with `ConditionalSelect` before it's summed. That mask is only needed because addition is non-idempotent; an idempotent operation such as a search could fold the overlapping tail in directly. A buffer on hardware with no vectorization at all falls through to `SumScalar`, an ordinary scalar loop. +`sizeof(T)` is also a JIT-time constant, so `SumVectorSmall` dispatches on the element width to a table sized for the widest vector's worth of elements—the same approach uses. (When the new memory safety model is enabled, `sizeof(T)` expressions on a type parameter with the `unmanaged` constraint is allowed in safe code.) Only the 4-byte table is shown; the 1-byte, 2-byte, and 8-byte tables share its shape. Its larger cases fold the leftover with a `Vector256` or `Vector128` using two overlapping loads—one from the start, one from the end—so the wider-remainder handling for the omitted `Vector512`/`Vector256` paths lives right in the jump table. The two loads overlap whenever the length isn't an exact multiple of the width, so the tail is masked to the additive identity with `ConditionalSelect` before it's summed. That mask is only needed because addition is non-idempotent; an idempotent operation such as a search could fold the overlapping tail in directly. A buffer on hardware with no vectorization at all falls through to `SumScalar`, an ordinary scalar loop. ### Loop over the input and handle the remainder