From c861142026e3270061141b89989572b6545da281 Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Mon, 20 Jul 2026 14:48:29 -0700 Subject: [PATCH 1/2] Expand lane-crossing note in SIMD article with a worked example Add a worked example under 'Cross-platform vectorization with Vector128' showing why lane-crossing operations (unlike element-wise ones) don't widen for free on x86/x64. Includes ASCII diagrams contrasting element-wise vs. pairwise combination and the Vector128 bit-layout / lower-upper split, plus a verified LaneCrossingExample.cs snippet demonstrating a correct Vector128 horizontal reduction, the naive (wrong) Vector256 extension, and the fix using GetLower/GetUpper. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/standard/simd.md | 63 ++++++++++++++++++- .../simd/csharp/LaneCrossingExample.cs | 57 +++++++++++++++++ docs/standard/snippets/simd/csharp/Program.cs | 12 ++++ 3 files changed, 131 insertions(+), 1 deletion(-) create mode 100644 docs/standard/snippets/simd/csharp/LaneCrossingExample.cs diff --git a/docs/standard/simd.md b/docs/standard/simd.md index 43b23787c424c..8ead81ddfd013 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -79,13 +79,74 @@ Types that aren't supported, such as `char` and `bool`, can still be vectorized ## Cross-platform vectorization with Vector128 -`Vector128` is the common denominator across every platform that supports vectorization, so it's the best place to start. It holds a 128-bit vector: 16 bytes, 8 shorts, 4 ints/floats, or 2 longs/doubles. `Vector256` is twice as wide, and `Vector512` twice again. Not all hardware supports the larger widths, so the examples that follow use `Vector128` for portability. +`Vector128` is the common denominator across every platform that supports vectorization, so it's the best place to start. It holds a 128-bit vector: 16 bytes, 8 shorts, 4 ints/floats, or 2 longs/doubles. + +``` +------------------------------128-bits--------------------------- +| 64 | 64 | +----------------------------------------------------------------- +| 32 | 32 | 32 | 32 | +----------------------------------------------------------------- +| 16 | 16 | 16 | 16 | 16 | 16 | 16 | 16 | +----------------------------------------------------------------- +| 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | +----------------------------------------------------------------- +``` + +`Vector256` is twice as wide, and `Vector512` twice again. Not all hardware supports the larger widths, so the examples that follow use `Vector128` for portability. Each width has a generic type (`Vector128`) for the data and a non-generic static class () that holds most of the operations, including static factory methods like `Create` and `Load`. Operators such as `+`, `&`, and `<<` are the idiomatic way to express arithmetic and bit operations; prefer them over the named method equivalents to avoid operator-precedence bugs and improve readability. For algorithms that depend on byte order, branch on , which the JIT also folds to a constant. > [!NOTE] > On x86/x64, `Vector256` operations are generally treated as two independent 128-bit "lanes". For most element-wise operations this is transparent, but operations that cross lanes (such as shuffles or pairwise/horizontal operations) might behave differently or cost more than the `Vector128` equivalent. Confirm with benchmarks before assuming a wider vector is faster. +### Lane-crossing operations don't widen for free + +Element-wise operations don't care about width: `v1 + v2` is the same per-element result whether `v1` and `v2` are `Vector128` or `Vector256`—widening just processes one more lane's worth of data per instruction. `Add` on `v = [a, b, c, d]` and `w = [e, f, g, h]` always combines same-index elements: + +``` +v: [ a | b | c | d ] +w: [ e | f | g | h ] + + + + + +r: [a+e|b+f|c+g|d+h] +``` + +Lane-crossing operations don't scale up that simply, because which elements get combined depends on the vector's width. A pairwise reduction combines *adjacent* elements instead of same-index ones, so widening changes what ends up paired together: + +``` +v: [ a | b | c | d ] + \_____/ \_____/ +round 1: [ a+b | c+d | a+b | c+d ] + \_________________/ +round 2: [ S | S | S | S ] (S = a+b+c+d) +``` + +That's exactly what a horizontal reduction does: sum a vector's elements with two rounds of pairwise adds. On `Vector128` (4 elements), two calls to get you the full sum: + +:::code language="csharp" source="./snippets/simd/csharp/LaneCrossingExample.cs" id="SumVector128"::: + +Extend the same two-call pattern to `Vector256` (8 elements) and it looks right—it isn't. doesn't operate across the whole 256-bit vector; it repeats the pairwise pattern independently within each 128-bit lane. Two rounds gives you the lower lane's sum (elements 0-3) broadcast across the lower lane and the upper lane's sum (elements 4-7) broadcast across the upper lane, not the sum of all eight elements: + +:::code language="csharp" source="./snippets/simd/csharp/LaneCrossingExample.cs" id="SumVector256Naive"::: + +To get the correct total, bridge the lane boundary explicitly: read each lane's partial sum with `GetLower`/`GetUpper` and add them together—`GetLower` and `GetUpper` split a vector into its first and second halves: + +``` +------------------------------128-bits--------------------------- +| LOWER | UPPER | +----------------------------------------------------------------- +| 32 | 32 | 32 | 32 | +----------------------------------------------------------------- +| 16 | 16 | 16 | 16 | 16 | 16 | 16 | 16 | +----------------------------------------------------------------- +| 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | 8 | +----------------------------------------------------------------- +``` + +:::code language="csharp" source="./snippets/simd/csharp/LaneCrossingExample.cs" id="SumVector256"::: + +That extra step is the real cost of crossing lanes. A lane-crossing algorithm doesn't just widen for free the way an element-wise one does—measure before assuming the wider vector wins. + ### Common operations `Vector128` and its wider siblings expose a large API surface. You don't need to memorize it—know the categories and look up the details when you need them. Every operation has a software fallback for platforms that can't accelerate it. The following table covers essentially the whole surface. diff --git a/docs/standard/snippets/simd/csharp/LaneCrossingExample.cs b/docs/standard/snippets/simd/csharp/LaneCrossingExample.cs new file mode 100644 index 0000000000000..a0d8c9aae7d1c --- /dev/null +++ b/docs/standard/snippets/simd/csharp/LaneCrossingExample.cs @@ -0,0 +1,57 @@ +using System.Diagnostics; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; + +namespace SimdSnippets; + +public static class LaneCrossingExample +{ + // + // Sums all four elements with two rounds of pairwise horizontal adds. + // HorizontalAdd(v, v) on [a, b, c, d] gives [a+b, c+d, a+b, c+d]; a second round + // collapses that to the full sum in every element. + public static float SumVector128(Vector128 v) + { + Debug.Assert(Sse3.IsSupported); + + Vector128 step1 = Sse3.HorizontalAdd(v, v); + Vector128 step2 = Sse3.HorizontalAdd(step1, step1); + + return step2.ToScalar(); + } + // + + // + // The same two-round pattern on Vector256 looks like it should sum all eight + // elements, but Avx.HorizontalAdd repeats the pairwise pattern independently within + // each 128-bit lane. The result holds the lower lane's sum (elements 0-3) broadcast + // across the lower lane and the upper lane's sum (elements 4-7) broadcast across the + // upper lane -- ToScalar only returns the lower lane's partial sum, not the total. + public static float SumVector256Naive(Vector256 v) + { + Debug.Assert(Avx.IsSupported); + + Vector256 step1 = Avx.HorizontalAdd(v, v); + Vector256 step2 = Avx.HorizontalAdd(step1, step1); + + return step2.ToScalar(); + } + // + + // + // Getting the full sum needs an explicit step to cross the lane boundary: read each + // lane's partial sum out with GetLower/GetUpper and add them together. + public static float SumVector256(Vector256 v) + { + Debug.Assert(Avx.IsSupported); + + Vector256 step1 = Avx.HorizontalAdd(v, v); + Vector256 step2 = Avx.HorizontalAdd(step1, step1); + + Vector128 lower = step2.GetLower(); + Vector128 upper = step2.GetUpper(); + + return lower.ToScalar() + upper.ToScalar(); + } + // +} diff --git a/docs/standard/snippets/simd/csharp/Program.cs b/docs/standard/snippets/simd/csharp/Program.cs index be7aeb320342f..af3c6acc6d29b 100644 --- a/docs/standard/snippets/simd/csharp/Program.cs +++ b/docs/standard/snippets/simd/csharp/Program.cs @@ -1,6 +1,7 @@ using System.Numerics; using System.Linq; using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; using SimdSnippets; // Verification runner. Not part of the published snippets. @@ -49,3 +50,14 @@ float[] c = [1, 1, 1, 1]; Console.WriteLine($"MultiplyAdd = [{string.Join(", ", TensorPrimitivesExample.MultiplyAdd(a, b, c))}]"); Console.WriteLine($"CosineSimilarity(a, a) = {TensorPrimitivesExample.CosineSimilarity(a, a)}"); +Console.WriteLine(); + +if (Sse3.IsSupported && Avx.IsSupported) +{ + Vector128 v128 = Vector128.Create(1f, 2f, 3f, 4f); + Vector256 v256 = Vector256.Create(1f, 2f, 3f, 4f, 10f, 20f, 30f, 40f); + + Console.WriteLine($"SumVector128 = {LaneCrossingExample.SumVector128(v128)}"); + Console.WriteLine($"SumVector256Naive = {LaneCrossingExample.SumVector256Naive(v256)}"); + Console.WriteLine($"SumVector256 = {LaneCrossingExample.SumVector256(v256)}"); +} From e14db94c8444fdb59e36cb9ff89f2550d96d532e Mon Sep 17 00:00:00 2001 From: Tanner Gooding Date: Mon, 20 Jul 2026 15:05:42 -0700 Subject: [PATCH 2/2] Address PR feedback: split ISA checks, clarify x86/x64 scope Split the Sse3/Avx gating in the verification runner so the Vector128 demo also runs on SSE3-only hardware without AVX. Add an explicit 'On x86/x64' lead-in where the worked example switches from the platform-neutral pairwise concept to the x86/x64-specific intrinsics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/standard/simd.md | 2 +- docs/standard/snippets/simd/csharp/Program.cs | 9 ++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/standard/simd.md b/docs/standard/simd.md index 8ead81ddfd013..c28218df490df 100644 --- a/docs/standard/simd.md +++ b/docs/standard/simd.md @@ -121,7 +121,7 @@ round 1: [ a+b | c+d | a+b | c+d ] round 2: [ S | S | S | S ] (S = a+b+c+d) ``` -That's exactly what a horizontal reduction does: sum a vector's elements with two rounds of pairwise adds. On `Vector128` (4 elements), two calls to get you the full sum: +That's exactly what a horizontal reduction does: sum a vector's elements with two rounds of pairwise adds. On x86/x64, `Vector128` (4 elements) gets you there with two calls to : :::code language="csharp" source="./snippets/simd/csharp/LaneCrossingExample.cs" id="SumVector128"::: diff --git a/docs/standard/snippets/simd/csharp/Program.cs b/docs/standard/snippets/simd/csharp/Program.cs index af3c6acc6d29b..6ad4ca6ce057f 100644 --- a/docs/standard/snippets/simd/csharp/Program.cs +++ b/docs/standard/snippets/simd/csharp/Program.cs @@ -52,12 +52,15 @@ Console.WriteLine($"CosineSimilarity(a, a) = {TensorPrimitivesExample.CosineSimilarity(a, a)}"); Console.WriteLine(); -if (Sse3.IsSupported && Avx.IsSupported) +if (Sse3.IsSupported) { Vector128 v128 = Vector128.Create(1f, 2f, 3f, 4f); - Vector256 v256 = Vector256.Create(1f, 2f, 3f, 4f, 10f, 20f, 30f, 40f); - Console.WriteLine($"SumVector128 = {LaneCrossingExample.SumVector128(v128)}"); +} + +if (Avx.IsSupported) +{ + Vector256 v256 = Vector256.Create(1f, 2f, 3f, 4f, 10f, 20f, 30f, 40f); Console.WriteLine($"SumVector256Naive = {LaneCrossingExample.SumVector256Naive(v256)}"); Console.WriteLine($"SumVector256 = {LaneCrossingExample.SumVector256(v256)}"); }