Design questions on porting SSE intrinsics that involve matrix operations and AVX intrinsics

[briancylui]

Process 1: SSE intrinsics tha involve matrix operations

Port code in src\Microsoft.ML.CpuMath\Sse.cs and src\Native\CpuMathNative\Sse.cpp to managed code in src\Microsoft.ML.CpuMath\CpuMathUtils.netcoreapp.cs and src\Microsoft.ML.CpuMath\SseIntrinsics

Progress

Implemented all required intrinsics, but there are some points that ask for further review. Those new intrinsics are:

1. MatMulA
2. MatMulPA
3. MatMulTranA
4. MatMulTranPA
5. MatMulRU
6. MatMulCU
7. MatMulDU
8. ZeroItemsU
9. ZeroMatrixItemsCore

Please let me know if I have missed any intrinsics needed.

Design questions

• Some naming change has been made:

1. pdLim becomes pDstEnd
2. pposLim becomes pposEnd (this changes some public function signatures)
3. ppossrc becomes pposSrc
4. srcValues becomes src
   Are these changes preferable?

• There were discussions about removing the dependency on the data structure AlignedArray. There is a function called Ptr that is closely associated with AlignedArray and frequently referenced. If we remove AlignedArray, we don't need Ptr anymore. Should we remove AlignedArray, and if so, how?

• Because some intrinsics are using AlignedArray and hence calling Ptr, some native methods are accepting pointers as input arguments, which are not desirable. This question issue has been easily solved when implementing the key intrinsics. For our current case, we could make Ptr internal instead of private and move the callsites of Ptr from CpuMathUtils to SseIntrinsics, which avoids fixing arrays twice and keeps only the SseIntrinsics functions unsafe, but not the CpuMathUtils ones. How does this suggestion sound?

• Does the order of fixed statements affect computational efficiency? For example, I saw both cases in Sse.cs:

fixed (float* pdst = &dst.Items[0])
fixed (float* pmat = &mat.Items[0])
fixed (float* psrc = &src.Items[0])

fixed (float* psrc = &src.Items[0])
fixed (float* pdst = &dst.Items[0])
fixed (float* pmat = &mat.Items[0])

• Should posMin be called posStart instead?

• In MatMulRU, pi, pii, and pindices are confusing - recommend using pidx, pIdxCurrent, pIdxEnd, etc., instead.

Process 2: AVX intrinsics

Progress:

Haven't implemented any AVX intrinsics, but have already created a new AvxIntrinsics class alongside SseIntrinsics under src\Microsoft.ML.CpuMath. The following AVX intrinsics are needed:

1. MatMulX
2. MatMulTranX
3. MatMulPX
4. MatMulTranPX
5. MatMulRX
6. MatMulCX
7. MatMulDX
8. ScaleX
9. AddScaleX
10. AddX

Please let me know if I have missed any intrinsics needed. Please note that the last 3 AVX intrinsics above are not even used/called unless the relevant questions below are addressed. It seems to me that the original AvxUtils class in src\Microsoft.ML.CpuMath\Avx.cs was written but not meant to be frequently called.

Design questions

In src\Microsoft.ML.CpuMath\Avx.cs:

• Should this line be Thunk.ScaleX(a, pd, count); instead of ScaleU for AVX?
  https://github.com/dotnet/machinelearning/blob/8087a20f3294671b522d265ccfa07543c5ff6fa5/src/Microsoft.ML.CpuMath/Avx.cs#L628

• Should this line be Thunk.AddScaleX(a, psrc, pdst, count); instead of AddScaleU for AVX?
  https://github.com/dotnet/machinelearning/blob/8087a20f3294671b522d265ccfa07543c5ff6fa5/src/Microsoft.ML.CpuMath/Avx.cs#L732

• Should this line be Thunk.AddX(ps, pd count); instead of AddU for AVX?
  https://github.com/dotnet/machinelearning/blob/8087a20f3294671b522d265ccfa07543c5ff6fa5/src/Microsoft.ML.CpuMath/Avx.cs#L782

• DotProductDense is missing, while DotProductSparse is using DotSU. There is no DotX or DotSX implemented.

• ZeroMatrixItems https://github.com/dotnet/machinelearning/blob/8087a20f3294671b522d265ccfa07543c5ff6fa5/src/Microsoft.ML.CpuMath/Avx.cs#L782

• There is no mapping from _mm256_zeroupper to C# code in Thunk. It is called after every AVX intrinsic to avoid perf hit (https://github.com/dotnet/machinelearning/blob/8087a20f3294671b522d265ccfa07543c5ff6fa5/src/Native/CpuMathNative/Sse.cpp#L18-L19). How should we proceed?

Process 3: Shared fields/methods between SSE and AVX

Design questions

• The following 3 fields / functions of the class CpuMathUtils are shared between SSE and AVX intrinsics implicitly:

1. field: pubic const int **CbAlign**
2. method: private static bool **Compat**
3. method: internal static bool **Ptr** (originally private but changed to internal so that SseIntrinsics and AvxIntrinsics methods can call it)
   Where should we place them?

TODOs

Find answers to the above questions, and understand the logic of matrix operations in order to implement reliable unit tests.