[BWARE] Tune compressed matmul fast paths and Spark execution decisions

[Baunsgaard]

Mixes two related performance changes: refined compressed multiply heuristics, and a Spark-vs-CP decision refresh on the Hop layer.

CLALib matmul changes:

• CLALibMMChain: for XtXv with few col groups and a wide-enough matrix, compute X' * X via leftMultByTransposeSelf and finish with a regular matrix multiply against v. Cheaper than chaining when the X' * X path can stay compressed
• CLALibTSMM: refactor leftMultByTransposeSelf into a package-private helper so MMChain can call it; widen the ColGroupUncompressed handling
• CLALibRightMultBy: stop forcing decompression for ASDC / ASDCZero inputs; they have working preAggregate paths that beat the dense fallback
• CLALibCompAgg: fix blklen rounding so the last partition is not short by k rows on parallel aggregates

Spark/CP exec-decision refresh (Hop, UnaryOp, BinaryOp):

• Hop: new helpers hasSparkOutput() and isScalarOrVectorBelowBlockSize() shared between unary and binary decision points
• UnaryOp.optFindExecType: replace the inline chain of negations with isDisallowedSparkOps(), allow Frame outputs, and pull unary ops into Spark when the input already has a Spark output; gated on the ALLOW_TRANSITIVE_SPARK_EXEC_TYPE flag so it shares a kill-switch with the binary path
• BinaryOp.optFindExecType: same kind of restructuring; allow matrix-or-frame outputs to be pulled into Spark when exactly one operand is a scalar or small vector

Instruction-side adjustments:

• VariableCPInstruction (CAST_AS_MATRIX from frame): use the parallel MatrixBlockFromFrame.convertToMatrixBlock(fin, k) path instead of the single-threaded DataConverter helper
• ParameterizedBuiltinCPInstruction (transformdecode): call the parallel decoder.decode(data, out, k) overload using InfrastructureAnalyzer.getLocalParallelism()
• DecoderComposite (parallel decode): parallelize over row blocks instead of over decoders, so the sub-decoders still run in order within each block (e.g. recode-on-output depends on the category indexes from the preceding dummycode decoder); fall back to the sequential path for k <= 1

Testing:

• New CompilerTestBase harness for compile-time exec-type assertions, plus SparkTransitiveExecTypeCompileTest and an end-to-end SparkTransitiveExecTypeTest (with DML) covering the unary/binary transitive Spark pull, its multi-consumer guard, the cumulative-op exclusion, and the flag-off kill-switch
• CLALibMMChainTest, CLALibRightMultBySDCTest, and DecoderCompositeTest for the compressed matmul fast paths, ASDC right-multiply, and parallel composite decode; shared helpers added to CompressedTestBase

[codecov]

Codecov Report

❌ Patch coverage is 93.65079% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.45%. Comparing base (e4f0987) to head (f4ead24).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/main/java/org/apache/sysds/hops/UnaryOp.java	72.72%	0 Missing and 3 partials ⚠️
src/main/java/org/apache/sysds/hops/BinaryOp.java	92.85%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2483      +/-   ##
============================================
- Coverage     71.47%   71.45%   -0.03%     
- Complexity    48883    48885       +2     
============================================
  Files          1573     1573              
  Lines        189238   189266      +28     
  Branches      37128    37137       +9     
============================================
- Hits         135261   135234      -27     
- Misses        43530    43572      +42     
- Partials      10447    10460      +13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.