According to Spark docs, reduce, reduceByKey, fold and foldByKey operations in RDDs should pass in a binary commutative and associative operation. This is an excerpt from Spark 2.1.0 code:
/**
* Reduces the elements of this RDD using the specified commutative and
* associative binary operator.
*/
def reduce(f: (T, T) => T): T = ...
So constraining the type to a Monoid is not enough as this only garantees associativity but not commutativity. These methods should be constraining in a cats.kernel.CommutativeMonoid in order to be safer.
Is also arguably whether they also need a Monoid at all as they do not make use of the empty operation and potentially a CommutativeSemigroup could suffice...
According to Spark docs,
reduce,reduceByKey,foldandfoldByKeyoperations in RDDs should pass in a binary commutative and associative operation. This is an excerpt from Spark 2.1.0 code:So constraining the type to a
Monoidis not enough as this only garantees associativity but not commutativity. These methods should be constraining in acats.kernel.CommutativeMonoidin order to be safer.Is also arguably whether they also need a
Monoidat all as they do not make use of theemptyoperation and potentially aCommutativeSemigroupcould suffice...