hasNumberOfDistinctValues and hasHistogramValues require arguments that should be optional?

[thvasilo]

Describe the bug
The docs and Scala code for hasNumberOfDistinctValues and hasHistogramValues indicate that providing the binningUdf, maxBins parameters should be optional, but from the function definitions they seem to be required.

To Reproduce
Steps to reproduce the behavior:

1. Try to define a check with check.hasNumberOfDistinctValues('column_name', lambda x: x == 6)
2. See error: TypeError: hasNumberOfDistinctValues() missing 2 required positional arguments: 'binningUdf' and 'maxBins'

Expected behavior
I'd like to be able to call the hasNumberOfDistinctValues and hasHistogramValues without specifying a binning function and maxBins.

[brunoRenzo6]

While these params are still defined as required... When it comes to binningUdf you could simply set it as None.

check = Check(spark, CheckLevel.Warning, "test hasHistogramValues")
result = (VerificationSuite(spark).onData(df)
          .addCheck(check
                    .hasHistogramValues("c_1",
                                             lambda x: x.apply("66").absolute() > 4500000, None, 2)
                    .hasHistogramValues("c_2",
                                             lambda x: x.apply("22").ratio() > 0.5, None, 2))
          
         ).run()

[thvasilo]

Thanks @brunoRenzo6, adding this to the method's docs could help.

[github-actions]

This is a confirmed bug. The hasNumberOfDistinctValues and hasHistogramValues methods in checks.py define binningUdf and maxBins as required positional arguments, but they should be optional with default values matching the Scala defaults.

The fix would be to change the signatures to use default values:

def hasNumberOfDistinctValues(self, column, assertion, binningUdf=None, maxBins=None, hint=None):
    assertion_func = ScalaFunction1(self._spark_session.sparkContext._gateway, assertion)
    hint = self._jvm.scala.Option.apply(hint)
    binningUdf = binningUdf if binningUdf else getattr(self._Check, "hasNumberOfDistinctValues$default$3")()
    maxBins = maxBins if maxBins else getattr(self._Check, "hasNumberOfDistinctValues$default$4")()
    self._Check = self._Check.hasNumberOfDistinctValues(column, assertion_func, binningUdf, maxBins, hint)
    return self

def hasHistogramValues(self, column, assertion, binningUdf=None, maxBins=None, hint=None):
    assertion_func = ScalaFunction1(self._spark_session.sparkContext._gateway, assertion)
    hint = self._jvm.scala.Option.apply(hint)
    binningUdf = binningUdf if binningUdf else getattr(self._Check, "hasHistogramValues$default$3")()
    maxBins = maxBins if maxBins else getattr(self._Check, "hasHistogramValues$default$4")()
    self._Check = self._Check.hasHistogramValues(column, assertion_func, binningUdf, maxBins, hint)
    return self

In the meantime, as @brunoRenzo6 noted, you can pass None for binningUdf and a value like 2 for maxBins as a workaround:

check.hasNumberOfDistinctValues('column_name', lambda x: x == 6, None, 2)

Would welcome a PR for this fix!

---

Generated by AI (model: us.anthropic.claude-opus-4-6-v1, prompt: cf3780e5) — may not be fully accurate. Reply if this doesn't help.