[SPARK-10274][MLlib] Add @since annotation to pyspark.mllib.fpm

[yu-iskw]

No description provided.

[SparkQA]

Test build #42191 has finished for PR 8665 at commit babb1ea.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class Instance(w: Double, a: Vector, b: Double)
  • class WeibullGenerator(
  • class IndexToString(JavaTransformer, HasInputCol, HasOutputCol):

[mengxr]

Does @since work on classes?

[yu-iskw]

@mengxr unfortunately, a class which inherits from object doesn't work. And as I commented on SPARK-10276, it doesn't work with @classmethod.

# Work
@since("1.6.0")
class Foo():

# Not Work
@since("1.6.0")
class Foo(object):

When I tried to add @since to a class which inherits from object, I got an error. It seems that we can't rewrite __doc__.

Traceback (most recent call last):
  File "tmp.py", line 24, in <module>
    class Foo(object):
  File "tmp.py", line 13, in deco
    f.__doc__ = ".. versionadded:: %s" % (version)
AttributeError: attribute '__doc__' of 'type' objects is not writable

[yu-iskw]

It seems that it works well with a class which inherits from object under python 3.3 or later. At least, it doesn't work under python 2.6.8, 2.7, 3.2.

This is a test script for @since. https://gist.github.com/yu-iskw/9f5a702b6657c1887b9b

[yu-iskw]

@davies do you have any good idea about the above issue?

[davies]

We should add .. addedversion:: 1.4 for classes

[yu-iskw]

@davies thank you for the comment. Just to be sure, we should add .. addedversion:: 1.4 to class methods, right?

[davies]

I think we should add that to the docstring of class, the class methods already had it, isn't it?

[yu-iskw]

Right. I understand @since depends on an order of decorators. @since should be called before @classmethod.

# Work
@classmethod
@since("1.4.0")
def foo(cls):

# Not Work
@since("1.4.0")
@classmethod
def bar(cls):

[yu-iskw]

@mengxr could you review it?

[SparkQA]

Test build #42230 has finished for PR 8665 at commit 0e093f1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[noel-smith]

@mengxr @davies Just to confirm, before @yu-iskw and I make the changes - we want to stick with the two-part version numbers (@since(1.4)) used in pyspark.sql instead of the full 3-part numbers (1.4.0) used in the other APIs - correct?

[davies]

@noel-smith I prefer two part version numbers (with a few exceptions), that's the way python standard libraries do, but we use three parts in Scala, @mengxr how do you think?

[mengxr]

We should use version numbers consistently between Python and Scala, and we do have a few APIs added in minor releases (DataFrames, some ML pipeline APIs). I would prefer using a.b.c instead of a.b.

[davies]

LGTM (using three parts version in MLlib, we could re-visit the SQL parts later)

[noel-smith]

Sounds good - thanks for confirming - I'll reinstate the thre-part version numbers in my PRs.

[mengxr]

Should be versionadded. Try make html under python/docs. Please look for compile warnings and check generated html doc.

[SparkQA]

Test build #42528 has finished for PR 8665 at commit 817b168.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[mengxr]

@yu-iskw Could you also add versionadded to FPGrowth and FreqItemset?

[SparkQA]

Test build #42582 has finished for PR 8665 at commit f1ce652.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[mengxr]

Merged into master. Thanks!