[Doc][Python] Run doctests on Python docstring examples

[asfimport]

We start to add more and more examples to the docstrings of Python methods (ARROW-15367), and so we could use the doctest functionality to ensure that those examples are actually correct (and keep being correct).

Pytest has integration for doctests (https://docs.pytest.org/en/6.2.x/doctest.html), and so you can do:

pytest python/pyarrow --doctest-modules

This currently fails for me because not having pyarrow.cuda, so we will need to find some ways to automatically skips those parts if not available. Normally, that should be possible with adding a conftest.py file in the main pyarrow directory, and then we can influence which files are found by defining pytest_runtest_setup or pytest_collection_modifyitems.

Reporter: Joris Van den Bossche / @jorisvandenbossche
Assignee: Alenka Frim / @AlenkaF

Related issues:

• [Doc][Python] The use of IPython directive or doctest code blocks in the python user guide (is related to)

PRs and other links:

• GitHub Pull Request #13199
• GitHub Pull Request #13204
• GitHub Pull Request #13216

_{Note: This issue was originally created as ARROW-16018. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Pytest itself doesn't directly support running the doctests in cython files. There is a pytest-cython plugin that allows this.

Its released version is however not very up to date, and only allows to run all doctests. With the master version (pip install git+https://github.com/lgpage/pytest-cython.git), you can run the doctests for a specific file with:

pytest --doctest-cython python/pyarrow/_csv.pyx

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Another option would be to run the doctests through sphinx with make doctest from the doc directory, which also allows running the doctests on embedded docstrings (and not only in the main .rst sphinx sources, xref ARROW-13159). See https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

However, for developing, I think it is important that you can run the doctests for a specific subset (the part of the code you are working on), and that doesn't seem to be easily possible with sphinx.

Another option might be to use the builtin doctest runner, eg python -m doctest -v python/pyarrow/_dataset.pyx, but I don't know if that will be feature-full enough (eg pytest has features to setup certain imports, or automatically skip files based on available modules, etc).

[asfimport]

Alessandro Molina / @amol-:
Issue resolved by pull request 13204
#13204