Type annotation for fixtures

[sodul]

As reported by others, the fact that fixture can exist magically is confusing for many users unfamiliar with pytest and it can be gruesome to track in larger projects with many fixtures.

Since type annotation is working well in python 3 I was thinking that one explicit way would be to have a new Fixture type to help annotate the fixture arguments.

from typing import Union
import pytest

Fixture = Union

@pytest.fixture
def bob() -> str:
    return '42'

@pytest.fixture
def alice() -> int:
    return 42

def test_foo(bob: Fixture[str], alice: Fixture[int]):
    assert bob != alice

In this example I 'abuse' Union so that existing tools are taking the hinting without any issue. For the person coming in and reading the code, especially in larger projects, the fact that the arguments are fixtures becomes very explicit and in the spirit of the Zen of Python:

Explicit is better than implicit.

Unfortunately mypy and other type checking tools don't seem to 'alias' Union since it is a special case.

This on the other hand works but I would prefer the annotation of the first example:

from typing import Union
import pytest

@pytest.fixture
def bob() -> str:
    return '42'

@pytest.fixture
def alice() -> int:
    return 42

FixtureStr = Union[str]
FixtureInt = Union[int]

def test_bar(bob: FixtureStr, alice: FixtureInt):
    assert bob != alice

IDEs such as PyCharm then now able to hint on bob being a string and as a user I can tell that it is a fixture argument.

[blueyed]

Maybe a mypy plugin could help here somehow?

(What already works of course is using the returned type of the fixture directly - but it does not make it obvious that it is a fixture then, of course.)

/cc @bluetech

[The-Compiler]

I haven't actually tested it, but you should be able to do something like:

T = typing.TypeVar('T')

class Fixture(typing.Generic[T]):
    pass

def test_bar(bob: Fixture[str]):
    ...

[davidhalter]

Why not just write it as def test_bar(bob: str): ... at that point? IMO the Fixture just makes it more confusing.

I just finally have to write pytest support in Jedi. I'm using both tools daily so I should really just do myself a favor :).

[sodul]

def test_bar(bob: str): is what we are doing today but while pytest is common, its use of magical fixtures is confusing for the majority of python developers that do not know about them.

I was refactoring some code to comply with PEP-8 recently so that pylint and other tools can better expose potential bugs but I had to double and triple check which arguments were safe to rename and which were not. The Fixture annotation would also allow us to better target which arguments need to be updated when renaming a fixture. For example to rename the bob fixture we could grep for bob: Fixture and have a much higher confidence that all the code that needed updating got updated.

Florian's solution passes mypy but then it does not allow PyCharm to detect that bob is a string and we have no type hinting, similar result as the Fixture = Union approach.

[Zac-HD]

This would have to be a mypy plugin... and even then, it's going to be full of edge cases. IMO if you need that level of visibility fixtures are just not suitable.

[asottile]

I second @davidhalter's message -- I don't think a special syntax improves the situation (and would require a mypy plugin to undo the outer Fixture[...] indirection to "unwrap" the inner type)

[davidhalter]

Support for fixtures just landed in Jedi's master branch so it is only a matter of time until Jedi is released and you all get to have IDE support for fixtures. Annotations are not necessary (but might help in very complicated cases).

[nicoddemus]

Thanks everyone for chiming in here, I think it was an useful discussion, but there's nothing really to be done on pytest AFAIU.

[adam-grant-hendry]

I know this is closed, but this is important for posterity.

pytest itself has type annotations for fixtures that you can reuse (see pytest-dev/pytest/src/_pytest/fixtures.py). You can import these from _pytest.fixtures even though the leading underscore indicates the package is intended to be private.

Namely:

# The value of the fixture -- return/yield of the fixture function (type variable).
_FixtureValue = TypeVar("_FixtureValue")

# The type of the fixture function (type variable).
_FixtureFunction = TypeVar("_FixtureFunction", bound=Callable[..., object])

# The type of a fixture function (type alias generic in fixture value).
_FixtureFunc = Union[
    Callable[..., _FixtureValue], Callable[..., Generator[_FixtureValue, None, None]]
]

...

class FixtureRequest:
    """A request for a fixture from a test or fixture function.

    A request object gives access to the requesting test context and has
    an optional ``param`` attribute in case the fixture is parametrized
    indirectly.
    """
    ...

...

@final
@attr.s(frozen=True)
class FixtureFunctionMarker:
    ...

The signature of fixture itself is:

def fixture(
    fixture_function: Optional[_FixtureFunction] = None,
    ...,  # Other irrelevant args excluded by me for brevity
) -> Union[FixtureFunctionMarker, _FixtureFunction]:
   fixture_marker = FixtureFunctionMarker(
        scope=scope,
        params=params,
        autouse=autouse,
        ids=ids,
        name=name,
    )

    # Direct decoration.
    if fixture_function:
        return fixture_marker(fixture_function)

    return fixture_marker

Since pytest defines these directly, you don't have to guess at what they should be. For instance, use type SubRequest for request objects declared as function arguments. Note also that when a test function uses a fixture as an argument, the fixture instance returned from the fixture function will be injected, so the type should be the type of the fixture instance.

If that's what they use, I would take it from the horse's mouth.

An example conftest.py for sphinx testing is:

# region docstring
"""Configuration file for ``pytest`` tests

This module contains integration test fixtures. This code was adapted from Paul Everitt's
GitHub repo (see [1]_).

References:
.. _[1]: https://github.com/pauleveritt/customizing_sphinx/blob/master/tests/integration/conftest.py
"""
# endregion

from typing import Any, Callable, Generator, List, Sequence, Union

import pytest
from _pytest.fixtures import SubRequest
from bs4 import BeautifulSoup
from sphinx.testing.path import path
from sphinx.testing.util import SphinxTestApp

pytest_plugins: Union[str, Sequence[str]] = [
    "sphinx.testing.fixtures",
]
"""A ``pytest`` global variable that registers plugins for use in testing."""

# Exclude 'roots' dirs for pytest test collector
collect_ignore: List[str] = ["roots", "__init__.py"]
"""A ``pytest`` global variable that excludes directories or modules from being collected by the ``pytest`` runner"""


@pytest.fixture()
def rootdir() -> Generator[str, None, None]:
    # region docstring
    """Root directory for sphinx tests.

    Yields:
        Generator[str, None, None]: Generator that yields test root directories
    """
    # endregion
    yield path(".\\docs\\source").abspath()


@pytest.fixture()
def content(
    make_app: Callable[..., Any], rootdir: str
) -> Generator[SphinxTestApp, None, None]:
    # region docstring
    """Content of generated html pages.

    Args:
        make_app (Callable[..., Any]): Function that creates a sphinx app for testing purposes
        rootdir (str): Test root directory

    Yields:
        Generator[SphinxTestApp, None, None]: Generator that yields sphinx apps for testing purposes
    """
    # endregion
    app = make_app("html")
    app.build()
    yield app


@pytest.fixture()
def page(
    content: SphinxTestApp, request: SubRequest
) -> Generator[BeautifulSoup, None, None]:
    # region docstring
    """Generated HTML page of docs

    Args:
        content (SphinxTestApp): A sphinx app for testing purposes
        request (SubRequest): A sub request for handling getting a fixture from a test function/fixture

    Yields:
        Generator[BeautifulSoup, None, None]: Generator of beautiful soup web scraper instances using the html5lib parser
    """
    # endregion
    page_name = request.param
    page_content = (content.outdir / page_name).read_text()

    yield BeautifulSoup(page_content, "html5lib")