Skip to content

Consolidate CLI testing helpers - #51

Merged
sirosen merged 2 commits into
pypa:mainfrom
sirosen:common-test-cli-runner
Aug 10, 2026
Merged

Consolidate CLI testing helpers#51
sirosen merged 2 commits into
pypa:mainfrom
sirosen:common-test-cli-runner

Conversation

@sirosen

@sirosen sirosen commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Move common helpers into conftest, so that behaviors can be easily shared.

This change also adds an assert_exit_code=0 kwarg to the test runner invocations, so that running CLI tests always asserts exit codes unless we explicitly opt-out. This prevents a common mistake when tests are added for a CLI tool, in which the exit code is not checked.


I write slightly different versions of this same testing tool in each Python CLI that I maintain.
Each one comes out slightly different, and in this case there ws good prior art to build upon. I like this iteration very well: it's short, simple, and effective.


📚 Documentation preview 📚: https://dependency-groups--51.org.readthedocs.build/en/51/

Move common helpers into conftest, so that behaviors can be easily
shared.

This change also adds an `assert_exit_code=0` kwarg to the test runner
invocations, so that running CLI tests always asserts exit codes unless
we explicitly opt-out. This prevents a common mistake when tests are
added for a CLI tool, in which the exit code is not checked.
@sirosen
sirosen requested a review from henryiii August 10, 2026 16:34
@henryiii

Copy link
Copy Markdown
Contributor

Nice! You can simplify that return value:

diff --git a/tests/conftest.py b/tests/conftest.py
index a138f13..f1f7fd1 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,6 +1,7 @@
 from __future__ import annotations

 import dataclasses
+import functools
 import typing as t

 import pytest
@@ -44,7 +45,4 @@ class CliRunner:

 @pytest.fixture
 def runner_factory(capsys: pytest.CaptureFixture[str]):
-    def bind(entry_point: CliEntryPoint, /) -> CliRunner:
-        return CliRunner(entry_point, capsys)
-
-    return bind
+    return functools.partial(CliRunner, capsys=capsys)

If you want, I can follow up and add types to the tests; I thought it would be easy (-> missing on runner_factory), so I asked Claude to add them locally, but I see from the diff it's pretty involved as they are missing in a few other places.

@sirosen

sirosen commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Oh, nice partial usage! I'll go ahead and apply.

Let's save more type coverage for the tests for later? It can get somewhat involved. The last time I worked on this for a package, I found that it's sometimes better to rephrase a fixture entirely rather than trying to describe what's there. (Stuff like protocols with __call__ vs adding a small class.)

- use `partial()`
- annotate the return type as a callable type
@sirosen
sirosen merged commit ddbcc5b into pypa:main Aug 10, 2026
5 checks passed
@sirosen
sirosen deleted the common-test-cli-runner branch August 10, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants