Exclude stdout/stderr log sections when --report-log-exclude-logs-on-passed-tests is given

[ento]

I noticed that the size of the log file written by this plugin was 100MB+, and sections like "Captured stderr call" were included, even though I'd specified the --report-log-exclude-logs-on-passed-tests flag.

Would it make sense to make the flag also exclude captured outputs too, in addition to captured logs?

If I'm reading it right, junitxml's log_passing_tests setting covers both stdout/stderr outputs and logging outputs:
https://github.com/pytest-dev/pytest/blob/9.0.2/src/_pytest/junitxml.py#L159-L180

[RonnyPfannschmidt]

this constitutes as breaking change - perhaps a option to configure whats skipped

also i wonder why your output is so big - because i don't want this option in here if all it does is work around missconfigured logging thats using the stderr fallback

[ento]

this constitutes as breaking change - perhaps a option to configure whats skipped

Pytest's Junit XML reporter's junit_log_passing_tests option controls both stdout/stderr and logging outputs at the same time - which I assumed could be used as a data point that there's unlikely to be people who need to include stdout/stderr logs for passing tests but not logging outputs.

But I'm happy to do it as a separate option if that's better. Any suggestion as to the flag name? --report-log-exclude-stdio-on-passed-tests maybe, but that might be clunky/unclear.

also i wonder why your output is so big - because i don't want this option in here if all it does is work around missconfigured logging thats using the stderr fallback

The project uses Django + structlog (through its stdlib integration), but it seems it boils down to the use of logging.config.dictConfig, or, I guess when StreamHandler is added to the root logger after pytest has set up its log capture facility.

With a test file that sets up a StreamHandler using dictConfig:

logging.config.dictConfig({
    "version": 1,
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
        }
    },
    "loggers": {
        "": {"handlers": ["console"], "level": "INFO"},
    }
})

Logging output shows up in both "Captured stderr call" and "Captured log call" sections.

------------------ Captured stderr call -------------------
dictConfig failure stderr
dictConfig failure log
-------------------- Captured log call --------------------
CRITICAL root:test_dict_config.py:28 dictConfig failure log

minimally reproducible script

if [ ! -d env ]; then
    python -m venv env
    . env/bin/activate
    cat <<EOF >requirements.txt
iniconfig==2.3.0
packaging==25.0
pluggy==1.6.0
Pygments==2.19.2
pytest==9.0.2
pytest-reportlog==1.0.0
EOF
    pip install -r requirements.txt
else
    . env/bin/activate
fi

cat <<EOF >basic_config.py
import logging
logger = logging.getLogger()
logging.basicConfig()
EOF
cat <<EOF >test_basic_config.py
import sys
from basic_config import logger

def test_success():
    sys.stdout.write("basicConfig success stdout\n")
    sys.stderr.write("basicConfig success stderr\n")
    logger.critical("basicConfig success log")
    assert True

def test_failing():
    sys.stdout.write("basicConfig failure stdout\n")
    sys.stderr.write("basicConfig failure stderr\n")
    logger.critical("basicConfig failure log")
    print("logger handlers:", logger.handlers)
    assert False
EOF


pytest test_basic_config.py \
    --report-log=basic_config.log \
    --report-log-exclude-logs-on-passed-tests


cat <<EOF >dict_config.py
import logging
import logging.config

logger = logging.getLogger()
logging.config.dictConfig({
    "version": 1,
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
        }
    },
    "loggers": {
        "": {"handlers": ["console"], "level": "INFO"},
    }
})
EOF
cat <<EOF >test_dict_config.py
import sys
from dict_config import logger

def test_success():
    sys.stdout.write("dictConfig success stdout\n")
    sys.stderr.write("dictConfig success stderr\n")
    logger.critical("dictConfig success log")
    assert True

def test_failing():
    sys.stdout.write("dictConfig failure stdout\n")
    sys.stderr.write("dictConfig failure stderr\n")
    logger.critical("dictConfig failure log")
    print("logger handlers:", logger.handlers)
    assert False
EOF

pytest test_dict_config.py \
    --report-log=dict_config.log \
    --report-log-exclude-logs-on-passed-tests


cat <<EOF >py_config.py
import logging

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

logger.addHandler(handler)
EOF
cat <<EOF >test_py_config.py
import sys
from py_config import logger

def test_success():
    sys.stdout.write("python config success stdout\n")
    sys.stderr.write("python config success stderr\n")
    logger.critical("python config success log")
    assert True

def test_failing():
    sys.stdout.write("python config failure stdout\n")
    sys.stderr.write("python config failure stderr\n")
    logger.critical("python config failure log")
    print("logger handlers:", logger.handlers)
    assert False
EOF

pytest test_py_config.py \
    --report-log=py_config.log \
    --report-log-exclude-logs-on-passed-tests


grep 'success std' *.log

Should logging be set up in a specific way/timing when integrating with pytest?

Even if this duplication is dealt with in some way, strings written directly to stdout/stderr will still be included in pytest-reportlog's output, which will be unexpected when --report-log-exclude-logs-on-passed-tests is specified. Or at least the ability to control their inclusion in the report log will still be missing functionality.

[RonnyPfannschmidt]

As far as I'm concerned configuration of Studio logging output under test with capture constitutes missconfiguration

[ento]

Setting up logging using a code snippet from Python's official "Logging HOWTO" doc still results in the same duplication. (I've updated the reproducible example in my previous comment to do the logging setup in a separate file to simulate a real code base more closely.) pytest's documentation on logging doesn't mention any need for a special setup, so I'm not sure what configuration is expected from pytest's side.

Regardless, it still stands that pytest-reportlog doesn't expose a way to control whether to include stdout/stderr sections for passing tests like the junit xml reporter does. It's alright if project maintainers think that such an option is unnecessary, but I think it's something to consider separately from any logging misconfiguration.