Pytest launch doesn't finish

[ageleisha-ms]

Describe the bug
The test launch never finishes.
We have encountered a deadlock inside the finish_suites() function.

For some reason, in the _build_item_paths() method, it appears the condition elif leaf['type'] != LeafType.ROOT: was changed to if leaf["type"] != LeafType.ROOT:.

This leads to two additional elements being added to the tree that will never be executed.

Steps to Reproduce
1.Set up the following test structure:
python_file -> python_class -> python_tests(for example 4 tests)
2.Run tests using pytest-xdist and pytest-reportportal. For example:
pytest --dist load --tx N*popen --reportportal
Expected behavior
Tests finish executing.
The Pytest run completes successfully.
The ReportPortal launch is marked as completed.
PyTestService._tree_path should contain 4 elements (matching the number of actual tests).

Actual behavior
Tests appear to finish executing.
The Pytest run hangs and does not complete.
The ReportPortal launch remains in progress indefinitely.
PyTestService._tree_path contains 6 elements (2 more than the actual number of tests).

Package versions

reportportal-client==5.6.2
pytest-reportportal==5.5.0
pytest-xdist==3.3.1
pytest==7.4.4

Additional context
We use pytest-xdist, although this problem might not be directly related to it.

[ramir-dn]

We are seeing the same issue of pytest stuck inside finish_suites after upgrading from pytest-reportportal 5.4.7 to 5.5.1 and the mentioned code change above was added as part of BDD support to 5.5.0.
Any chance to promote this ticket?

[HardNorth]

@ageleisha-ms @ramir-dn I don't see such issue:

My dependencies:

Name: pytest-reportportal
Version: 5.5.1
Name: reportportal-client
Version: 5.6.5

I'm running our examples using Python 3.11:
https://github.com/reportportal/examples-python/blob/master/pytest/run-xdist.sh

And only change I did is put API key here: https://github.com/reportportal/examples-python/blob/master/pytest/pytest.ini#L2

So, what is your:

• Python version
• Server version
• Other properties

And do you have pytest-bdd installed? If so it has to be of version newer than 7.2.0:
https://github.com/reportportal/agent-python-pytest/blob/develop/requirements-dev-bdd.txt

[ramir-dn]

@HardNorth , we minimized what is needed to reproduce
A tests class suites with minimal 2 tests in it:

> cat test_root_pytest.py 
class TestSuiteExample:

    def test_subtraction(self):
        pass

    def test_multiplication(self):
        pass

For us it reproduced only with xdist, see the settings in pytest.ini below for the command args and it requires rp_hierarchy_code to be set to True, otherwise the suite level is not part of the node tree.

Here is the pytest.ini:

[pytest]
rp_hierarchy_code = True
addopts = --reportportal --dist load --tx 2*popen
rp_endpoint = ... 
rp_project = ...
rp_api_key = ...

The versions we have:

python==3.10.12
pytest==8.3.3
reportportal-client==5.6.5
pytest-reportportal==5.5.1
pytest-xdist==3.7.0

No bdd used or installed

Run using pytest test_root_pytest.py and it just stuck.

We debugged and printed the nodes tree in the finish_suites function when it is stuck in the loop with the sleep call, and printed the nodes and their exec status (mind those that are in the IN_PROGRESS state:

{
    "root": {"exec": "CREATED"},
    "test_root_pytest.py": {"exec": "IN_PROGRESS"},
    "TestSuiteExample": {"exec": "IN_PROGRESS"},
    "test_subtraction": {"exec": "FINISHED", "status": "PASSED"},
    "test_multiplication": {"exec": "CREATED"}
}

versus running without xdist, when it isn't stuck:

{
    "root": {"exec": "CREATED"},
    "test_root_pytest.py": {"exec": "FINISHED"},
    "TestSuiteExample": {"exec": "FINISHED"},
    "test_subtraction": {"exec": "FINISHED", "status": "PASSED"},
    "test_multiplication": {"exec": "FINISHED", "status": "PASSED"}
}

This doesn't reproduce if downgrading the pytest-reportportal==5.4.7.
I hope that this will give some direction

[HardNorth]

@ramir-dn No luck, even using Python 3.10.11 and rp_hierarchy_code which is, actually, officially not supported with x-dist.

[ramir-dn]

Ouch, I missed that xdist is stated as not supported with the rp_hierarchy_code @HardNorth.

Do you want to give it another shot anyway?
If so, I prepared a github repo with docker compose file that reproduces the hang.
The readme file states the steps, but it is basically to clone the repo, run docker compose up -d, setting the RP env vars, and then running the pytest command using docker compose exec pytest ... to run it inside the container.
You can find it here: https://github.com/ramir-dn/pytest-rp

(BTW, I am don't know if @ageleisha-ms, who originally reported this ticket was also using rp_hierarchy_code, it is just the way I am using it and managed to reproduce)

[HardNorth]

@ramir-dn Got a repro, looks like it's matter of slow connection. I'll take a look on the next week.

[ramir-dn]

Great, thanks for the update @HardNorth !
Just sharing that in our case, we run a local RP server close to where the client runs, and the reprouction is stable.

[HardNorth]

@ramir-dn @ageleisha-ms Try using this release: https://github.com/reportportal/agent-python-pytest/releases/tag/5.5.2

[ramir-dn]

Looks good @HardNorth , thanks!