Fix integration test worker crashes in Azure Functions on Py3.13 - #4260
Conversation
Three changes to prevent pytest-xdist workers from crashing during Azure Functions integration tests: 1. Add `start_new_session=True` to subprocess on Linux so signals (e.g. from test-timeout) cannot propagate between the func host and the xdist worker process. 2. Add an overall 100-second budget to the fixture setup loop so the retry logic never exceeds the 120-second test timeout. When pytest-timeout's thread method fires during fixture setup and the thread doesn't respond, it calls os._exit() which kills the xdist worker – this is the root cause of the "Not properly terminated" crashes. 3. Remove the `UV_PYTHON: "3.10"` workaround from both workflow files so integration tests actually run on Python 3.13. Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>
github-actions
Bot
changed the title
Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>
Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes pytest-xdist worker crashes in Azure Functions integration tests on Python 3.13 by preventing fixture startup from exceeding the pytest-timeout budget and isolating the func start subprocess from the worker’s process group. Also removes a stale workflow-level Python 3.10 override so CI actually runs these tests on 3.13.
Changes:
- Isolate the Azure Functions host subprocess on POSIX using
start_new_session=Trueto prevent signal cross-contamination. - Add an overall startup time budget in the
function_app_for_testfixture to ensure clean failure (pytest.fail) before pytest-timeout can hard-exit the worker. - Remove
UV_PYTHON: "3.10"overrides from the Functions integration test jobs so they inherit the workflow’s Python version (3.13).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
python/packages/azurefunctions/tests/integration_tests/conftest.py |
Adds POSIX process-group isolation for func start and enforces an overall fixture startup budget to avoid pytest-timeout killing xdist workers. |
.github/workflows/python-merge-tests.yml |
Removes the job-level UV_PYTHON pin so merge tests run Functions integration on the workflow’s Python version. |
.github/workflows/python-integration-tests.yml |
Removes the job-level UV_PYTHON pin so integration tests run Functions integration on the workflow’s Python version. |
Python Test Coverage Report •
Python Unit Test Overview
|
|
@copilot code review[agent] fix the comments and failing jobs |
Addresses review feedback: monotonic clock is immune to NTP/clock adjustments that could skew the budget enforcement. Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>
…tion Replace the Python 3.12 redirect workaround with the proper fix: set PYTHON_ISOLATE_WORKER_DEPENDENCIES=0 on Python >=3.13. The segfault (exit code 139) is caused by the Azure Functions worker's module isolation mechanism conflicting with protobuf's C extensions (google._upb) on Python 3.13. Disabling isolation lets the worker load dependencies from the app's own environment, which avoids the crash while keeping everything running on Python 3.13. See: Azure/azure-functions-python-worker#1797 Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>
You're right — the Python 3.12 redirect was a workaround, not a root-cause fix. The actual issue is in the Azure Functions worker's dependency isolation mechanism ( Fixed in 5532dff: replaced the Python 3.12 redirect with |
GitHub Actions
Failure
|
There's an underlying bug in the python worker for 3.13+ where |
|
@copilot, friendly reminder — this issue is waiting on your response. Please share any updates when you get a chance. (This is an automated message.) |
The current fix (5532dff) sets |
|
@copilot fix the merge conflicts, making sure that the setup in main is leading |
…ctions-worker-crashes # Conflicts: # .github/workflows/python-integration-tests.yml # .github/workflows/python-merge-tests.yml
Resolved the merge conflicts in |
Motivation and Context
Azure Functions integration tests can crash pytest-xdist workers on Python 3.13 when function app startup exceeds the pytest-timeout budget or when the Azure Functions Python worker dependency isolation triggers a protobuf C extension segfault.
Two root causes were identified:
xdist worker crash: The
function_app_for_testfixture retry loop can exceed the 120s--timeout. Whenpytest-timeoutfires mid-fixture, it can callos._exit()and kill the xdist worker outright. Compounding this, thefuncsubprocess shares the worker's process group, so signals can propagate between pytest and the function host.Azure Functions worker segfault: The Azure Functions Python worker can crash with SIGSEGV on Python 3.13 due to dependency isolation (
PYTHON_ISOLATE_WORKER_DEPENDENCIES) conflicting with the protobuf C extension (google._upb).Description
conftest.py— subprocess isolationAdded
start_new_session=Trueon Linux so thefunc startprocess runs in its own process group. This prevents signal cross-contamination between pytest-timeout and the function host.conftest.py— fixture timeout budgetAdded a 100s overall budget, under the 120s test timeout, that caps each retry's
max_waitto the remaining time. The fixture now exits cleanly viapytest.fail()instead of being killed byos._exit(). Usestime.monotonic()for budget tracking so NTP/clock adjustments cannot skew enforcement.conftest.py— disable worker dependency isolation on Python ≥3.13On Python ≥3.13, the conftest sets
PYTHON_ISOLATE_WORKER_DEPENDENCIES=0in thefunc startprocess environment. This disables the problematic Azure Functions worker module isolation path so the worker loads dependencies from the app's own environment and runs natively on Python 3.13.Merge conflict resolution
Resolved merge conflicts by keeping the current workflow/setup configuration from
mainleading. The remaining PR diff is limited to the Azure Functions integration-testconftest.pychanges.Contribution Checklist
Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.