multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False)

[brianquinlan]

BPO	39205
Nosy	@brianquinlan
PRs	bpo-39205: Tests that highlight a hang on ProcessPoolExecutor shutdown #18221

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/brianquinlan'
closed_at = None
created_at = <Date 2020-01-03.21:26:24.394>
labels = ['type-bug', '3.9']
title = 'multiprocessing: Hang when interpreter exits after ProcessPoolExecutor.shutdown(wait=False)'
updated_at = <Date 2020-01-28.01:40:13.223>
user = 'https://github.com/brianquinlan'

bugs.python.org fields:

activity = <Date 2020-01-28.01:40:13.223>
actor = 'vstinner'
assignee = 'bquinlan'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-01-03.21:26:24.394>
creator = 'bquinlan'
dependencies = []
files = []
hgrepos = []
issue_num = 39205
keywords = ['patch']
message_count = 2.0
messages = ['359257', '360828']
nosy_count = 1.0
nosy_names = ['bquinlan']
pr_nums = ['18221']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39205'
versions = ['Python 3.9']

Linked PRs

• gh-83386: Enable test_hang_gh83386 for ProcessPoolExecutor #152976
• [3.15] gh-83386: Enable test_hang_gh83386 for ProcessPoolExecutor (GH-152976) #152982
• [3.14] gh-83386: Enable test_hang_gh83386 for ProcessPoolExecutor (GH-152976) #152983

[brianquinlan]

from concurrent.futures import ProcessPoolExecutor
import time

t = ProcessPoolExecutor(max_workers=3)
t.map(time.sleep, [1,2,3])
t.shutdown(wait=False)

Results in this exception and then a hang (i.e. Python doesn't terminate):

Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/usr/local/google/home/bquinlan/cpython/Lib/threading.py", line 944, in _bootstrap_inner
    self.run()
  File "/usr/local/google/home/bquinlan/cpython/Lib/threading.py", line 882, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/google/home/bquinlan/cpython/Lib/concurrent/futures/process.py", line 352, in _queue_management_worker
    _add_call_item_to_queue(pending_work_items,
  File "/usr/local/google/home/bquinlan/cpython/Lib/concurrent/futures/process.py", line 280, in _add_call_item_to_queue
    call_queue.put(_CallItem(work_id,
  File "/usr/local/google/home/bquinlan/cpython/Lib/multiprocessing/queues.py", line 82, in put
    raise ValueError(f"Queue {self!r} is closed")
ValueError: Queue <concurrent.futures.process._SafeQueue object at 0x7f371c4ae7f0> is closed

[brianquinlan]

New changeset 884eb89 by Brian Quinlan in branch 'master':
bpo-39205: Tests that highlight a hang on ProcessPoolExecutor shutdown (bpo-18221)
884eb89