Don't shut down unresponsive workers on gather()

[crusaderky]

• Closes gather() should not remove unresponsive workers #7995
• Adds test for fallback from direct=True to direct=False

This PR plugs the last known case (to me at least) where a GIL blockage will cause computations to fail and workers to be shut down. Happy to hear if there are any more.

CC @fjetter @hendrikmakait

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       20 files  ±       0         20 suites  ±0   12h 8m 34s ⏱️ + 1h 51m 7s
  3 758 tests +       1    3 648 ✔️  -     1     106 💤 ±  0  4 ❌ +2 
36 352 runs  +1 050  34 597 ✔️ +992  1 750 💤 +55  5 ❌ +3 

For more details on these failures, see this check.

Results for commit b68af81. ± Comparison against base commit b3dde5c.

This pull request removes 1 and adds 2 tests. Note that renamed tests count towards both.

distributed.tests.test_scheduler ‑ test_gather_bad_worker_removed

distributed.tests.test_scheduler ‑ test_gather_bad_worker[False]
distributed.tests.test_scheduler ‑ test_gather_bad_worker[True]

[crusaderky]

All failures seem unreleated; ready for review and merge

[hendrikmakait]

Thanks, @crusaderky!