TaskPool currently handles both generic task-pool behavior and Dask-specific execution details. That makes the class harder to follow, since non-Dask flows and Dask scheduler/worker management are interleaved in the same implementation.
This refactor would introduce a DaskTaskPool(TaskPool) subclass and move Dask-only behavior into that subclass. The base TaskPool would remain focused on common task-pool responsibilities such as queueing, submission flow, active/finished task tracking, and cleanup, while DaskTaskPool would own Dask-specific setup, submission, event handling, result gathering, and shutdown.
Benefits:
- gives
TaskPool a clearer single responsibility
- isolates Dask scheduler/worker logic from generic task-pool behavior
- reduces branching and Dask conditionals in the base class
- makes both standard and Dask-backed execution paths easier to maintain and test
Scope:
- add a new
DaskTaskPool subclass
- move Dask-specific state and methods from
TaskPool into the subclass
- preserve current task-pool behavior and public APIs
- keep task-pool creation/submission flows compatible with existing callers
TaskPoolcurrently handles both generic task-pool behavior and Dask-specific execution details. That makes the class harder to follow, since non-Dask flows and Dask scheduler/worker management are interleaved in the same implementation.This refactor would introduce a
DaskTaskPool(TaskPool)subclass and move Dask-only behavior into that subclass. The baseTaskPoolwould remain focused on common task-pool responsibilities such as queueing, submission flow, active/finished task tracking, and cleanup, whileDaskTaskPoolwould own Dask-specific setup, submission, event handling, result gathering, and shutdown.Benefits:
TaskPoola clearer single responsibilityScope:
DaskTaskPoolsubclassTaskPoolinto the subclass