Skip to content

Rename get_default_shuffle_algorithm to get_default_shuffle_method #7902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hendrikmakait merged 6 commits into from
Jun 13, 2023
Merged

Rename get_default_shuffle_algorithm to get_default_shuffle_method #7902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d028e04
Rename to get_default_shuffle_method
hendrikmakait Jun 12, 2023
47c4fcd
Temporarily point to dask/dask PR [skip-caching]
jrbourbeau Jun 12, 2023
bd310aa
gpuCI
jrbourbeau Jun 12, 2023
a260ce7
pre-commit [skip-caching]
jrbourbeau Jun 12, 2023
cec93c4
gpuCI again [skip-caching]
jrbourbeau Jun 12, 2023
43f746a
Revert "Temporarily point to dask/dask PR [skip-caching]"
hendrikmakait Jun 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,5 +63,6 @@ repos:
- pytest
- tornado
- pyarrow
- git+https://github.com/dask/dask
# - git+https://github.com/dask/dask
- git+https://github.com/hendrikmakait/dask.git@default-shuffle-ignores-fuse
- git+https://github.com/dask/zict
7 changes: 4 additions & 3 deletions continuous_integration/gpuci/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,13 @@ gpuci_logger "Activate conda env"
. /opt/conda/etc/profile.d/conda.sh
conda activate dask

gpuci_logger "Install dask"
python -m pip install git+https://github.com/dask/dask

gpuci_logger "Install distributed"
python -m pip install -e .

gpuci_logger "Install dask"
# python -m pip install git+https://github.com/dask/dask
python -m pip install git+https://github.com/hendrikmakait/dask.git@default-shuffle-ignores-fuse

gpuci_logger "Check Python versions"
python --version

Expand Down
19 changes: 7 additions & 12 deletions distributed/tests/test_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,7 @@
import dask.bag as db
from dask import delayed
from dask.optimization import SubgraphCallable
from dask.utils import (
get_default_shuffle_algorithm,
parse_timedelta,
stringify,
tmpfile,
)
from dask.utils import get_default_shuffle_method, parse_timedelta, stringify, tmpfile

from distributed import (
CancelledError,
Expand Down Expand Up @@ -3229,25 +3224,25 @@ def test_default_get(loop_in_thread):
# These may change in the future but the selection below should not
distributed_default = "p2p" if has_pyarrow else "tasks"
local_default = "disk"
assert get_default_shuffle_algorithm() == local_default
assert get_default_shuffle_method() == local_default
with Client(s["address"], set_as_default=True, loop=loop) as c:
assert dask.base.get_scheduler() == c.get
assert get_default_shuffle_algorithm() == distributed_default
assert get_default_shuffle_method() == distributed_default

assert dask.base.get_scheduler() == pre_get
assert get_default_shuffle_algorithm() == local_default
assert get_default_shuffle_method() == local_default

c = Client(s["address"], set_as_default=False, loop=loop)
assert dask.base.get_scheduler() == pre_get
assert get_default_shuffle_algorithm() == local_default
assert get_default_shuffle_method() == local_default
c.close()

c = Client(s["address"], set_as_default=True, loop=loop)
assert get_default_shuffle_algorithm() == distributed_default
assert get_default_shuffle_method() == distributed_default
assert dask.base.get_scheduler() == c.get
c.close()
assert dask.base.get_scheduler() == pre_get
assert get_default_shuffle_algorithm() == local_default
assert get_default_shuffle_method() == local_default

with Client(s["address"], loop=loop) as c:
assert dask.base.get_scheduler() == c.get
Expand Down