Use the grouped workers `worker_key` function in scale outside of adaptive

[guillaumeeb]

Seen thanks to dask/dask-kubernetes#98.

Currently, _job_id_from_worker_name function is used as worker_key in adaptive options: https://github.com/dask/dask-jobqueue/blob/master/dask_jobqueue/core.py#L145.

However, this is not in place when manually scaling down or up. This could be a nice thing to avoid stopping too many workers when scaling down.

This may have to be changed upstream in distributed, somewhere here: https://github.com/dask/distributed/blob/master/distributed/deploy/cluster.py#L103. We could use a self.worker_key method if defined as another kwarg.

@mrocklin any opinion on that?

[mrocklin]

Yes. That makes sense to me.

…

On Mon, Sep 17, 2018 at 9:19 AM, Guillaume Eynard-Bontemps < ***@***.***> wrote: Seen thanks to dask/dask-kubernetes#98 <dask/dask-kubernetes#98>. Currently, _job_id_from_worker_name function is used as worker_key in adaptive options: https://github.com/dask/dask-jobqueue/blob/master/dask_ jobqueue/core.py#L145. However, this is not inplace when manually scaling down or up. This could be a nice thing to stopping to much worker when scaling down. This may have to be change upstream in distributed, somewhere here: https://github.com/dask/distributed/blob/master/ distributed/deploy/cluster.py#L103. We could use a self.worker_key method if defined as another kwarg. @mrocklin <https://github.com/mrocklin> any opinion on that? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#152>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszH-3VVy1QfgZMAIRoh0J-UoVDZPGks5ub6FKgaJpZM4Wr5vV> .