'HelmCluster' object has no attribute '_cluster_info'

[amerenda]

What happened:

The code block:

from dask_kubernetes import HelmCluster

cluster = HelmCluster(release_name="dask",namespace="dask")
cluster.scale(1) 

Returns:

AttributeError: 'HelmCluster' object has no attribute '_cluster_info' 

from distributed/deploy/cluster.py
Full Traceback: https://paste.ee/p/0sn15

What you expected to happen:
The kubernetes cluster should spawn one worker.

Minimal Complete Verifiable Example:

from dask_kubernetes import HelmCluster

cluster = HelmCluster(release_name="dask",namespace="dask")
cluster.scale(1) 

Anything else we need to know?:
This is running in JupuyterLab on Google Kubernetes Engine
This command works if dask and distributed 2021.8.1 is installed
I do not believe this is an issue with helm, kubernetes, or GKE as 2021.8.1 works correctly.

Environment:

• helm dask worker version: 2021.9.1
• dask version: 2021.9.1
• distributed version: 2021.9.1
• dask_kubernetes version: 2021.3.1
• Python version: 3.9.6
• Operating System: Linux container
• Install method (conda, pip, source): pip

[jrbourbeau]

Thanks for raising an issue @amerenda! I believe this was fixed by #5305 (which was merged into main yesterday). Could try using the latest development version of distributed to see if the issue you're running into is resolved?

cc @jacobtomlinson for visibility

[jrbourbeau]

Closing as #5305 should have resolved this issue. @amerenda feel free to re-open if that's not the case

[graingert]

copying the traceback from the pastebin for future reference:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_78/3113707470.py in <module>
      1 from dask_kubernetes import HelmCluster
      2 
----> 3 cluster = HelmCluster(release_name="dask",namespace="dask")
      4 cluster.scale(0)  # specify number of workers explicitly

/opt/conda/lib/python3.9/site-packages/dask_kubernetes/helm.py in __init__(self, release_name, auth, namespace, port_forward_cluster_ip, loop, asynchronous, scheduler_name, worker_name, node_host, node_port, **kwargs)
     96         self.release_name = release_name
     97         self.namespace = namespace or namespace_default()
---> 98         self.name = self.release_name + "." + self.namespace
     99         check_dependency("helm")
    100         check_dependency("kubectl")

/opt/conda/lib/python3.9/site-packages/distributed/deploy/cluster.py in name(self, name)
     73     @name.setter
     74     def name(self, name):
---> 75         self._cluster_info["name"] = name
     76 
     77     async def _start(self):

AttributeError: 'HelmCluster' object has no attribute '_cluster_info'