Add `.get_client()` method to Cluster class

[mrocklin]

I'm sitting with @costrouc and he's typing cluster.get_client() based on his experience with Dask Gateway. Should we upstream this to dask.distributed.deploy.Cluster?

cc @jacobtomlinson @jcrist

[mrocklin]

I spoke with @jsignell and @jacobtomlinson about this. They were hesitant about this method because it seemed magical and created a second way to do something. My inclination was to go ahead and do it. It seems unharmful to me. I don't strongly care though. cc @jcrist in case he has experience with dask-gateway

[jsignell]

Specifically my worry is that get_client already exists on distributed.worker and returns an existing client that the worker is connected to. distributed.worker.get_client is what get_scheduler in dask itself relies on. That being said, I am not clear on what the impact of having multiple client object that point to the same cluster. Maybe that is fine?

I would be totally fine with this method if it is something like:

class Cluster:
    ....
    def get_client(self):
        from distributed.worker import get_client as get_current_client
    
        current_client = get_current_client()
        if current_client and current_client.cluster == self:
            return current_client
        else:
            return Client(self)

[jacobtomlinson]

@jsignell that sounds reasonable.

My main concern was that client = cluster.get_client() is just client = Client(cluster) and we should try to avoid having multiple equally straightforward ways to do the same thing. But if there is nuance around distributed.worker.get_client that definitely adds value to implementing this method.

[mrocklin]

Mostly the concern that I feel is when writing basic materials I have to ask users to import different things from different libraries.

from dask_jobqueue import SLURMCluster
cluster = SLURMCluster()

from dask.distributed import Client
client = Client(cluster)

I remember reading Spark materials and getting confused about the Conf and Session and SQLSession objects that I had to create. The ceremony to get a basic example running felt odd.

Adding Cluster.get_client would let us rework our common basic example to the following:

from dask_jobqueue import SLURMCluster
client = SLURMCluster().get_client()

Not a big deal. My guess is that it's less friction for early users. Not a big deal though.

[jsignell]

I agree that the mulitple imports isn't very elegant and would totally support an implementation like I outlined in #6732 (comment)