PBS resources spec and Dask resources keywords link

[guillaumeeb]

I'm also wondering if we should have a tighter link between PBS resources spec and the dask processes/worker/memory options.

Basically, given select=x:ncpus=y:mem=z resource_spec from PBS:

• x should always be one, as we launch one dask-worker per node,
• y should be equal to dask nthreads * nprocesses, if I'm not mistaken,
• z should be equal to dask nprocesses * memory-limit, again correct me if I'm wrong.

Should we implement this behaviour in the code? This would mean that we don't need the resource_spec keyword arg anymore, but we can just build it using dask worker options.

[jhamman]

I agree with the constraints you've laid out here. My only concern is that we could end up being overly prescriptive. Not all PBS systems are configured the same and it is difficult to know where the line of consistency will be.

[guillaumeeb]

I understand and share your concern. Maybe it is over engineering and not really pythonic to do this.
I was thinking about this also seeing the use of processes in the SLURMCluster class (which is maybe wrong, as I wrote about it here: #10 ).

Anyway, if we agree that resource_spec is always in the form of select=x:ncpus=y:mem=z (which for some old PBS/Torque version, it may not be the case?), maybe we should just declare two parameters:

• ncpus
• job_memory
  and build the resource_spec from here?
  I believe that we can agree that we will only be selecting one chunk or one node, for a dask-worker.

[guillaumeeb]

A kind of implementation of this has ben done in #10, only if resource_spec is not provided.