As mentionned in #122 and #139, default values for job output files in LSF and Slurm implementation are
"-e %s.err" % self.name
"-o %s.out" % self.name
If those configuration parameters have similar behavior as those from PBSPro, then this means every job submitted by dask-jobqueue will use this file name for writting its stdout and stderr when finished. So at one time, we will have only the last finished job stderr and stdout.
I'm not sure this is what we want. At least when trying to understand some bug, this can be problematic. I propose to use templating propose by both scheduler, e.g. for Slurm, something like (maybe with escaping char):
"-e %s-%j.err" % self.name
"-o %s-%j.out" % self.name
On the other hand, if we specify a value different for each job, we will get (as is happening currently with default PBSCluster implementation) the submission folder flooded with jobs stdout and stderr file.
As mentionned in #122 and #139, default values for job output files in LSF and Slurm implementation are
If those configuration parameters have similar behavior as those from PBSPro, then this means every job submitted by dask-jobqueue will use this file name for writting its stdout and stderr when finished. So at one time, we will have only the last finished job stderr and stdout.
I'm not sure this is what we want. At least when trying to understand some bug, this can be problematic. I propose to use templating propose by both scheduler, e.g. for Slurm, something like (maybe with escaping char):
On the other hand, if we specify a value different for each job, we will get (as is happening currently with default PBSCluster implementation) the submission folder flooded with jobs stdout and stderr file.