Skip to content

CI for Schedulers #2

Description

@jakirkham

Linking to @jhamman's comment, one of the challenges here is setting up CI for Dask. There has been some already towards using SGE (particularly at dask-drmaa) for testing. Configuring other schedulers (e.g. SLURM, PBS, LSF, etc.) is possible, but can be a bit tricky. Docker helps with this, but it requires a fair bit of configuration to get right.

One resource that might be useful are these Docker images, which include SLURM and TORQUE (similar to PBS). That said, I'm not sure what license applies to these. Also they may not work out of the box (as was my experience with using their Grid Engine some time back). That said, they should at least provide useful insights on how to get working Docker images for these different cases. For simplicity, would propose building and hosting images for respective schedulers so that CI can just download them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions