Skip to content

basic sge implementation#6

Closed
jhamman wants to merge 3 commits into
dask:masterfrom
jhamman:sge
Closed

basic sge implementation#6
jhamman wants to merge 3 commits into
dask:masterfrom
jhamman:sge

Conversation

@jhamman

@jhamman jhamman commented Mar 2, 2018

Copy link
Copy Markdown
Member

closes #3

I don't think this is ready to merge. Turns out the SGE cluster I used to have access to is no more. Perhaps we should get a basic CI implementation setup before pushing more content into the repo?

Comment thread dask_jobqueue/sge.py Outdated
self.n = 0
self._adaptive = None
self._submitcmd = 'qsub'
self._cancelcmd = 'qdel'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should maybe be class variables?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, agree for submit and cancel commands.

Comment thread dask_jobqueue/sge.py
project = project or os.environ.get('SGE_ACCOUNT')
if not project:
raise ValueError("Must specify a project like `project='UCLB1234' "
"or set SGE_ACCOUNT environment variable")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on #7 ?


def test_basic(loop):
with SGECluster(walltime='00:02:00', threads_per_worker=2, memory='7GB',
interface='ib0', loop=loop) as cluster:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interface keyword option here will likely fail on a CI system.

@jakirkham

Copy link
Copy Markdown
Member

Perhaps we should get a basic CI implementation setup before pushing more content into the repo?

So Travis CI is enabled, but does not have a configuration file yet. Would encourage just stealing stuff from dask-drmaa to get this working. There's a few files that you would want to grab outside of .travis.yml. They would be the docker-compose.yml, Dockerfiles, various assorted shell scripts, and some text files use for configuration.

@jhamman

jhamman commented Mar 27, 2018

Copy link
Copy Markdown
Member Author

@lesteve - afaik, you're the only current dask-jobqueue user/developer with access to an sge cluster. You may be interested in this initial implementation of the SGECluster. It will require some reworking following #10 but may provide a starting point.

@lesteve

lesteve commented Mar 28, 2018

Copy link
Copy Markdown
Member

Thanks I'll try to take a look.

@lesteve lesteve mentioned this pull request Apr 4, 2018
@jhamman jhamman closed this in #27 Apr 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Grid Engine

5 participants