Skip to content

Feature Request: Assign replicas to GPUs #11863

@kheyer

Description

@kheyer

Description

I have a service that runs on a single GPU. If I have multiple GPUs available, I would like to create one replica of this service for each GPU available.

Currently, I can only do this by explicitly creating the service multiple times in the docker-compose file and changing the device_ids section of the resources

  resources:
    reservations:
      devices:
      - driver: "nvidia"
        device_ids: ["0"]
        capabilities: [gpu]

Having to create 8 near-identical replicas of the same service in the config file is unwieldy.

I would like to specify the service once and set the replicas with

replicas: ${GPU_COUNT}

However this requires some way for each replica to know what GPU to use. My understanding is currently there is no way to do this, and there isn't a good way to go off a "replica index" for each container (see #9153).

Some method of mapping replicas of a service to different GPUs would be helpful.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions