[Feature]: Make Device Config part of values file

[OguzPastirmaci]

Suggestion Description

The current design requires a two step deployment and removal design with tools like Terraform. It would be great if the device config is included in Helm values file instead.

Operating System

No response

GPU

No response

ROCm Component

No response

[yansun1996]

Hi @OguzPastirmaci thanks for the suggestion, yep this is a good way to reduce one more step to configure the cluster, will drop an update once the helm support for that is added. We're discussing to push on default CustomResource DeviceConfig within the helm chart, but in that way it means we also need to handle the lifecycle of the default DeviceConfig during the helm install/upgrade/uninstall.

[xiongzubiao]

This feature will be useful! I noticed that there is PR from @yansun1996 already. Can't wait to try it out. However, for a cluster with both PF and AF GPUs, it can happen that some nodes are labeled with feature.node.kubernetes.io/amd-gpu, and some other are labeled with feature.node.kubernetes.io/amd-vgpu. In this case, would two DeviceConfig objects are needed? One DeviceConfig can handle only one label selector (adding two labels in the selector would match no node).

[yansun1996]

yep @xiongzubiao @OguzPastirmaci the one-step Operator installation + configuration has been committed and it will be available in v1.3.0 release, which will be published soon.

@xiongzubiao amd-gpu shows the PF device and amd-vgpu shows the VF device, the amd-vgpu is mostly for detecting the VF device being used in guest VM environment.

We suggest each cluster manage one type of workload if anyway you put multiple types of GPU for the workload in the same cluster then based on currently implementation you need 2 DeviceConfigs. If you want to ask for a feature by using single label to manage both PF + VF devices, you can submit a new feature request and let's discuss it in the new ticket.

[yansun1996]

Hi @OguzPastirmaci , we've enabled this feature in the latest release. Starting from v1.3.0, the helm install command by default will create a default DeviceConfig, whose values are also configurable via helm charts itself.

helm show values rocm/gpu-operator-charts --version=v1.3.0