How to install GPU Discovery on bottlerocket?

[asluborski]

I am following the EKS quickstart and for gpu nodes, the guide suggests to just follow the helm install instructions for gpu discovery.

The instructions are here:

helm upgrade -i nvdp nvdp/nvidia-device-plugin \
    --version=0.15.0 \
    --namespace nvidia-device-plugin \
    --create-namespace \
    --set gfd.enabled=true

The thing is, if i am correct, bottlerocket comes with the k8 nvidia device plugin already but I want to add this as a subchart to the existing plugin so I can use gpu sharing. This command seems to already install nvidia-device-plugin from helm which I fear will conflict with the existing device plugin that is pre-installed. What is the correct way to enable gfd?

[asluborski]

That will also install node feature discovery. A follow-up, the command creates a new namespace but should I add this to the namespace where my gpu nodes are? I am only using that one namespace for my deployments.

[aradwan777]

I had a similar question, but I think it is a better option to install gpu operator with specific configurations keeping in mind bottlerocket has driver pre-installed. I have opened a new topic about gpu operator questions.

[vigh-m]

Hi,

I would like to clarify some things here. You can set up GPU discovery and should set it up in the same namespace are your containers.
We are working on enabling timeslicing to allow for GPU over-subscription and sharing and hope to enable that feature for our customers. As a result, for now, GPU discovery would not enable GPU Sharing. You should get other features like the auto generated labels for the nodes.

[ChristopherEdwardsNTC]

When will GPU over-subscription and sharing be enabled in bottle rocket? in a different discussion I posted some results in getting the GPU operator mostly running, but it appears without device plugin support any configuration that I have in my GPU Operator deployment won't do much but label nodes.