dra-driver-nvidia-gpu

packaged by Chainguard

Chainguard Container for dra-driver-nvidia-gpu

Kubernetes Dynamic Resource Allocation (DRA) driver for NVIDIA GPUs and multi-node NVLink ComputeDomains (IMEX)

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/dra-driver-nvidia-gpu:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

Chainguard's dra-driver-nvidia-gpu image is a drop-in replacement for the upstream image published at registry.k8s.io/dra-driver-nvidia/dra-driver-nvidia-gpu. Like upstream, a single image ships every component of the driver, and the Helm chart selects which one each pod runs:

• gpu-kubelet-plugin and compute-domain-kubelet-plugin, the per-node kubelet plugins
• compute-domain-controller and compute-domain-daemon, which manage ComputeDomains and their IMEX daemons
• webhook, the optional validating admission webhook
• kubelet-plugin-prestart.sh, the init container that checks for the host GPU driver
• nvidia-cdi-hook and the templates under /templates

The image keeps the upstream entrypoint (/busybox/sh), working directory, NVIDIA_* environment variables, and runs as root, because the kubelet plugins need access to the host driver and kubelet plugin directories. It is built on Wolfi and adds the mount and umount utilities the kubelet plugins call at runtime.

A FIPS variant is available as dra-driver-nvidia-gpu-fips.

Prerequisites

• A Kubernetes cluster at version 1.34.2 or later (1.32 or later for ComputeDomains only, with the DynamicResourceAllocation feature gate and resource.k8s.io API groups enabled before 1.34).
• For GPU allocation: NVIDIA GPU nodes with the NVIDIA driver installed on the host or provided by the NVIDIA GPU Operator. For ComputeDomains: Multi-Node NVLink nodes with NVIDIA driver 570.158.01 or later, nvidia.com/gpu.clique labels from GPU Feature Discovery, and the host nvidia-imex.service disabled and masked, because the driver runs its own IMEX daemons.
• Helm 3.8 or later, to install the chart from the upstream OCI registry.

See the upstream prerequisites for cluster requirements.

Getting Started

Create a values file that points the chart at the Chainguard image. This example enables the ComputeDomain resources only; see Configuration for GPU allocation:

cat > values.yaml <<EOF
image:
  repository: cgr.dev/ORGANIZATION/dra-driver-nvidia-gpu
  tag: latest
resources:
  gpus:
    enabled: false
EOF

Install the upstream chart into its own namespace:

helm install dra-driver-nvidia-gpu \
  oci://registry.k8s.io/dra-driver-nvidia/charts/dra-driver-nvidia-gpu \
  --version 0.5.1 \
  --namespace dra-driver-nvidia-gpu --create-namespace \
  --values values.yaml

Confirm that the controller is running:

kubectl rollout status deployment/dra-driver-nvidia-gpu-controller \
  --namespace dra-driver-nvidia-gpu

Create a ComputeDomain. The controller adds its finalizer, renders a ResourceClaimTemplate for the workload channel, and starts an IMEX daemon DaemonSet that uses this same image:

cat <<EOF | kubectl apply -f -
apiVersion: resource.nvidia.com/v1beta1
kind: ComputeDomain
metadata:
  name: my-compute-domain
spec:
  numNodes: 1
  channel:
    resourceClaimTemplate:
      name: my-compute-domain-channel
EOF

Check the domain status and the rendered claim template:

kubectl get computedomain my-compute-domain -o jsonpath='{.status.status}'
kubectl get resourceclaimtemplate my-compute-domain-channel

The status reports NotReady until the IMEX daemons are running on the domain's nodes.

Configuration

To allocate GPUs as well as ComputeDomains, the chart requires an explicit acknowledgement that the standard NVIDIA device plugin is not deployed on the same nodes. When the NVIDIA GPU Operator provides the driver, also point the chart at the driver root it mounts on the host:

cat > values.yaml <<EOF
image:
  repository: cgr.dev/ORGANIZATION/dra-driver-nvidia-gpu
  tag: latest
resources:
  gpus:
    enabled: true
gpuResourcesEnabledOverride: true
nvidiaDriverRoot: /run/nvidia/driver
EOF

If the kubelet plugin pods stay in Init, read the logs of their init-container container. The kubelet-plugin-prestart.sh check reports where it looked for nvidia-smi and libnvidia-ml.so.1, and suggests a nvidiaDriverRoot value when the driver is found elsewhere.

Documentation and Resources

• Upstream repository
• Upstream wiki
• Kubernetes Dynamic Resource Allocation
• Chainguard Academy

What are Chainguard Containers?

Chainguard's free tier of Starter container images are built with Wolfi, our minimal Linux undistro.

All other Chainguard Containers are built with Chainguard OS, Chainguard's minimal Linux operating system designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

• Minimal design, without unnecessary software bloat
• Daily builds to ensure container images are up-to-date with available security patches
• High quality build-time SBOMs attesting to the provenance of all artifacts within the image
• Verifiable signatures provided by Sigstore
• Reproducible builds with Cosign and apko (read more about reproducibility)

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a development, or -dev, variant.

In all other cases, including Chainguard Containers tagged as :latest or with a specific version number, the container images include only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager.

Although the -dev container image variants have similar security features as their more minimal versions, they include additional software that is typically not necessary in production environments. We recommend using multi-stage builds to copy artifacts from the -dev variant into a more minimal production image.

Need additional packages?

To improve security, Chainguard Containers include only essential dependencies. Need more packages? Chainguard customers can use Custom Assembly to add packages, either through the Console, chainctl, or API.

To use Custom Assembly in the Chainguard Console: navigate to the image you'd like to customize in your Organization's list of images, and click on the Customize image button at the top of the page.

Learn More

Refer to our Chainguard Containers documentation on Chainguard Academy. Chainguard also offers VMs and Libraries — contact us for access.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.