Allocate existing devices to workloads with DRA

You can flexibly request devices for your Google Kubernetes Engine (GKE) workloads by using dynamic resource allocation (DRA). This document shows you how to create a ResourceClaimTemplate to request existing devices in node pools in your cluster, and then create a workload to observe how Kubernetes flexibly allocates the devices to your Pods.

This document is intended for Application operators and Data engineers who run workloads like AI/ML or high performance computing (HPC).

About requesting devices with DRA

When you set up your GKE infrastructure for DRA, the DRA drivers on your nodes create DeviceClass objects in the cluster. A DeviceClass defines a category of devices, such as GPUs, that are available to request for workloads. A platform administrator can optionally deploy additional DeviceClasses that limit which devices you can request in specific workloads.

To request devices within a DeviceClass, you create one of the following objects:

  • ResourceClaim: A ResourceClaim lets a Pod or a user request hardware resources by filtering for certain parameters within a DeviceClass.
  • ResourceClaimTemplate: A ResourceClaimTemplate defines a template that Pods can use to automatically create new per-Pod ResourceClaims.

For more information about ResourceClaims and ResourceClaimTemplates, see When to use ResourceClaims and ResourceClaimTemplates.

The examples in this document use a basic ResourceClaimTemplate to request the specified device configuration. For more information about all of the fields that you can specify, see the ResourceClaimTemplate API reference.

Limitations

The following limitations apply:

Before you begin

Before you start, make sure that you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
    • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.

Claim devices and deploy workloads

To request per-Pod device allocation, you create a ResourceClaimTemplate that has your requested device configuration, such as GPUs of a specific type. When you deploy a workload that references the ResourceClaimTemplate, Kubernetes creates ResourceClaims for each Pod in the workload based on the ResourceClaimTemplate. Kubernetes allocates the requested resources and schedules the Pods on corresponding nodes.

To request GPUs in a workload with DRA, follow these steps:

  1. Save the following manifest as gpu-claim-template.yaml:

    apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-claim-template
spec:
  spec:
    devices:
      requests:
      - name: single-gpu
        exactly:
          deviceClassName: gpu.nvidia.com
          allocationMode: ExactCount
          count: 1

  2. Create the ResourceClaimTemplate:

    kubectl create -f gpu-claim-template.yaml

  3. To create a workload that references the ResourceClaimTemplate, save the following manifest as dra-gpu-example.yaml:

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: dra-gpu-example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dra-gpu-example
  template:
    metadata:
      labels:
        app: dra-gpu-example
    spec:
      containers:
      - name: ctr
        image: ubuntu:22.04
        command: ["bash", "-c"]
        args: ["echo $(nvidia-smi -L || echo Waiting...); sleep infinity"]
        resources:
          claims:
          - name: single-gpu
      resourceClaims:
      - name: single-gpu
        resourceClaimTemplateName: gpu-claim-template
      tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"

  4. Deploy the workload:

    kubectl create -f dra-gpu-example.yaml

Verify the hardware allocation

You can verify that your workloads have been allocated hardware by checking the ResourceClaim or by looking at the logs for your Pod. To verify the allocation, follow these steps:

  1. Get the ResourceClaim associated with the workload that you deployed:

    kubectl get resourceclaims

    The output is similar to the following:

    NAME                                               STATE                AGE
dra-gpu-example-64b75dc6b-x8bd6-single-gpu-jwwdh   allocated,reserved   9s

  2. Get more details about the hardware assigned to the Pod:

    kubectl describe resourceclaims RESOURCECLAIM

    Replace RESOURCECLAIM with the full name of the ResourceClaim that you got from the output of the previous step.

    The output is similar to the following:

    Name:         dra-gpu-example-68f595d7dc-prv27-single-gpu-qgjq5
Namespace:    default
Labels:       <none>
Annotations:  resource.kubernetes.io/pod-claim-name: single-gpu
API Version:  resource.k8s.io/v1
Kind:         ResourceClaim
Metadata:
# Multiple lines are omitted here.
Spec:
  Devices:
    Requests:
      Exactly:
        Allocation Mode:    ExactCount
        Count:              1
        Device Class Name:  gpu.nvidia.com
      Name:                 single-gpu
Status:
  Allocation:
    Devices:
      Results:
        Device:   gpu-0
        Driver:   gpu.nvidia.com
        Pool:     gke-cluster-1-dra-gpu-pool-b56c4961-7vnm
        Request:  single-gpu
    Node Selector:
      Node Selector Terms:
        Match Fields:
          Key:       metadata.name
          Operator:  In
          Values:
            gke-cluster-1-dra-gpu-pool-b56c4961-7vnm
  Reserved For:
    Name:      dra-gpu-example-68f595d7dc-prv27
    Resource:  pods
    UID:       e16c2813-08ef-411b-8d92-a72f27ebf5ef
Events:        <none>

  3. Get logs for the workload that you deployed:

    kubectl logs deployment/dra-gpu-example --all-pods=true

    The output is similar to the following:

    [pod/dra-gpu-example-64b75dc6b-x8bd6/ctr] GPU 0: Tesla T4 (UUID: GPU-2087ac7a-f781-8cd7-eb6b-b00943cc13ef)

    This output indicates that GKE allocated one GPU to the container.

What's next