Using GPUs with your service

This page describes how to drive deep-learning tasks such as image recognition, natural language processing, as well as other compute-intensive tasks using node pools with NVIDIA graphics processing unit (GPU) hardware accelerators for compute power with your Cloud Run for Anthos container instance.

Adding a node pool with GPUs to your GKE cluster

Have an administrator create a node pool with GPUs:

  1. Add a GPU-enabled node pool to your GKE cluster.

  2. Install NVIDIA's device drivers to the nodes.

Setting up your service to consume GPUs

Specify a resource limit to consume GPUs for your service. You can use either the gcloud command line or the console:

Console

  1. Go to Cloud Run

  2. Click Create service to display the Create service form.

    image

  3. In the Service settings section:

    • Select Cloud Run for Anthos on Google Cloud as your development platform.
    • Select the GKE cluster with the GPU-enabled node pool.
    • Specify the name you want to give to your service.
    • Select which connectivity you would like to use to invoke the service.
    • Click Next to continue to the next page of the service creation form.

  4. In the Configure the service's first revision section:

    image

    • Add a container image URL.
    • Click SHOW ADVANCED SETTINGS and in the GPU Allocated dropdown menu, select the number of GPUs you would like to allocate to your container instance. image

  5. Click Create to deploy the image to Cloud Run for Anthos and wait for the deployment to finish.

Command line

  1. Download your service configuration into a service.yaml file for your local workspace:

    gcloud run services describe SERVICE --format export > service.yaml

  2. Update the resources attribute with the nvidia.com/gpu value:

    apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: SERVICE
spec:
template:
  spec:
    containers:
    - image: IMAGE
      resources:
        limits:
         nvidia.com/gpu: GPU_UNITS

    Replace:

    • SERVICE with the name of your Cloud Run for Anthos service.
    • IMAGE with the URL of your container image.
    • GPU_UNITS with the desired GPU value in Kubernetes GPU units. For example, specify 1 for 1 GPU.

  3. Update your service with its new configuration:

    gcloud run services replace service.yaml

For more information on GPU performance and cost, see GPUs.