This is a Preview version of the Cloud Run for Anthos documentation for use with Anthos fleets and Anthos Service Mesh. Learn more.

The current GA version of the Cloud Run for Anthos documentation remains available for existing users. New product evaluations during the free trial should use the Preview installation documentation.

Using NVIDIA GPUs

This page describes how to drive deep-learning tasks such as image recognition, natural language processing, as well as other compute-intensive tasks using node pools with NVIDIA graphics processing unit (GPU) hardware accelerators for compute power with your Cloud Run for Anthos container instance.

Adding a node pool with GPUs to your GKE cluster

Have an administrator create a node pool with GPUs:

  1. Add a GPU-enabled node pool to your GKE cluster.

  2. Install NVIDIA's device drivers to the nodes.

Setting up your service to consume GPUs

You can specify a resource limit to consume GPUs for your service by using the Cloud Console or the gcloud command-line tool when you deploy a new service, update an existing service, or deploy a revision:

Console

  1. Go to Cloud Run for Anthos
  2. Click Create service to display the Create service form.

  3. In the Service settings section:

    1. Select the GKE cluster with the GPU-enabled node pool.
    2. Specify the name you want to give to your service.
    3. Click Next to continue to the next section.
  4. In the Configure the service's first revision section:

    image

    1. Add a container image URL.
    2. Click Advanced settings and in the GPU allocated menu, select the number of GPUs that you want to allocate to your service. image
  5. Click Next to continue to the next section.

  6. In the Configure how this service is triggered section, select which connectivity you would like to use to invoke the service.

  7. Click Create to deploy the image to Cloud Run for Anthos and wait for the deployment to finish.

Command line

You can download the configuration of an existing service into a YAML file with the gcloud run services describe command by using the --format=export flag. You can then modify that YAML file and deploy those changes with the gcloud beta run services replace command. You must ensure that you modify only the specified attributes.

  1. Download the configuration of your service into a file named service.yaml on local workspace:

    gcloud run services describe SERVICE --format export > service.yaml

    Replace SERVICE with the name of your Cloud Run for Anthos service.

  2. In your local file, update the nvidia.com/gpu attribute:

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
    name: SERVICE_NAME
    spec:
    template:
     spec:
       containers:
       – image: IMAGE_URL
         resources:
           limits:
            nvidia.com/gpu: "GPU_UNITS"
    

    Replace GPU_UNITS with the desired GPU value in Kubernetes GPU units. For example, specify 1 for 1 GPU.

  3. Deploy the YAML file and replace your service with the new configuration by running the following command:

    gcloud beta run services replace service.yaml

For more information on GPU performance and cost, see GPUs.