This page describes how to drive deep-learning tasks such as image recognition, natural language processing, as well as other compute-intensive tasks using node pools with NVIDIA graphics processing unit (GPU) hardware accelerators for compute power with your Cloud Run for Anthos container instance.
Adding a node pool with GPUs to your GKE cluster
Have an administrator create a node pool with GPUs:
Setting up your service to consume GPUs
Specify a resource limit
to consume GPUs for your service. You can use either the gcloud
command line
or the console:
Console
- Go to Cloud Run
Click Create service to display the Create service form.
In the Service settings section:
- Select Cloud Run for Anthos on Google Cloud as your development platform.
- Select the GKE cluster with the GPU-enabled node pool.
- Specify the name you want to give to your service.
- Select which connectivity you would like to use to invoke the service.
- Click Next to continue to the next page of the service creation form.
In the Configure the service's first revision section:
- Add a container image URL.
- Click SHOW ADVANCED SETTINGS and in the GPU Allocated dropdown
menu, select the number of GPUs
you would like to allocate to your container instance.
Click Create to deploy the image to Cloud Run for Anthos and wait for the deployment to finish.
Command line
Download your service configuration into a
service.yaml
file for your local workspace:gcloud run services describe SERVICE --format export > service.yaml
Update the
resources
attribute with thenvidia.com/gpu
value:apiVersion: serving.knative.dev/v1 kind: Service metadata: name: SERVICE spec: template: spec: containers: - image: IMAGE resources: limits: nvidia.com/gpu: GPU_UNITS
Replace:
- SERVICE with the name of your Cloud Run for Anthos service.
- IMAGE with the URL of your container image.
- GPU_UNITS with the desired GPU value
in Kubernetes GPU units. For example, specify
1
for 1 GPU.
Update your service with its new configuration:
gcloud run services replace service.yaml
For more information on GPU performance and cost, see GPUs.