Connect to Vertex AI generative AI models

Preview — Using the Cloud Run integrations

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Vertex AI is a machine learning (ML) platform that lets you train, tune, and deploy ML models and AI applications. To add generative AI capabilities to an existing Cloud Run service, you can use Vertex AI to access generative AI models such as Gemini, PaLM, Imagen, Codey, and more.

This guide covers how to use Cloud Run integrations to quickly set up access to Vertex AI for your Cloud Run service.

Before you begin

• Make sure the Cloud Run service you are integrating with is already deployed.
• Review the Vertex AI pricing page. You are billed for individual resources used in the integration.

Required roles

To use the Cloud Run integrations, you or your administrator must grant IAM roles on two different principals.

Click to view required roles for your Google Account

To get the permissions that you need to use Cloud Run integrations, ask your administrator to grant your Google Account the following IAM roles on your project:

• Cloud Run Developer (roles/run.developer)
• Compute Viewer (roles/compute.viewer)
• Serverless Integrations Developer (roles/runapps.developer)
• Serverless Integrations Operator (roles/runapps.operator)
• Service Account User (roles/iam.serviceAccountUser)

Click to view the required roles for the service account

To deploy the Cloud Run service, you can either use the auto-created Compute Engine default service account, or specify a user-managed service account. The service account must have the following roles:

• Cloud Run Developer (roles/run.developer)
• Logs Bucket Writer (roles/logging.bucketWriter)
• Service Account User (roles/iam.serviceAccountUser)
• Storage Admin (roles/storage.admin)
• Project IAM Admin (roles/project.iamAdmin)

Connect a service to Vertex AI

When you use Cloud Run integrations to connect to Vertex AI, the integration does the following:

• Enables the Vertex AI API for the Cloud Run service to access the Vertex AI generative AI models.
• Grants the service identity the required roles.

You can create the integration by using the console or the command line.

Console

1. In the Google Cloud console, go to the Cloud Run page.

   Go to Cloud Run

2. Click the service in the services list that you want to use in this integration.

3. Click the Integrations tab.

4. Click Add integration.

5. Click Vertex AI.

6. Use the default name or change the name to the value you want to use.

7. If prompted to enable a list of APIs, click Enable and wait for the APIs to be enabled.

8. Click Submit and wait for the integration to complete. It can take up to a minute.

Command line

1. Update to the latest Google Cloud CLI:

   gcloud components update
   

2. Create the integration:

   gcloud beta run integrations create \
   --type=vertex-genai \
   --service=SERVICE \
   

   Replace:

   • SERVICE with your Cloud Run service name.

   Optionally, include the --service-account=SERVICE_ACCOUNT_EMAIL flag.

   • Replace SERVICE_ACCOUNT_EMAIL with the user-managed service account email address ([e-mail]) that is used to deploy the service. Omit this flag if you want to use the default compute service account (SERVICE_ACCOUNT_ID@PROJECT_ID.iam.gserviceaccount.com).

     To continue using the specified service account when deploying Cloud Run integrations, run:

     gcloud config set runapps/deployment-service-account
     

3. Wait up to a minute, during which time your Cloud Run service is granted permissions to call the Vertex AI API.

   [vertex-genai] integration [vertex-genai-xxx] has been created successfully.
   

   You can check the status using gcloud beta run integrations describe.

View Vertex AI integrations

To view the current status of Vertex AI integrations for your Cloud Run service:

Console

1. In the Google Cloud console, go to the Cloud Run page.

   Go to Cloud Run

2. Click the service in the services list that you want to use in this integration.

3. Click the Integrations tab.

4. Locate the Vertex AI integration you are interested in and click View details.

Command line

1. Use the following command to get a list of available integrations:

   gcloud beta run integrations list

2. Using a name from the returned list, display the details for it:

   gcloud beta run integrations describe INTEGRATION_NAME

   Replace INTEGRATION_NAME with the name of your Vertex AI integration.

Delete Vertex AI integrations

Deleting an integration will also delete all of the Google Cloud resources associated with that integration, but it will not delete your Cloud Run service.

To delete a Vertex AI integration from your Cloud Run service:

Console

1. In the Google Cloud console, go to the Cloud Run page.

   Go to Cloud Run

2. Click the service you want to delete an integration from.

3. Click the Integrations tab.

4. Locate the Vertex AI integration you are interested in and click the ellipsis icon to the right of the integration, then click Delete.

Command line

1. Use the following command to get a list of available integrations:

   gcloud beta run integrations list

2. Using a name from the returned list, delete the integration:

   gcloud beta run integrations delete INTEGRATION_NAME
   

   Replace:

   • INTEGRATION_NAME with the name of your Vertex AI integration.

   Optionally, you can use the following flag options:

   Option	Description
   --service-account	The user-managed service account email address to perform the delete operation.

Access Vertex AI generative AI models in your code

After you complete the integration, you can send requests to the Vertex AI model APIs, and use the Vertex AI Studio to quickly test and customize generative AI models in your application.

Limitations

The Cloud Run integrations is only supported in certain regions. If you need to use a region that is not supported by the Cloud Run integrations, refer to Vertex AI locations.