Send requests to the Vertex AI API for Gemini
Send requests to the Gemini API in Vertex AI by using a programming language SDK or the REST API to begin building your generative AI application on Google Cloud.
Create a Google Cloud account to get started
In order to try this quickstart, you'll need to create a Google Cloud account. With this account, you'll get $300 in free credit, plus free usage of over 20 products— not charged against your $300 credit.
Get started for freeFor more information on getting set up on Google Cloud after your account is created, see Get set up on Google Cloud.
Set up your environment
Click to learn how to set up your environment
Learn how to set up your environment by selecting one of the following tabs:
Python
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
-
Create local authentication credentials for your user account:
gcloud auth application-default login
Install or update the Vertex AI SDK for Python by running the following command:
pip3 install --upgrade "google-cloud-aiplatform>=1.38"
Node.js
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
-
Create local authentication credentials for your user account:
gcloud auth application-default login
Install or update the Vertex AI SDK for Node.js by running the following command:
npm install @google-cloud/vertexai
Java
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
-
Create local authentication credentials for your user account:
gcloud auth application-default login
-
To add
google-cloud-vertexai
as a dependency, add the appropriate code for your environment:Maven with BOM
Add the following HTML to your
pom.xml
:<dependencyManagement> <dependencies> <dependency> <groupId>com.google.cloud</groupId> <artifactId>libraries-bom</artifactId> <version>26.32.0</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>com.google.cloud</groupId> <artifactId>google-cloud-vertexai</artifactId> </dependency> </dependencies>
Maven without BOM
Add the following HTML to your
pom.xml
:<dependency> <groupId>com.google.cloud</groupId> <artifactId>google-cloud-vertexai</artifactId> <version>0.4.0</version> </dependency>
Gradle without BOM
Add the following to your
build.gradle
implementation 'com.google.cloud:google-cloud-vertexai:0.4.0'
Go
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
-
Create local authentication credentials for your user account:
gcloud auth application-default login
Review the available Vertex AI API Go packages to determine which package best meets your project's needs:
Package cloud.google.com/go/vertexai (recommended)
vertexai
is a human authored package that provides access to common capabilities and features.This package is recommended as the starting point for most developers building with the Vertex AI API. To access capabilities and features not yet covered by this package, use the auto-generated
aiplatform
instead.Package cloud.google.com/go/aiplatform
aiplatform
is an auto-generated package.This package is intended for projects that require access to Vertex AI API capabilities and features not yet provided by the human authored
vertexai
package.
Install the desired Go package based on your project's needs by running one of the following commands:
# Human authored package. Recommended for most developers. go get cloud.google.com/go/vertexai
# Auto-generated package. go get cloud.google.com/go/aiplatform
C#
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
-
Create local authentication credentials for your user account:
gcloud auth application-default login
REST
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Enable the Vertex AI API.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
- Configure environment variables by entering the following. Replace
PROJECT_ID
with the ID of your Google Cloud project.MODEL_ID="gemini-1.5-flash-001" PROJECT_ID="PROJECT_ID"
- Provision the endpoint:
gcloud beta services identity create --service=aiplatform.googleapis.com --project=${PROJECT_ID}
-
Optional: If you are using Cloud Shell and you are asked to authorize Cloud Shell, click Authorize.
Send a text-only request
After you set up your local environment, send a text-only request to the Vertex AI Gemini API. The following sample returns a list of possible names for a specialty flower store.
Python
To send a prompt request, create a Python file (.py
) and copy
the following code into the file. Replace PROJECT_ID
with the ID of your Google Cloud project. Then, run the Python file from the
command line or in an IDE, or embed the code into your application where
appropriate.
Node.js
To send a prompt request, create a Node.js file (.js
) and copy
the following code into the file. Replace PROJECT_ID
with the ID of your Google Cloud project. After, run the Javascript file from the
command line or in an IDE, or embed the code into your application where
appropriate.
Java
To send a prompt request, create a Java file (.java
) and copy
the following code into the file. Set your-google-cloud-project-id
to your Google Cloud project ID. After, run the Java file from the command line or
in an IDE, or embed the code into your application where appropriate.
Go
To send a prompt request, create a Go file (.go
) and copy the
following code into the file. Replace projectID
with
the ID of your Google Cloud project. After, run the Go file from the command line
or in an IDE, or embed the code into your application where appropriate.
C#
To send a prompt request, create a C# file (.cs
) and copy the
following code into the file. Set your-project-id
to your
Google Cloud project ID. After, run the C# file from the command line or in an
IDE, or embed the code into your application where appropriate.
REST
You can send this prompt request from Cloud Shell, the command line, or in an IDE, or you can embed the REST call into your application where appropriate.
curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:streamGenerateContent -d \ $'{ "contents": { "role": "user", "parts": [ { "text": "What\'s a good name for a flower shop that specializes in selling bouquets of dried flowers?" } ] } }'The model returns a response. Note that the response is generated in sections with each section separately evaluated for safety.
Send a request that includes an image
After you set up your local environment, send a request that includes text and an image to the Vertex AI Gemini API. The following sample returns a description of the provided image (image for Java sample).
Python
To send a prompt request, create a Python file (.py
) and copy
the following code into the file. Replace PROJECT_ID
with the ID of your Google Cloud project. Then, run the Python file from the
command line or in an IDE, or embed the code into your application where
appropriate.
Node.js
To send a prompt request, create a Node.js file (.js
) and copy
the following code into the file. Replace PROJECT_ID
with the ID of your Google Cloud project. After, run the Javascript file from the
command line or in an IDE, or embed the code into your application where
appropriate.
Java
To send a prompt request, create a Java file (.java
) and copy
the following code into the file. Set your-google-cloud-project-id
to your Google Cloud project ID. After, run the Java file from the command line or
in an IDE, or embed the code into your application where appropriate.
Go
To send a prompt request, create a Go file (.go
) and copy the
following code into the file. Replace projectID
with
the ID of your Google Cloud project. After, run the Go file from the command line
or in an IDE, or embed the code into your application where appropriate.
C#
To send a prompt request, create a C# file (.cs
) and copy the
following code into the file. Set your-project-id
to your
Google Cloud project ID. After, run the C# file from the command line or in an
IDE, or embed the code into your application where appropriate.
REST
You can send this prompt request from Cloud Shell, the command line, or in an IDE, or you can embed the REST call into your application where appropriate.
curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:streamGenerateContent -d \ $'{ "contents": { "role": "user", "parts": [ { "fileData": { "mimeType": "image/jpeg", "fileUri": "gs://generativeai-downloads/images/scones.jpg" } }, { "text": "Describe this picture." } ] } }'
The model returns a response. Note that the response is generated in sections with each section separately evaluated for safety.
Send a request that includes audio and video
After you set up your local environment, send a request that includes text, audio, and video to the Vertex AI Gemini API. The following sample returns a description of the provided video, including anything important from the audio track.
Python
To send a prompt request, create a Python file (.py
) and copy
the following code into the file. Replace PROJECT_ID
with the ID of your Google Cloud project. Then, run the Python file from the
command line or in an IDE, or embed the code into your application where
appropriate.
Node.js
To send a prompt request, create a Node.js file (.js
) and copy
the following code into the file. Replace PROJECT_ID
with the ID of your Google Cloud project. After, run the Javascript file from the
command line or in an IDE, or embed the code into your application where
appropriate.
Java
To send a prompt request, create a Java file (.java
) and copy
the following code into the file. Set your-google-cloud-project-id
to your Google Cloud project ID. After, run the Java file from the command line or
in an IDE, or embed the code into your application where appropriate.
Go
To send a prompt request, create a Go file (.go
) and copy the
following code into the file. Replace projectID
with
the ID of your Google Cloud project. After, run the Go file from the command line
or in an IDE, or embed the code into your application where appropriate.
C#
To send a prompt request, create a C# file (.cs
) and copy the
following code into the file. Set your-project-id
to your
Google Cloud project ID. After, run the C# file from the command line or in an
IDE, or embed the code into your application where appropriate.
REST
You can send this prompt request from Cloud Shell, the command line, or in an IDE, or you can embed the REST call into your application where appropriate.
curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:streamGenerateContent -d \ $'{ "contents": { "role": "user", "parts": [ { "fileData": { "mimeType": "video/mp4", "fileUri": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4" } }, { "text": "Provide a description of the video. The description should also contain anything important which people say in the video." } ] } }'
The model returns a response. Note that the response is generated in sections with each section separately evaluated for safety.
What's next
- Learn more about the Gemini API in Vertex AI.
- Explore the Vertex AI Gemini API SDK reference for Python, Node.js, Java, Go, or C#.
- See the Model API for Gemini in Vertex AI.
- Learn about calling Vertex AI models by using the OpenAI library.