The Vertex AI API for Gemini lets you create an application with Gemini models. Use it to create requests and then receive responses to help create applications for your use case. The following topics include some example uses cases for Gemini models:
- Gemini 1.5 Pro and Gemini 1.5 Flash use cases
- Gemini 1.0 Pro Vision use cases
- Gemini 1.0 Pro use cases
Create a Google Cloud account to get started
To start using the Vertex AI Model API for Gemini, create a Google Cloud account.
After creating your account, use this document to review the Gemini model request body, model parameters, response body, and some sample requests and responses.
When you're ready, see the Vertex AI API for Gemini quickstart to learn how to send a request to the Vertex AI Gemini API using a using a programming language SDK or the REST API.
Send an HTTP request
The following tabs show you how to send an HTTP request with each Gemini model:
Gemini 1.5 Pro
POST https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/publishers/google/models/gemini-1.5-pro:streamGenerateContent
Gemini 1.5 Flash
POST https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/publishers/google/models/gemini-1.5-flash:streamGenerateContent
Gemini 1.0 Pro
POST https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/publishers/google/models/gemini-1.0-pro:streamGenerateContent
Gemini 1.0 Pro Vision
POST https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/publishers/google/models/gemini-1.0-pro-vision:streamGenerateContent
To send a stream request to the model, see the
streamGenerateContent
method for more information.
To send a non-streaming request to the model, use the
generateContent
method instead.
For a list of supported regions, see Available locations.
Model versions
To use the auto-updated version,
specify the model name without the trailing version number, for example gemini-1.0-pro
instead of gemini-1.0-pro-001
.
For more information, see Gemini model versions and lifecycle.
Request body
The request body contains data with the following structure:
{ "contents": [ { "role": string, "parts": [ { // Union field data can be only one of the following: "text": string, "inlineData": { "mimeType": string, "data": string }, "fileData": { "mimeType": string, "fileUri": string }, // End of list of possible types for union field data. "videoMetadata": { "startOffset": { "seconds": integer, "nanos": integer }, "endOffset": { "seconds": integer, "nanos": integer } } } ] } ], "systemInstruction": { "role": string, "parts": [ { "text": string } ] }, "tools": [ { "functionDeclarations": [ { "name": string, "description": string, "parameters": { object (OpenAPI Object Schema) } } ] } ], "safetySettings": [ { "category": enum (HarmCategory), "threshold": enum (HarmBlockThreshold) } ], "generationConfig": { "temperature": number, "topP": number, "topK": number, "candidateCount": integer, "maxOutputTokens": integer, "presencePenalty": float, "frequencyPenalty": float, "stopSequences": [ string ], "responseMimeType": string } }
Gemini model parameters
You can use the following parameters in your request body:
Parameter | Description |
---|---|
role |
The role in a conversation associated with the content. Specifying a role is required even in
singleturn use cases.
Acceptable values include the following:
|
parts |
Ordered parts that make up the input. Parts may have different MIME types.
For gemini-1.0-pro , only the text field is valid.
The token limit is 32,760.
For gemini-1.0-pro-vision , you may specify either text only,
text and up to 16 images, or text and 1 video clip. The token limit is
16,384.
For gemini-1.5-pro , you can specify any combination and
number of text, image, video, and audio files. The token limit is
1,000,000.
To compute the number of tokens in your request, see Get token count. |
text |
The text instructions or chat dialogue to include in the prompt. |
inlineData |
Serialized bytes data of the image, audio clip, or video clip.
For gemini-1.0-pro-vision , you can specify at most 1 image
with inlineData . To specify up to 16 images, use
fileData .
|
mimeType |
The media type of the file specified in the data or fileUri
fields. Acceptable values include the following:
Click to expand MIME types
For gemini-1.0-pro-vision , the maximum video length is 2
minutes.
For Gemini 1.5 Pro , the maximum length of an audio file is
8.4 hours and the maximum length of a video file (without audio) is
one hour. For more information, see
Gemini 1.5 Pro media requirements.
Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. |
data |
The base64 encoding of the image, PDF, or video
to include inline in the prompt. When including media inline, you must also specify MIMETYPE.
size limit: 20MB |
fileUri |
The Cloud Storage URI of the file to include in the prompt. The bucket object must either be
publicly readable or reside in the same Google Cloud project that's sending the request. You must also
specify the media type (MIMETYPE) of the file.
For gemini-1.5-pro , the size limit is 2GB.
For gemini-1.0-pro-vision , the size limit is 20MB.
|
videoMetadata |
Optional. For video input, the start and end offset of the video in
Duration
format. For example, to specify a 10 second clip starting at 1:00, set
"start_offset": { "seconds": 60 } and
"end_offset": { "seconds": 70 } .
|
systemInstruction (Preview)
|
Optional. Available for gemini-1.5-pro and gemini-1.0-pro-002 .
Instructions for the model to steer it toward better performance. For example, "Answer as concisely as possible" or "Don't use technical terms in your response". The text strings count toward the token limit.
The role field of systemInstruction is ignored and doesn't affect the performance of the model.
|
tools |
A piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. |
functionDeclarations |
One or more function declarations. Each function declaration contains
information about one function that includes the following:
For more information, see Function calling. |
category |
The safety category to configure a threshold for. Acceptable values include the following:
Click to expand safety categories
|
threshold |
The threshold for blocking responses that could belong to the specified
safety category based on probability.
|
temperature |
The temperature is used for sampling during response generation, which occurs when topP
and topK are applied. Temperature controls the degree of randomness in token selection.
Lower temperatures are good for prompts that require a less open-ended or creative response, while
higher temperatures can lead to more diverse or creative results. A temperature of 0
means that the highest probability tokens are always selected. In this case, responses for a given
prompt are mostly deterministic, but a small amount of variation is still possible.
If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature.
|
maxOutputTokens |
Maximum number of tokens that can be generated in the response. A token is
approximately four characters. 100 tokens correspond to roughly 60-80 words.
Specify a lower value for shorter responses and a higher value for potentially longer responses. Range for gemini-1.5-pro: 1-8192 (default: 8192 )
Range for gemini-1.0-pro: 1-8192 (default: 8192 )
Range for gemini-1.0-pro-vision: 1-2048 (default: 2048 )
|
topK |
Top-K changes how the model selects tokens for output. A top-K of
1 means the next selected token is the most probable among all
tokens in the model's vocabulary (also called greedy decoding), while a top-K of
3 means that the next token is selected from among the three most
probable tokens by using temperature.
For each token selection step, the top-K tokens with the highest probabilities are sampled. Then tokens are further filtered based on top-P with the final token selected using temperature sampling. Specify a lower value for less random responses and a higher value for more random responses. Range: 1-40
gemini-1.0-pro and gemini-1.5-pro don't support topK
Default for gemini-1.0-pro-vision: 32
|
topP |
Top-P changes how the model selects tokens for output. Tokens are selected
from the most (see top-K) to least probable until the sum of their probabilities
equals the top-P value. For example, if tokens A, B, and C have a probability of
0.3, 0.2, and 0.1 and the top-P value is 0.5 , then the model will
select either A or B as the next token by using temperature and excludes C as a
candidate.
Specify a lower value for less random responses and a higher value for more random responses. Range: 0.0 - 1.0
gemini-1.5-pro: 0.94
Default for gemini-1.0-pro: 1
Default for gemini-1.0-pro-vision: 1
|
frequencyPenalty |
Positive values penalize tokens that repeatedly appear in the generated text, decreasing the probability of repeating content.
This maximum value for frequencyPenalty is up to, but not including, 2.0 . Its minimum value is -2.0 .
|
presencePenalty |
Positive values penalize tokens that already appear in the generated text, increasing the probability of generating more diverse content.
This maximum value for presencePenalty is up to, but not including, 2.0 . Its minimum value is -2.0 .
|
candidateCount |
The number of response variations to return.
This value must be 1. |
stopSequences |
Specifies a list of strings that tells the model to stop generating text if one
of the strings is encountered in the response. If a string appears multiple
times in the response, then the response truncates where it's first encountered.
The strings are case-sensitive.
For example, if the following is the returned response when stopSequences isn't specified:
public
static string reverse(string myString)
Then the returned response with stopSequences set to ["Str",
"reverse"] is:
public static string
Maximum 5 items in the list. |
responseMimeType (Preview)
|
Optional. Available for gemini-1.5-pro .
The output format of the generated candidate text. Supported MIME types:
|
Response body
{ "candidates": [ { "content": { "parts": [ { "text": string } ] }, "finishReason": enum (FinishReason), "safetyRatings": [ { "category": enum (HarmCategory), "probability": enum (HarmProbability), "blocked": boolean } ], "citationMetadata": { "citations": [ { "startIndex": integer, "endIndex": integer, "uri": string, "title": string, "license": string, "publicationDate": { "year": integer, "month": integer, "day": integer } } ] } } ], "usageMetadata": { "promptTokenCount": integer, "candidatesTokenCount": integer, "totalTokenCount": integer } }
Response element | Description |
---|---|
text |
The generated text. |
finishReason |
The reason why the model stopped generating tokens. If empty, the model
has not stopped generating the tokens. Because the response uses the
prompt for context, it's not possible to change the behavior of how the
model stops generating tokens.
|
category |
The safety category to configure a threshold for. Acceptable values include the following:
Click to expand safety categories
|
probability |
The harm probability levels in the content.
|
blocked |
A boolean flag associated with a safety attribute that indicates if the model's input or output was blocked. |
startIndex |
An integer that specifies where a citation starts in the content. |
endIndex |
An integer that specifies where a citation ends in the content .
|
url |
The URL of a citation source. Examples of a URL source might be a news website or a GitHub repository. |
title |
The title of a citation source. Examples of source titles might be that of a news article or a book. |
license |
The license associated with a citation. |
publicationDate |
The date a citation was published. Its valid formats are
YYYY , YYYY-MM , and YYYY-MM-DD .
|
promptTokenCount |
Number of tokens in the request. |
candidatesTokenCount |
Number of tokens in the response(s). |
totalTokenCount |
Number of tokens in the request and response(s). |
Sample requests
Text
REST
To test a text prompt by using the Vertex AI API with Server-sent events (SSE)
enabled, send a POST request to the publisher model endpoint with
?alt=sse
at the end of the URL.
Before using any of the request data, make the following replacements:
- PROJECT_ID: Your project ID.
HTTP method and URL:
POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent?alt=sse
Request JSON body:
{ "contents": { "role": "user", "parts": { "text": "Give me a recipe for banana bread." } }, "safety_settings": { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_LOW_AND_ABOVE" }, "generation_config": { "temperature": 0.2, "topP": 0.8, "topK": 40 } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent?alt=sse"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent?alt=sse" | Select-Object -Expand Content
You should receive a JSON response similar to the sample response.
Chat
Also see Send chat prompt requests (Gemini).
REST
To test a chat prompt by using the Vertex AI API with Server-sent events (SSE)
enabled, send a POST request to the publisher model endpoint with
?alt=sse
at the end of the URL.
Before using any of the request data, make the following replacements:
- PROJECT_ID: Your project ID.
HTTP method and URL:
POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent?alt=sse
Request JSON body:
{ "contents": [ { "role": "USER", "parts": { "text": "Hello!" } }, { "role": "MODEL", "parts": { "text": "Argh! What brings ye to my ship?" } }, { "role": "USER", "parts": { "text": "Wow! You are a real-life priate!" } } ], "safety_settings": { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_LOW_AND_ABOVE" }, "generation_config": { "temperature": 0.2, "topP": 0.8, "topK": 40, "maxOutputTokens": 200, } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent?alt=sse"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent?alt=sse" | Select-Object -Expand Content
You should receive a JSON response similar to the sample response.
Python
To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Node.js API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
Before trying this sample, follow the Java setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Java API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
Before trying this sample, follow the Go setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Go API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Multimodal
Also see Send multimodal prompt requests.
REST
To test a multimodal prompt by using the Vertex AI API, send a POST request to the publisher model endpoint.
Before using any of the request data, make the following replacements:
- PROJECT_ID: Your project ID.
HTTP method and URL:
POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro-vision:streamGenerateContent
Request JSON body:
{ "contents": { "role": "user", "parts": [ { "fileData": { "mimeType": "image/jpeg", "fileUri": "gs://cloud-samples-data/ai-platform/flowers/daisy/10559679065_50d2b16f6d.jpg" } }, { "text": "Describe this picture." } ] }, "safety_settings": { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_LOW_AND_ABOVE" }, "generation_config": { "temperature": 0.4, "topP": 1.0, "topK": 32, "maxOutputTokens": 2048 } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro-vision:streamGenerateContent"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro-vision:streamGenerateContent" | Select-Object -Expand Content
You should receive a JSON response similar to the sample response.
Python
To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Node.js API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
Before trying this sample, follow the Java setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Java API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
Before trying this sample, follow the Go setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Go API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
C#
Before trying this sample, follow the C# setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI C# API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Function
Also see Function calling.
REST
To test a function prompt by using the Vertex AI API, send a POST request to the publisher model endpoint.
Before using any of the request data, make the following replacements:
- PROJECT_ID: Your project ID.
HTTP method and URL:
POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent
Request JSON body:
{ "contents": { "role": "user", "parts": { "text": "Which theaters in Mountain View show Barbie movie?" } }, "tools": [ { "function_declarations": [ { "name": "find_movies", "description": "find movie titles currently playing in theaters based on any description, genre, title words, etc.", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616" }, "description": { "type": "string", "description": "Any kind of description including category or genre, title words, attributes, etc." } }, "required": [ "description" ] } }, { "name": "find_theaters", "description": "find theaters based on location and optionally movie title which are is currently playing in theaters", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616" }, "movie": { "type": "string", "description": "Any movie title" } }, "required": [ "location" ] } }, { "name": "get_showtimes", "description": "Find the start times for movies playing in a specific theater", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616" }, "movie": { "type": "string", "description": "Any movie title" }, "theater": { "type": "string", "description": "Name of the theater" }, "date": { "type": "string", "description": "Date for requested showtime" } }, "required": [ "location", "movie", "theater", "date" ] } } ] } ] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent" | Select-Object -Expand Content
You should receive a JSON response similar to the sample response.
Python
To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.
Sample responses
Text
data: {"candidates": [{"content": {"role": "model","parts": [{"text": "Ingredients:\n\n- 3 ripe bananas, mashed\n- 1 cup sugar"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": "\n- 1/2 cup (1 stick) unsalted butter, softened\n"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": "- 2 large eggs\n- 2 cups all-purpose flour\n- 1 teaspoon baking soda\n- 1/2 teaspoon salt\n- "}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": "1/2 cup chopped walnuts (optional)\n\nInstructions:\n\n1. Preheat oven to 350 degrees F (175 degrees C). Grease"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": " and flour a 9x5 inch loaf pan.\n2. In a large bowl, cream together the butter and sugar until light and fluffy. Beat in the eggs one at a time, then stir in the mashed bananas.\n3"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}],"citationMetadata": {"citations": [{"startIndex": 322,"endIndex": 451,"uri": "https://discover.texasrealfood.com/texas-home-cooking/whats-in-season-plums-exploring-health-benefits-varieties-and-recipes"}]}}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": ". In a separate bowl, whisk together the flour, baking soda, and salt. Gradually add the dry ingredients to the wet ingredients, mixing until just combined. Fold in the walnuts, if desired.\n4. Pour the batter into the"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}],"citationMetadata": {"citations": [{"startIndex": 472,"endIndex": 614,"uri": "https://commandame.com/urban-cookhouse-half-baked-cookie-recipe/"}]}}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": " prepared loaf pan and bake for 50-60 minutes, or until a toothpick inserted into the center comes out clean.\n5. Let the bread cool in the pan for 10 minutes before turning it out onto a wire rack to cool completely."}]},"finishReason": "STOP","safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}],"citationMetadata": {"citations": [{"startIndex": 666,"endIndex": 796,"uri": "https://dessertdonelight.com/healthy-hawaiian-banana-bread/"},{"startIndex": 728,"endIndex": 851,"uri": "https://earlsgrocery.com/gluten-free-bread/gluten-free-yeast-free-bread-healthy-and-nutition"}]}}],"usageMetadata": {"promptTokenCount": 8,"candidatesTokenCount": 245,"totalTokenCount": 253}}
Chat
data: {"candidates": [{"content": {"role": "model","parts": [{"text": "Avast there, landlubber! Ye be mistaken. I be but a"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "LOW"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": " humble pirate of the seven seas, brought to life by the magic of artificial intelligence"}]},"safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}]}
data: {"candidates": [{"content": {"role": "model","parts": [{"text": ". I be no real-life pirate, but I be mighty good at pretendin'!"}]},"finishReason": "STOP","safetyRatings": [{"category": "HARM_CATEGORY_HARASSMENT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_HATE_SPEECH","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT","probability": "NEGLIGIBLE"},{"category": "HARM_CATEGORY_DANGEROUS_CONTENT","probability": "NEGLIGIBLE"}]}],"usageMetadata": {"promptTokenCount": 23,"candidatesTokenCount": 50,"totalTokenCount": 73}}
Multimodal
[{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"text": " A daisy is growing up through a pile of brown and yellow fall leaves"
}
]
},
"finishReason": "STOP",
"safetyRatings": [
{
"category": "HARM_CATEGORY_HARASSMENT",
"probability": "NEGLIGIBLE"
},
{
"category": "HARM_CATEGORY_HATE_SPEECH",
"probability": "NEGLIGIBLE"
},
{
"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
"probability": "NEGLIGIBLE"
},
{
"category": "HARM_CATEGORY_DANGEROUS_CONTENT",
"probability": "NEGLIGIBLE"
}
]
}
],
"usageMetadata": {
"promptTokenCount": 262,
"candidatesTokenCount": 14,
"totalTokenCount": 276
}
}]
Function
[{
"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"name": "find_theaters",
"args": {
"movie": "Barbie",
"location": "Mountain View, CA"
}
}
}
]
},
"finishReason": "STOP",
"safetyRatings": [
{
"category": "HARM_CATEGORY_HARASSMENT",
"probability": "NEGLIGIBLE"
},
{
"category": "HARM_CATEGORY_HATE_SPEECH",
"probability": "NEGLIGIBLE"
},
{
"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
"probability": "NEGLIGIBLE"
},
{
"category": "HARM_CATEGORY_DANGEROUS_CONTENT",
"probability": "NEGLIGIBLE"
}
]
}
],
"usageMetadata": {
"promptTokenCount": 9,
"totalTokenCount": 9
}
}]
What's next
Learn how to use the Vertex AI API for Gemini:
- Learn how to send chat prompt requests.
- Learn how to send multimodal prompt requests.
- Learn how to call functions.
- Learn how to get token count.
- Learn how to configure safety attributes.