Use the CountTokens API to prevent requests from exceeding the model context window, and estimate potential costs based on billable characters.
The CountTokens API can use the same contents
parameter as Gemini API
inference requests.
Supported Models:
Model | Code |
---|---|
Gemini 1.5 Flash | gemini-1.5-flash-002 gemini-1.5-flash-001 gemini-1.5-flash-preview-0514 |
Gemini 1.5 Pro | gemini-1.5-pro-002 gemini-1.5-pro-001 gemini-1.5-pro-preview-0514 |
Gemini 1.0 Pro Vision | gemini-1.0-pro-vision gemini-1.0-pro-vision-001 |
Gemini 1.0 Pro | gemini-1.0-pro gemini-1.0-pro-001 gemini-1.0-pro-002 |
Gemini Experimental | gemini-experimental |
Limitations:
gemini-1.0-pro-vision-001
and gemini-1.0-ultra-vision-001
use a fixed
number of tokens for video inputs.
Example syntax
Syntax to send a count tokens request.
curl
curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:countTokens \ -d '{ "contents": [{ ... }], "system_instruction": { "role": "...", "parts": [{ ... }], "tools": [{ "function_declarations": [{ ... }] }], } }'
Python
gemini_model = GenerativeModel(MODEL_ID) model_response = gemini_model.count_tokens([...])
Parameter list
This class consists of two main properties: role
and parts
. The role
property denotes the individual producing the content, while the parts
property contains multiple elements, each representing a segment of data within a
message.
Parameters | |
---|---|
|
Optional: The identity of the entity that creates the message. Set the string to one of the following:
The For non-multi-turn conversations, this field can be left blank or unset. |
|
A list of ordered parts that make up a single message. Different parts may have different IANA MIME types. |
Part
A data type containing media that is part of a multi-part Content
message.
Parameters | |
---|---|
|
Optional: A text prompt or code snippet. |
|
Optional: Inline data in raw bytes. |
|
Optional: Data stored in a file. |
Blob
Content blob. If possible this send as text rather than raw bytes.
Parameters | |
---|---|
|
IANA MIME type of the data. |
|
Raw bytes. |
FileData
URI based data.
Parameters | |
---|---|
|
IANA MIME type of the data. |
|
The Cloud Storage URI to the file storing the data. |
system_instruction
This field is for user provided system_instructions
. It is the same
as contents
but with a limited support of the content types.
Parameters | |
---|---|
|
IANA MIME type of the data. This field is ignored internally. |
|
Text only. Instructions that users want to pass to the model. |
FunctionDeclaration
A structured representation of a function declaration as defined by the OpenAPI 3.0 specification that represents a function the model may generate JSON inputs for.
Parameters | |
---|---|
|
The name of the function to call. |
|
Optional: Description and purpose of the function. |
|
Optional: Describes the parameters of the function in the OpenAPI JSON Schema Object format: OpenAPI 3.0 specification. |
|
Optional: Describes the output from the function in the OpenAPI JSON Schema Object format: OpenAPI 3.0 specification. |
Examples
Get token count from text prompt
This example counts the tokens of a single text prompt:
REST
To get the token count and the number of billable characters for a prompt by
using the Vertex AI API, send a POST
request to the publisher model
endpoint.
Before using any of the request data, make the following replacements:
- LOCATION: The region to process the request. Available
options include the following:
Click to expand a partial list of available regions
us-central1
us-west4
northamerica-northeast1
us-east4
us-west1
asia-northeast3
asia-southeast1
asia-northeast1
- PROJECT_ID: Your project ID.
- MODEL_ID: The model ID of the multimodal model that you want to use.
- ROLE:
The role in a conversation associated with the content. Specifying a role is required even in
singleturn use cases.
Acceptable values include the following:
USER
: Specifies content that's sent by you.
- TEXT: The text instructions to include in the prompt.
- NAME: The name of the function to call.
- DESCRIPTION: Description and purpose of the function.
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens
Request JSON body:
{ "contents": [{ "role": "ROLE", "parts": [{ "text": "TEXT" }] }], "system_instruction": { "role": "ROLE", "parts": [{ "text": "TEXT" }] }, "tools": [{ "function_declarations": [ { "name": "NAME", "description": "DESCRIPTION", "parameters": { "type": "OBJECT", "properties": { "location": { "type": "TYPE", "description": "DESCRIPTION" } }, "required": [ "location" ] } } ] }] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens" | Select-Object -Expand Content
You should receive a JSON response similar to the following.
Python
NodeJS
Java
Go
Get token count from media prompt
This example counts the tokens of a prompt that uses various media types.
REST
To get the token count and the number of billable characters for a prompt by
using the Vertex AI API, send a POST
request to the publisher model
endpoint.
Before using any of the request data, make the following replacements:
- LOCATION: The region to process the request. Available
options include the following:
Click to expand a partial list of available regions
us-central1
us-west4
northamerica-northeast1
us-east4
us-west1
asia-northeast3
asia-southeast1
asia-northeast1
- PROJECT_ID: Your project ID.
- MODEL_ID: The model ID of the multimodal model that you want to use.
- ROLE:
The role in a conversation associated with the content. Specifying a role is required even in
singleturn use cases.
Acceptable values include the following:
USER
: Specifies content that's sent by you.
- TEXT: The text instructions to include in the prompt.
- FILE_URI:
The URI or URL of the file to include in the prompt. Acceptable values include the following:
- Cloud Storage bucket URI: The object must either be publicly readable or reside in
the same Google Cloud project that's sending the request. For
gemini-1.5-pro
andgemini-1.5-flash
, the size limit is 2 GB. Forgemini-1.0-pro-vision
, the size limit is 20 MB. - HTTP URL: The file URL must be publicly readable. You can specify one video file, one audio file, and up to 10 image files per request. Audio files, video files, and documents can't exceed 15 MB.
- YouTube video URL:The YouTube video must be either owned by the account that you used to sign in to the Google Cloud console or is public. Only one YouTube video URL is supported per request.
When specifying a
fileURI
, you must also specify the media type (mimeType
) of the file. - Cloud Storage bucket URI: The object must either be publicly readable or reside in
the same Google Cloud project that's sending the request. For
- MIME_TYPE:
The media type of the file specified in the
data
orfileUri
fields. Acceptable values include the following:Click to expand MIME types
application/pdf
audio/mpeg
audio/mp3
audio/wav
image/png
image/jpeg
image/webp
text/plain
video/mov
video/mpeg
video/mp4
video/mpg
video/avi
video/wmv
video/mpegps
video/flv
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens
Request JSON body:
{ "contents": [{ "role": "ROLE", "parts": [ { "file_data": { "file_uri": "FILE_URI", "mime_type": "MIME_TYPE" } }, { "text": "TEXT } ] }] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:countTokens" | Select-Object -Expand Content
You should receive a JSON response similar to the following.
Python
NodeJS
Java
Go
What's next
- Learn more about the Gemini API.