Context caching overview

To see an example of context caching, run the "Intro to context caching" notebook in one of the following environments:

Open in Colab | Open in Colab Enterprise | Open in Vertex AI Workbench | View on GitHub

Context caching aims to reduce the cost and latency of requests to Gemini that contain repeated content. By default, Google automatically caches inputs for all Gemini models to reduce latency and accelerate responses for subsequent prompts. Through the Vertex AI API, you can create context caches and exercise more control over them by:

• Using a context cache by referencing its contents in a prompt request with its resource name.
• Updating the expiration time (TTL) of a context cache beyond the default 60 minutes.
• Deleting a context cache when you no longer need it.

You can also use the Vertex AI API to get information about a context cache.

Caches created using the Vertex AI API interact with default Google caching, which can result in additional caching beyond the contents specified when creating a cache. To achieve zero cache data retention, disable default Google caching and refrain from creating caches with the Vertex API. See enabling and disabling caching for more information.

For the Gemini 2.5 and later model families, the cached input tokens are charged at a 75% discount relative to standard input tokens when a cache hit occurs.

View cache hit token information in the responses metadata field. To disable this, refer to Generative AI and data governance.

When to use context caching

Context caching is particularly well suited to scenarios where a substantial initial context is referenced repeatedly by subsequent requests.

Cached context items, such as a large amount of text, an audio file, or a video file, can be used in prompt requests to the Gemini API to generate output. Requests that use the same cache in the prompt also include text unique to each prompt. For example, each prompt request that composes a chat conversation might include the same context cache that references a video along with unique text that comprises each turn in the chat.

Consider using context caching for use cases such as:

• Chatbots with extensive system instructions
• Repetitive analysis of lengthy video files
• Recurring queries against large document sets
• Frequent code repository analysis or bug fixing

Important: Using the same context cache and prompt doesn't guarantee consistent model responses because the responses from LLMs are nondeterministic. A context cache doesn't cache any output.

Cost-efficiency through caching

Context caching is a paid feature designed to reduce overall operational costs. Billing is based on the following factors:

• Cache token count: The number of input tokens cached, billed at a reduced rate when included in subsequent prompts.
• Storage duration: The amount of time cached tokens are stored, billed hourly. The cached tokens are deleted when a context cache expires.
• Other factors: Other charges apply, such as for non-cached input tokens and output tokens.

Note: When you create the cached contents, the first call allocates the cache storage. For this initial call, you are charged at the normal rate based on the number of input tokens. Subsequent calls that refer to the cached contents are charged at the reduced rate. For pricing details, see Gemini and context caching on the Gemini pricing page.

The number of tokens in the cached part of your input can be found in the metadata field of your response, under the cachedContentTokenCount field.

Context caching support for Provisioned Throughput is in Preview for default caching. Context caching using the Vertex AI API is not supported for Provisioned Throughput. Refer to the Provisioned Throughput guide for more details.

Note: Context caching is now supported by both base and fine-tuned Gemini models(see Context cache for fine-tuned Gemini models).

Supported models

The following Gemini models support context caching:

• Gemini 2.5 Flash-Lite
• Gemini 2.5 Pro
• Gemini 2.5 Flash
• Gemini 2.0 Flash
• Gemini 2.0 Flash-Lite

For more information, see Available Gemini stable model versions. Note that context caching supports all MIME types for supported models.

Availability

Context caching is available in regions where Generative AI on Vertex AI is available. For more information, see Generative AI on Vertex AI locations.

VPC Service Controls support

Context caching supports VPC Service Controls, meaning your cache cannot be exfiltrated beyond your service perimeter. If you use Cloud Storage to build your cache, include your bucket in your service perimeter as well to protect your cache content.

For more information, see VPC Service Controls with Vertex AI in the Vertex AI documentation.

What's next

• Learn about the Gemini API.
• Learn how to use multimodal prompts.