Generate text by using a context cache

Generate text by using an existing context cache.

Explore further

For detailed documentation that includes this code sample, see the following:

• Use a context cache

Code sample

C#

Before trying this sample, follow the C# setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI C# API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

using Google.Cloud.AIPlatform.V1Beta1;
using System;
using System.Threading.Tasks;

public class UseContextCache
{
    public async Task<string> Use(string projectId, CachedContentName name)
    {
        var predictionServiceClient = new PredictionServiceClientBuilder
        {
            Endpoint = $"us-central1-aiplatform.googleapis.com"
        }.Build();

        var generateContentRequest = new GenerateContentRequest
        {
            CachedContentAsCachedContentName = name,
            Model = $"projects/{projectId}/locations/us-central1/publishers/google/models/gemini-1.5-pro-001",
            Contents =
            {
                new Content
                {
                    Role = "USER",
                    Parts =
                    {
                        new Part { Text = "What are the papers about?" },
                    }
                }
            }
        };

        GenerateContentResponse response = await predictionServiceClient.GenerateContentAsync(generateContentRequest);

        string responseText = response.Candidates[0].Content.Parts[0].Text;
        Console.WriteLine($"Response: {responseText}");

        return responseText;
    }
}

Go

Before trying this sample, follow the Go setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Go API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// useContextCache shows how to use an existing cached content, when prompting the model
// contentName is the ID of the cached content
func useContextCache(w io.Writer, contentName string, projectID, location, modelName string) error {
	// location := "us-central1"
	// modelName := "gemini-1.5-pro-001"
	ctx := context.Background()

	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)
	model.CachedContentName = contentName
	prompt := genai.Text("What are the papers about?")

	res, err := model.GenerateContent(ctx, prompt)
	if err != nil {
		return fmt.Errorf("error generating content: %w", err)
	}

	if len(res.Candidates) == 0 ||
		len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	fmt.Fprintf(w, "generated response: %s\n", res.Candidates[0].Content.Parts[0])
	return nil
}

Python

Before trying this sample, follow the Python setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Python API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

import vertexai

from vertexai.preview.generative_models import GenerativeModel
from vertexai.preview import caching

# TODO(developer): Update and un-comment below lines
# PROJECT_ID = "your-project-id"
# cache_id = "your-cache-id"

vertexai.init(project=PROJECT_ID, location="us-central1")

cached_content = caching.CachedContent(cached_content_name=cache_id)

model = GenerativeModel.from_cached_content(cached_content=cached_content)

response = model.generate_content("What are the papers about?")

print(response.text)
# Example response:
# The provided text is about a new family of multimodal models called Gemini, developed by Google.
# ...

What's next

To search and filter code samples for other Google Cloud products, see the Google Cloud sample browser.