Generative AI on Vertex AI Cookbook

A collection of guides and examples for Generative AI on Vertex AI.

Explore more in the Generative AI Repository on GitHub.

Gemini Quickstarts

• 

  Intro to Gemini

  Get started with Gemini in Vertex AI with the Python SDK.

  Colab | Colab Enterprise | GitHub

• 

  Intro to Prompt Engineering

  Learn the essentials and best practices of prompt engineering.

  Colab | Colab Enterprise | GitHub

• 

  Function Calling with Gemini

  Connect Gemini to external tools using function calling.

  Colab | Colab Enterprise | GitHub

• 

  Grounding with Gemini

  Connect Gemini to real-world data from Google Search or Vertex AI Search to improve response quality.

  Colab | Colab Enterprise | GitHub

• 

  Gemini Batch prediction

  Use Batch Prediction to run inference on a large number of examples.

  Colab | Colab Enterprise | GitHub

• 

  Long Context Window

  Use the Long Context Window to process large amounts of multimodal data.

  Colab | Colab Enterprise | GitHub

• 

  Intro to Context Caching

  Use context caching to store frequently used data.

  Colab | Colab Enterprise | GitHub

• 

  Intro to Controlled Generation with the Gemini API

  Learn to control Gemini API output formats for easier data processing.

  Colab | Colab Enterprise | GitHub

• 

  Call Gemini by using the OpenAI Library

  Learn how to call Gemini using Chat Completions.

  Colab | Colab Enterprise | GitHub

Featured Tutorials

• 

  Get started with Chirp 2 for Speech-to-Text

  Learn about how to use Chirp 2, the latest generation of Google's multilingual ASR-specific models.

  Colab | Colab Enterprise | GitHub

• 

  Intro to Imagen 3 Image Generation

  Use Imagen 3 to create photorealistic images.

  Colab | Colab Enterprise | GitHub

• 

  Get Started with Text Embeddings + Vertex AI Vector Search

  Use AI's multitool, Embeddings, and Vertex AI Vector Search perform semantic matching.

  Colab | Colab Enterprise | GitHub

• 

  Intro to LangGraph with Gemini

  Learn how to combine LangGraph's workflow capabilities with Gemini's language understanding and generation skills to streamline and automate complex financial analysis tasks.

  Colab | Colab Enterprise | GitHub

• 

  Create custom podcast episodes

  Use Gemini, LangGraph, and Text-to-Speech to create custom podcast episodes.

  Colab | Colab Enterprise | GitHub

• 

  Storytelling with Gemini and Text-to-Speech

  Use Gemini and Text-to-Speech to create and read a story with multiple characters.

  Colab | Colab Enterprise | GitHub

• 

  Analyze a codebase

  Use Gemini to generate code, summarize a codebase, debug, improve code, and assess code.

  Colab | Colab Enterprise | GitHub

• 

  LLM Security for developers

  Learn about prompt injection attacks and how to mitigate them.

  Colab | Colab Enterprise | GitHub

• 

  Gen AI Evaluation Service

  Evaluate Gemini responses using metrics and custom datasets.

  Colab | Colab Enterprise | GitHub

• 

  Supervised Fine-tuning with Gemini

  Learn how to fine-tune Gemini for specific tasks.

  Colab | Colab Enterprise | GitHub

• 

  Intro to Vertex AI RAG Engine

  Build custom RAG workflows with Gemini and Vertex AI RAG Engine.

  Colab | Colab Enterprise | GitHub

• 

  Document Processing with Gemini

  Use Gemini to process documents for classification, extraction, and summarization.

  Colab | Colab Enterprise | GitHub

All Tutorials

Filter by: Batch Chirp Code Documents Embeddings Evaluation Function Calling Gemini Grounding Image Imagen JSON LangChain LangGraph Multimodal Orchestration Prompting RAG REST Search Security Speech Tuning Vector Search

Gemini	Intro to Gemini Get started with Gemini in Vertex AI with the Python SDK.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Prompting	Intro to Prompt Engineering Learn the essentials and best practices of prompt engineering.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Function Calling	Function Calling with Gemini Connect Gemini to external tools using function calling.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Grounding Search RAG	Grounding with Gemini Connect Gemini to real-world data from Google Search or Vertex AI Search to improve response quality.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Batch	Gemini Batch prediction Use Batch Prediction to run inference on a large number of examples.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini	Long Context Window Use the Long Context Window to process large amounts of multimodal data.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini	Intro to Context Caching Use context caching to store frequently used data.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini JSON	Intro to Controlled Generation with the Gemini API Learn to control Gemini API output formats for easier data processing.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini	Call Gemini by using the OpenAI Library Learn how to call Gemini using Chat Completions.	Open in Colab Open in Colab Enterprise View on GitHub
Speech Chirp	Get started with Chirp 2 for Speech-to-Text Learn about how to use Chirp 2, the latest generation of Google's multilingual ASR-specific models.	Open in Colab Open in Colab Enterprise View on GitHub
Imagen Image	Intro to Imagen 3 Image Generation Use Imagen 3 to create photorealistic images.	Open in Colab Open in Colab Enterprise View on GitHub
Embeddings Vector Search	Get Started with Text Embeddings + Vertex AI Vector Search Use AI's multitool, Embeddings, and Vertex AI Vector Search perform semantic matching.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini LangGraph LangChain Orchestration	Intro to LangGraph with Gemini Learn how to combine LangGraph's workflow capabilities with Gemini's language understanding and generation skills to streamline and automate complex financial analysis tasks.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini LangGraph LangChain Orchestration Speech	Create custom podcast episodes Use Gemini, LangGraph, and Text-to-Speech to create custom podcast episodes.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Speech	Storytelling with Gemini and Text-to-Speech Use Gemini and Text-to-Speech to create and read a story with multiple characters.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Code	Analyze a codebase Use Gemini to generate code, summarize a codebase, debug, improve code, and assess code.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Security Prompting	LLM Security for developers Learn about prompt injection attacks and how to mitigate them.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Evaluation	Gen AI Evaluation Service Evaluate Gemini responses using metrics and custom datasets.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Tuning	Supervised Fine-tuning with Gemini Learn how to fine-tune Gemini for specific tasks.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini RAG	Intro to Vertex AI RAG Engine Build custom RAG workflows with Gemini and Vertex AI RAG Engine.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Documents Multimodal	Document Processing with Gemini Use Gemini to process documents for classification, extraction, and summarization.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Multimodal Prompting	Gemini with Multimodal Data How to prompt Gemini with multimodal data (text, documents, images, video, and audio).	Open in Colab Open in Colab Enterprise View on GitHub
Gemini REST	Get started with the Gemini REST API (cURL) Use the Gemini API with REST/curl.	Open in Colab Open in Colab Enterprise View on GitHub
Gemini Evaluation Prompting	Prompt Engineering, Evaluation, and Prompt Templating Use Gen AI Evaluation Service SDK for prompt engineering and evaluation.	Open in Colab Open in Colab Enterprise View on GitHub
Embeddings Multimodal	Intro to Multimodal Embeddings Learn about multimodal embeddings.	Open in Colab Open in Colab Enterprise View on GitHub
Embeddings Tuning	Intro to Embeddings Tuning Learn how to tune embeddings models.	Open in Colab Open in Colab Enterprise View on GitHub
Embeddings Vector Search	Task Type Embeddings Learn how to get better embeddings for your specific task.	Open in Colab Open in Colab Enterprise View on GitHub
Vector Search Embeddings	Hybrid Search with Vertex AI Vector Search Learn how to use hybrid search with Vertex AI Vector Search combining semantic & keyword search.	Open in Colab Open in Colab Enterprise View on GitHub

What's next

• Learn about LLMs, Vertex AI, and Generative AI models with Generative AI beginner's guide.
• Explore more resources in the Generative AI GitHub repo.