With Vertex AI, you can ground model outputs in the following ways:
- Ground with Google Search - ground a model with publicly available web data.
- Ground to your own data - ground a model with your own data from Vertex AI Search as a data store (Preview).
Grounding with Google Search
Use Grounding with Google Search if you want to connect the model with world knowledge, a wide possible range of topics, or up-to-date information on the internet.
You have to display a Google Search entry point when using this feature. To learn more about the requirements, see Google Search entry point.
To learn more about model grounding in Vertex AI, see the Grounding overview.
Supported models
The following models support grounding:
- Gemini 1.5 Pro with text input only
- Gemini 1.5 Flash with text input only
- Gemini 1.0 Pro with text input only
Supported languages
- English (en)
- Spanish (es)
- Japanese (ja)
Ground your model with Google Search
Use the following instructions to ground a model with publicly available web data.
Considerations
To use grounding with Google Search, you must enable a Google Search entry point. See more at Google Search entry point.
For ideal results, use a temperature of
0.0
. To learn more about setting this config, see the Gemini API request body from the model reference.Grounding with Google Search has a limit of one million queries per day. If you require more, reach out to your support team.
REST
Before using any of the request data, make the following replacements:
- LOCATION: The region to process the request.
- PROJECT_ID: Your project ID.
- MODEL_ID: The model ID of the multimodal model.
- TEXT: The text instructions to include in the prompt.
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent
Request JSON body:
{ "contents": [{ "role": "user", "parts": [{ "text": "TEXT" }] }], "tools": [{ "googleSearchRetrieval": {} }], "model": "projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID" }
To send your request, expand one of these options:
You should receive a JSON response similar to the following:
{ "candidates": [ { "content": { "role": "model", "parts": [ { "text": "Chicago weather changes rapidly, so layers let you adjust easily. Consider a base layer, a warm mid-layer (sweater-fleece), and a weatherproof outer layer." } ] }, "finishReason": "STOP", "safetyRatings":[ "..." ], "groundingMetadata": { "webSearchQueries": [ "What's the weather in Chicago this weekend?" ], "searchEntryPoint": { "renderedContent": "....................." } } } ], "usageMetadata": { "..." } }
Console
To use Grounding with Google Search with the Vertex AI Studio, follow these steps:
- In the Google Cloud console, go to the Vertex AI Studio page.
- Click the Freeform tab.
- In the side panel, click the Ground model responses toggle.
- Click Customize and set Google Search as the source.
- Enter your prompt in the text box and click Submit.
Your prompt responses now ground to Google Search.
Python
To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.
Understand your response
If your model prompt successfully grounds to Google Search from the Vertex AI Studio or from the API, then the responses include metadata with source links (web URLs). However, there are several reasons this metadata might not be provided, and the prompt response won't be grounded. These reasons include low source relevance or incomplete information within the model's response.
Citations
Displaying citations is highly recommended. They aid users in validating responses from publishers themselves and add avenues for further learning.
Citations for responses from Google Search sources should be shown both inline and in aggregate. See the following image as a suggestion on how to do this.
Use of alternative search engine options
Customer's use of Grounding with Google Search does not prevent Customer from offering alternative search engine options, making alternative search options the default option for Customer Applications, or displaying their own or third party search suggestions or search results in Customer Applications, provided that that any such non-Google Search services or associated results are displayed separately from the Grounded Results and Search Entry Points and cannot reasonably be attributed to, or confused with results provided by, Google.
Ground Gemini to your data
This section shows you how to ground text responses to a Vertex AI Agent Builder data store by using the Vertex AI API.
Supported models
The following models support grounding:
- Gemini 1.5 Pro with text input only
- Gemini 1.5 Flash with text input only
- Gemini 1.0 Pro with text input only
Prerequisites
There are prerequisites needed before you can ground model output to your data.
- Enable Vertex AI Agent Builder and activate the API.
- Create a Vertex AI Agent Builder data source and app.
- Link your data store to your app in Vertex AI Agent Builder. The data source serves as the foundation for grounding Gemini 1.0 Pro in Vertex AI.
- Enable Enterprise edition for your data store.
See the Introduction to Vertex AI Search for more.
Enable Vertex AI Agent Builder
In the Google Cloud console, go to the Agent Builder page.
Read and agree to the terms of service, then click Continue and activate the API.
Vertex AI Agent Builder is available in the global
location, or the eu
and us
multi-region. To
learn more, see Vertex AI Agent Builder locations
Create a data store in Vertex AI Agent Builder
To ground your models to your source data, you need to have prepared and saved your data to Vertex AI Search. To do this, you need to create a data store in Vertex AI Agent Builder.
If you are starting from scratch, you need to prepare your data for ingestion into Vertex AI Agent Builder. See Prepare data for ingesting to get started. Depending on the size of your data, ingestion can take several minutes to several hours. Only unstructured data stores are supported for grounding.
After you've prepared your data for ingestion, you can Create a search data store. After you've successfully created a data store, Create a search app to link to it and Turn Enterprise edition on.
Example: Ground the Gemini 1.5 Flash model
Use the following instructions to ground a model with your own data.
If you don't know your data store ID, follow these steps:
In the Google Cloud console, go to the Vertex AI Agent Builder page and in the navigation menu, click Data stores.
Click the name of your data store.
On the Data page for your data store, get the data store ID.
REST
To test a text prompt by using the Vertex AI API, send a POST request to the publisher model endpoint.
Before using any of the request data, make the following replacements:
- LOCATION: The region to process the request.
- PROJECT_ID: Your project ID.
- MODEL_ID: The model ID of the multimodal model.
- TEXT: The text instructions to include in the prompt.
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent
Request JSON body:
{ "contents": [{ "role": "user", "parts": [{ "text": "TEXT" }] }], "tools": [{ "retrieval": { "vertexAiSearch": { "datastore": projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID } } }], "model": "projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID" }
To send your request, expand one of these options:
You should receive a JSON response similar to the following:
{ "candidates": [ { "content": { "role": "model", "parts": [ { "text": "You can make an appointment on the website https://dmv.gov/" } ] }, "finishReason": "STOP", "safetyRatings": [ "..." ], "groundingMetadata": { "retrievalQueries": [ "How to make appointment to renew driving license?" ], "groundingChunks": [ { "retrievedContext": { "uri": "gs://my-data-bucket/dmv.txt", "title": "dmv" } } ], "groundingSupport": [ { "segment": { "startIndex": 25, "endIndex": 147 }, "segment_text": "ipsum lorem ...", "supportChunkIndices": [1, 2], "confidenceScore": [0.9541752, 0.97726375] }, { "segment": { "startIndex": 294, "endIndex": 439 }, "segment_text": "ipsum lorem ...", "supportChunkIndices": [1], "confidenceScore": [0.9541752, 0.9325467] } ] } } ], "usageMetadata": { "..." } }
Understanding your response
groundingSupport
is a response field which appears in the groundingMetadata
field. It
contains the information about the support chunks that were used to generate the response.
This field contains the content from where it got its information
from. It's like providing footnotes or citations, so you can check if the AI's response is
accurate and trustworthy.
Python
To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.
Console
To ground your model output to Vertex AI Agent Builder by using Vertex AI Studio in the Google Cloud console, follow these steps:
- In the Google Cloud console, go to the Vertex AI Studio page.
- Click the Freeform tab.
- In the side panel, click the Ground model responses toggle to enable grounding.
- Click Customize and set Vertex AI Agent Builder as the source. The path should follow this format: projects/project_id/locations/global/collections/default_collection/dataStores/data_store_id.
- Enter your prompt in the text box and click Submit.
Your prompt responses now ground to Vertex AI Agent Builder.
What's next
- Learn how to send chat prompt requests.
- Learn about responsible AI best practices and Vertex AI's safety filters.
- To learn how to ground the PaLM models, see Grounding in Vertex AI