Vertex AI's Grounding with Google Search: how to use it and why

Introduction

Once in a while, you come across a feature that is so easy to use and so useful that you don’t know how you lived without it before. For me, Vertex AI’s Grounding with Google Search is one of those features. 

In this blog post, I explain why you need grounding with large language models (LLMs) and how Vertex AI’s Grounding with Google Search can help with minimal effort on your part.

Why grounding with LLMs?

You might be asking: LLMs are great, why do I even need grounding?

LLMs are great at generating content but they fall short in a number of ways:

1. Generated content can be fake (hallucinations).

2. Generated content can be stale (not based on up-to-date information).

3. Generated content has no citation to its source.

4. Generated content has no relation to your private data (based on public knowledge only).

Grounding LLMs is essential to go around these limitations and build more reliable and trustworthy generative AI applications.

Grounding in Vertex AI

Google Cloud’s Vertex AI Agent Builder provides different grounding solutions based on public or private data:

In this blog post, I focus on Grounding with Google Search that allows LLMs to ground their responses on trusted Google Search world knowledge and public facts. This helps to improve hallucinations and provide more up-to-data to LLMs.

Without Grounding

First, let’s look at some examples without grounding. Let’s ask a text-based model on Vertex AI some questions that require recent data.

I’m an Arsenal fan and curious if the model knows about Arsenal’s latest game result.

Prompt: What was Arsenal FC’s latest game result?

Response: I do not have access to real-time information, so I cannot provide the most recent game results for Arsenal. To get the most up-to-date information, I recommend checking official sports websites or news sources.

In London, where I live, we talk about the weather all the time. Can the model answer questions about the weather?

Prompt: What was the weather like in London yesterday?

Response: I do not have access to real-time information, including weather data. To get the most up-to-date weather information, I recommend checking a reliable weather source for London. You can find weather forecasts and historical data on websites and apps like BBC Weather, AccuWeather, or The Weather Channel.

Not surprisingly, the model cannot answer these questions, as they require recent data. 

Next, let’s enable grounding and see how the responses change.

Enable Grounding with Google Search

To enable Grounding with Google Search, go to Vertex AI studio in the Google Cloud console, then create or select a prompt. On the left, in the advanced section, you should see a Enable grounding toggle:

Once you enable it, you’re prompted to customize it:

Choose Google search:

With Grounding

Now that we’ve enabled Grounding with Google Search, let’s ask the same questions again:

Prompt: What was Arsenal FC’s latest game result?

Response: Arsenal's latest game was a 2-1 win against Everton on the final day of the Premier League season.[1] However, they fell just short in the title race.[1]

Grounding Sources

1. Arsenal vs Everton LIVE! Premier League result, match stream and latest updates today 

That’s interesting! I get a response and a citation for the response. But is it correct? To verify, I go to Google and search for Arsenal game results and voila, it’s indeed correct:

Let’s now ask the other question:

Prompt: What was the weather like in London yesterday?

Response: On May 20, 2024, London's weather was pleasant, with temperatures reaching 17°C during the day and dropping to 8°C at night.[3] There was an average of 6 hours of sunshine per day, and 13 days with some rainfall.[3][4] The weather in London in May is typically mild and sunny, with occasional showers.[1][2] However, the weather can be unpredictable, so it is always a good idea to check the latest forecast before planning any outdoor activities.

That’s a much better response than before. I can confirm that the weather was indeed pleasant and sunny on May 20! 

This seems like magic but how do you actually code this into your application?

How to code for Grounding with Google Search

You might think that coding something powerful like Grounding with Google Search is involved but in reality, it’s very straightforward. 

Here’s an example gemini_grounding_example.py from our Python docs. It is just a matter of defining the Google Search Retrieval tool and using it in the generate content call:

Grounding is not just limited to Python. Here is GroundingWebSample.cs in C# where you define the Google Search Retrieval tool in the generate content request:

Conclusion

In this blog post, I explored Vertex AI’s Grounding with Google Search. It’s an essential tool when factuality and up-to-date information are important, and I’m pleased with how useful and easy it was to use it. If you want to learn more, here are some more resources:

• Docs: Grounding overview

• Talk: Improve the factuality of your generative AI apps by grounding responses in your data.

• Getting Started with Grounding with Gemini in Vertex AI notebook

• Grounding sample in Python

• Grounding sample in C#

• Vertex AI Search for grounding with your own data