Get started with generic search

Get started with generic search

Vertex AI Search brings together the power of deep information retrieval, state-of-the-art natural language processing, and the latest in large language processing to understand user intent and return the most relevant results for the user.

With Vertex AI Search, you can create apps for searching and for making recommendations. Vertex AI Search also has special capabilities for some industries, such as media, healthcare, and retail.

This tutorial focuses on creating a search app for generic data; that is, data which are not specific to an industry vertical.

This tutorial explains how to create search apps for different kinds of data:

• Website data from crawling cloud.google.com/generative-ai-app-builder/
• Structured data in the form of an NDJSON-formatted dataset ingested from a Cloud Storage bucket
• Unstructured data in the form of PDFs ingested from a Cloud Storage bucket

Before following this tutorial, make sure you have done the steps in Before you begin.

Note: This tutorial is for Vertex AI Search. For getting started with Vertex AI Agents, see Codelab: Create a generative chat app.

---

To follow step-by-step guidance for this task directly in the Google Cloud console, click Guide me:

Guide me

---

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Vertex AI Agent Builder (Discovery Engine), BigQuery, Cloud Storage APIs.

   Enable the APIs

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the Vertex AI Agent Builder (Discovery Engine), BigQuery, Cloud Storage APIs.

   Enable the APIs

Enable Vertex AI Agent Builder

1. In the Google Cloud console, go to the Agent Builder page.

   Agent Builder

2. Read and agree to the Terms of Service, then click Continue and activate the API.

Create a data store

First, you'll create a data store which contains the data that your search app will index.

Click the tab for the type of data that you want to put in your data store: website, structured, or unstructured.

Website data

1. Go to the Create data store page.

   Agent Builder - Create data store

2. In the Select a data source pane, select Website content.

3. In the Specify the websites for your data store pane, make sure that Advanced website indexing is turned off.

4. In the Sites to include field, enter:

   cloud.google.com/generative-ai-app-builder/*
   

5. Click Continue.

6. In the Configure your data store pane, select global (Global) as the location for your data store.

7. Enter a name for your data store. Note the ID that is generated. You'll need this later.

8. Click Create.

Structured data

1. Go to the Create data store page.

   Agent Builder - Create data store

2. In the Select a data source pane, select Cloud Storage.

3. In the Import data from Cloud Storage pane, select Structured data (JSONL).

4. Make sure File is selected.

5. In the gs:// field, enter the following value:

   cloud-samples-data/gen-app-builder/search/kaggle_movies/movie_metadata.ndjson
   

   This Cloud Storage bucket contains an NDJSON-formatted file of movies made available by Kaggle.

6. Click Continue.

7. Assign key properties as follows:

   Field name	Key property
   homepage	uri
   overview	description

   And, click Continue.

8. In the Configure your data store pane, select global (Global) as the location for your data store.

9. Enter a name for your data store. Note the ID that is generated. You'll need this later.

10. Click Create.

Unstructured data

1. Go to the Create data store page.

   Agent Builder - Create data store

2. In the Select a data source pane, select Cloud Storage.

3. In the Import data from Cloud Storage pane, select Unstructured documents (PDF, HTML, TXT and more).

4. Make sure Folder is selected.

5. In the gs:// field, enter the following value:

   cloud-samples-data/gen-app-builder/search/alphabet-investor-pdfs
   

   This Cloud Storage bucket contains earnings report PDFs from the Alphabet investor site.

6. In the Configure your data store pane, select global (Global) as the location for your data store.

7. Enter a name for your data store. Note the ID that is generated. You'll need this later.

8. Click Create.

Create a search app

Next, you'll create a search app and link the data store you created previously.

Website data

1. Go to the Create App page.

   Agent Builder - Create App

2. In the Select app type pane, select Search.

3. Make sure the content is Generic and that Enterprise features is turned on.

4. In the Your app name field, enter a name for your app. Your app ID appears under the app name.

5. In the External name of your company or organization field, enter the company or organization name. For this tutorial, you can use Google Cloud, because the app will search a Google Cloud website.

6. Select global (Global) as the location for your app, and then click Continue.

7. In the list of data stores, select the data store that you created earlier, and then click Create.

8. Go to your app's Data page to view the list of indexed websites and change them if needed.

Structured data

1. Go to the Create App page.

   Agent Builder - Create App

2. In the Select app type pane, select Search.

3. Make sure the content is Generic and that Enterprise features is turned on.

4. In the Your app name field, enter a name for your app. Your app ID appears under the app name.

5. In the External name of your company or organization field, enter the company or organization name. For this tutorial, you can use Cymbal Cinemas, because the app will search a data store of movies.

6. Select global (Global) as the location for your app, and then click Continue.

7. In the list of data stores, select the data store that you created earlier, and then click Create.

8. On your app's Data page, click the Activity tab to see the status of your data ingestion. Import completed displays in the Status column when the import process is complete. For this dataset, this typically takes two to three minutes. You might need to click Refresh to see Import completed.

9. Click the Documents tab to see the number of documents imported.

Unstructured data

1. Go to the Create App page.

   Agent Builder - Create App

2. In the Select app type pane, select Search.

3. Make sure the content is Generic and that Enterprise features is turned on.

4. In the Your app name field, enter a name for your app. Your app ID appears under the app name.

5. In the External name of your company or organization field, enter the company or organization name. For this tutorial, you can use Alphabet, because the app will search over Alphabet documents.

6. Select global (Global) as the location for your app, and then click Continue.

7. In the list of data stores, select the data store that you created earlier, and then click Create.

8. On your app's Data page, click the Activity tab to see the status of your data ingestion. When the import process is complete, the Status column displays Import completed. For this dataset, the import process typically takes two to three minutes. You might need to click Refresh to see the Import completed status.

9. Click the Documents tab to see the number of documents imported.

Preview your app

Website data

1. On the Apps page, click the name of your new app.

2. In the navigation menu, click Preview to test the search app.

3. Optional: Click the view icons to switch between mobile view and desktop view.

4. In the search bar, perform a search for Vertex AI Search to view results about the product.

Structured data

1. On the Apps page, click the name of your new app.

2. In the navigation menu, click Preview to test the search app.

3. Optional: Click the view icons to switch between mobile view and desktop view.

4. In the search bar, enter trains, and then press Enter to get results for movies related to trains.

Unstructured data

1. On the Apps page, click the name of your new app.

2. In the navigation menu, click Preview to test the search app.

3. Optional: Click the view icons to switch between mobile view and desktop view.

4. In the search bar, enter Google, and then press Enter to view your results.

Configure the search widget

1. In the navigation menu, click Configurations.

2. On the UI tab, configure settings for the search app.

   Different UI configurations are available for different data types. For more information, see Configure results for the search widget.

3. If you change configurations on the UI tab, click Save and publish.

4. Go to the Autocomplete tab to configure specific settings for autocomplete.

   For more information, see Configure autocomplete.

5. Go to the Advanced tab to configure advanced features.

   For more information, see About advanced features.

Deploy a search app

1. On the Apps page, click the name of the app that you want to deploy.

2. In the navigation menu, click Integration.

3. Make sure the Widget tab is selected.

4. Select JWT or OAuth based as the widget authorization type.

5. In the Domain field, enter the domain name for web page where you'll put the widget. For example, if you are going to copy the widget to the web page example.com/ai.html, enter example.com as the domain.

6. Click Add, and then click Save.

7. Copy the code snippet provided in the Copy the following code to your web application section.

8. In your codebase, generate an authorization token.

9. To pass the authorization token to your widget, use the "Set authorization token" code snippet provided in the Copy the following code to your web application section and replace the text <JWT or OAuth token provided by your backend> with your authorization token.

10. Test your deployed app. You can further adjust the configuration by following the steps "Configure the search widget". Make sure to click Save and publish so that the changes take effect on the deployed widget.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

1. To avoid unnecessary Google Cloud charges, use the Google Cloud console to delete your project if you don't need it.
2. If you created a new project to learn about Vertex AI Agent Builder and you no longer need the project, delete the project.
3. If you used an existing Google Cloud project, delete the resources you created to avoid incurring charges to your account. For more information, see Delete an app.
4. Follow the steps in Turn off Vertex AI Agent Builder.

What's next

• Introduction to Vertex AI Search
• About data, data stores, and engines