Website search checklist

This page provides a checklist of the steps required to create a website search app, which is a type of generic search app. For a checklist to create other types of generic search apps, see Generic search checklist.

If you're new to Vertex AI Agent Builder, consider following the Get started with generic search tutorial to create a sample website data app.

Choose between basic website search and advanced website indexing

Vertex AI Agent Builder offers two types of website data stores: basic website search and advanced website indexing.

You can choose to turn on advanced website indexing to index the web pages in your search data store. Advanced website indexing requires domain verification and offers several additional features over basic website search. For more information, see Advanced website indexing.

Depending on the type of website data store you require for your business needs, select one of the following checklists:

Basic website search Advanced website indexing

Tip: You can try the Vertex AI Search widget that indexes the Vertex AI Search documentation using advanced website indexing. Try Vertex AI Search!

Set up a Google Cloud project

Set up a Google Cloud project, turn on Vertex AI Agent Builder, and set up access control for your project. You can use an existing Google Cloud project if you already have one.

Actions

1. Review the Before you begin page and confirm that you have completed the steps.

Prepare your data

Understand how a website search app (a type of generic search app) behaves and the prerequisites to include websites in a website data store.

Actions

1. Review the information about the relationship between generic apps and data stores in About apps and data stores and what constitutes website data. When advanced website indexing is turned on for a data store, you can use that data store for blended search, where multiple data stores are connected to an app.

2. Prepare your website data according to the requirements in Website data.

Create a data store

Create a data store by providing the website URL patterns to include and the URL patterns to exclude. The data store creation process involves steps to turn on advanced website indexing, which requires domain verification.

Actions

1. Follow the instructions in Create a data store using website URLs.

   If the data store creation and the domain verification are successful, Vertex AI Agent Builder generates an initial index, which is automatically refreshed on a best-effort basis.

   The initial indexing process might take several hours.

2. Optional: Create a data store that lets you perform sitemap-only indexing and refresh. To do so, follow the instructions in Enable sitemap-only refresh in a data store.

   To understand sitemap-based indexing and refreshing, see Index and refresh web pages using sitemap.

Create your app

Create your website search app and connect it to your new data store.

Actions

1. Create a search app. Ensure to turn on enterprise edition.

Configure your search settings at a data store level

Vertex AI Search provides many configuration options. Some options depend on whether you plan to deploy a search widget.

Actions

Depending on your use case and whether you plan to deploy the out-of-the-box search widget or integrate search API calls into your own code, Vertex AI Search provides several options for configuration.

• You can embed a search widget into your website that automatically provides a search bar and expandable search interface. If you plan to deploy the search widget, configure the following:

  • Search widget results. See Configure results for the search widget.

  • Search widget facets (Preview). See Configure facets for the search widget.

  • Autocomplete. See Update autocomplete settings.

  • Search tuning (Preview). See Improve search results with search tuning.

• If you plan to integrate search API calls into your server or application instead of using the widget, you can configure your search settings using the following options:

  • Autocomplete. Depending on your data, set up autocomplete suggestions based on document content, fields, search history, or user events. See Configure autocomplete.

  • Serving controls (Preview with allowlist). Control when search results are boosted, buried, filtered, or redirected, or whether certain queries are synonyms of other queries. See Configure serving controls for search.

  • Search tuning (Preview). Tune the search model with your own training data. See Improve search results with search tuning.

Configure your search results at a request level

If you plan to deploy your app by integrating search API calls into your own code, Vertex AI Search provides additional options for configuring how your search results are returned.

Actions

1. Configure your search results with the following options:

   • Filter website search.

   • Boost search results.

   • Order web search results.

   • Get snippets.

Preview search results

You can preview your search results to check if your app configurations are working as expected.

Actions

1. To preview your search results, use the console or the API.

   • Console. Use the Vertex AI Agent Builder console Preview page to preview how search widget configurations affect your results. See the Console instructions in Get search results.

   • API. If you're integrating API calls into your application, you can make curl calls to preview your search configurations. See the REST instructions in Get search results.

Refine your search results

Enrich your data store's index by leveraging the structured data that's added to your web pages.

Actions

Addition of structured data to your web pages is an iterative process:

1. Modify the structured data on the web pages. For more information, see Use structured data for advanced website indexing.
2. Update your data store schema with the structured data attributes as necessary.
3. Reconfigure your search results. See Step 6 of this checklist.
4. Preview the search results. See Step 7 of this checklist.
5. Repeat this process as necessary to achieve the required search behaviour before you deploy you search app.

Deploy your search app

When you're happy with the preview version of your search app, share it with your users by deploying it to your website.

Actions

1. You can deploy your search app in either of the following ways:

   • Embed the search widget into your website. Vertex AI Search provides code that you can copy into your website or web application. This deploys the search widget. You can preview search results in the console. See Add the search widget to a webpage.

   • Integrate search API calls into your server or application. For full control over how your search results are displayed, you can integrate API calls into your server or applications. For more information about making API calls, see Get search results. For client library resources, see Vertex AI Agent Builder client libraries.

Maintain your app

You can maintain your app to ensure that latest and necessary data is available in your data store.

Actions

1. Refresh your data.
2. Optional: If your data store has sitemap-based indexing and refresh, submit your sitemaps or the sitemap indexes and keep them up-to-date. The data store index is updated automatically and the included URLs are refreshed daily. To understand sitemap-based indexing and refreshing, see Index and refresh web pages using sitemap.