Turn on advanced website indexing

Turning on advanced website indexing generates an index for the websites that you specify when you create your data store. This requires domain verification and imposes a quota on the number of pages that you can index. You can upgrade this quota if necessary. Advanced website indexing incurs additional cost. For more information, see Advanced website indexing.

To turn on advanced website indexing for an existing data store, follow these steps:

  1. In the Google Cloud console, go to the Vertex AI Agent Builder page.

    Agent Builder

  2. In the navigation menu, click Data stores.

  3. Click the data store for which you want to turn on advanced website search.

  4. On your data store page, in the Type row, click Upgrade.

  5. On the Verify your domains page, follow these steps:

    1. Click Verify for an unverified domain. The Welcome to Google Search Console page appears.

    2. Follow the on-screen instructions to verify a domain or a URL prefix, depending on whether you are verifying an entire domain or a URL prefix that is part of a domain.

      When you have finished the domain verification workflow, the Data page for your data store appears. The Status column should indicate that indexing is in progress for the website. For more information, see Verify your site ownership in the Search Console Help Center.

    3. Repeat for every unverified domain, then click Upgrade to advanced indexing.

For more information about verifying domain ownership, see Verify website domains.