Connect a third-party data source

This page describes how to connect third-party data sources to Vertex AI Search.

When you connect a third-party data source, Vertex AI Search creates a data connector, and associates data stores (called entity data stores) with it for the entities that you specify. Entity types are specific to the data source that you're connecting to. For example, Jira Cloud entities include issues, attachments, comments, and worklogs.

Third-party data sources are available only for generic search apps. Chat, recommendations, and agent apps can't use third-party data sources.

Third-party connectors are not CMEK-compliant.

To import data from a Google data source instead, see Create a search data store.

Before you begin

Contact your Google account team and ask to be added to the allowlist for third-party data source connectors.
Go to the section for the source you plan to use:

Connect Confluence Cloud

Use the following procedure to sync data from Confluence Cloud to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Have the following authentication information ready:
- Instance URL. In the form https://EXAMPLE.atlassian.net—for example, https://google.atlassian.net.
- Instance ID. This is the cloudid and can be found by going to https://EXAMPLE.atlassian.net/_edge/tenant_info and copying the value of cloudId.
Enable OAuth 2.0 and get the client ID and client secret.

Use https://vertexaisearch.cloud.google.com/console/oauth/confluence_oauth.html as the callback URL. For information about enabling OAuth 2.0 for Confluence Cloud and getting the client ID and client secret, see OAuth 2.0 (3LO) apps in the Atlassian Developer documentation.
When setting OAuth 2.0 permission scopes, configure the following scopes:
View scopes
- read:attachment:confluence
- read:configuration:confluence
- read:content-details:confluence
- read:content.metadata:confluence
- read:group:confluence
- read:space:confluence
- read:user:confluence
For user permissions to apply correctly, each Confluence Cloud user must make their email visible to all users. To do so, change the email visibility settings in Confluence Cloud and set the visibility to Anyone. For more information, see Set your email visibility in the Atlassian documentation.

Console

To use the Google Cloud console to sync data from Confluence Cloud to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click New data store.
On the Select a data source page, go to the Third-party sources section and select Confluence.
Enter your authentication information and click Authenticate.
A new window appears. Enter the instance username and password. Check that the authentication succeeded before returning to the Specify the Confluence source for your data store page.
Select which entities to sync, then click Continue.
Select a region for your data connector.
Enter a name for your data connector.
Select a synchronization frequency.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data Stores page.
To check the status of your ingestion, go to the Data Stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
See Preview results for apps with third-party access control.

Connect Jira Cloud

Use the following procedure to sync data from Jira Cloud to Vertex AI Search.

After you set up your data source and import data the first time, you can choose how often the data store syncs with that source.

Before you begin

Before setting up your connection:

Set up access control. Ensure that access control is properly configured for your data source. This step ensures that only authorized users can access and manage the data. For more information, see Use data source access control documentation.
For user permissions to apply correctly, Jira Cloud users must provide sharing consent.
Make sure that you have an Atlassian account, Jira instance, and project.
Verify that you have administrator access to the Jira instance, and project.

Set up authentication and permissions in Jira

Using the instructions in the following sections, ensure you have the necessary authentication details and admin access to your Jira instance. Create a Client ID and Client Secret through the Atlassian Developer Console, configure the required OAuth 2.0 scopes, and set up permissions for users. Finally, retrieve your instance URL and ID, configure roles, and authenticate to sync data between Jira Cloud and Vertex AI Search.

Create client ID and client secret

NOTE: To enable OAuth 2.0 and obtain the client ID and secret, see OAuth 2.0 (3LO) apps in the Atlassian Developer documentation.

Sign in to developer.atlassian.com.
Click the profile icon in the top right corner and select Developer Console.
Click Create and select OAuth 2.0 Integration.
Enter a name for the app.
- Check the terms and conditions checkbox.
- Click Create.
You will find five options: Overview, Distribution, Permissions, Authorization, and Settings. Start with Authorization:

a. Click Authorization.

b. In the Authorization type table, select Add for OAuth 2.0 (3LO).
In the Callback URL field, enter https://vertexaisearch.cloud.google.com/console/oauth/jira_oauth.html, and then click Save changes.

NOTE: If you see the warning: Your app doesn't have any APIs. Add APIs to your app., this will be addressed in the next section.
Select Permissions:

a. Go to Jira API, click Add, then click Configure.

NOTE: When you click Add, the button changes to Configure.

b. Go to the Classic scopes tab and click Edit Scopes. Select the following scopes:
View Scopes
- read:avatar:jira
- read:audit-log:jira
- read:group:jira
- read:issue-security-level:jira
- read:issue-security-scheme:jira
- read:jira-user
- read:jira-work
- read:user:jira
Confirm that 7 scopes are selected, then save your changes.
Click Distribution, select Edit, and do the following:
- Select the Sharing radio button first to enable editing other fields.
- Fill out the remaining fields.
- Select Yes when asked: Does your app store personal data?
Select Settings to copy your Client ID and Client Secret.

Retrieve instance URL and instance ID

To get the instance URL:

Go to atlassian.net and sign in with your admin account.
Select the app you want to sync. For example, sync the first app.
Find the instance URL, which is the subdomain in the address bar. It will look like: Assign appropriate permissions.

To get the instance ID:

Open a new tab, copy the instance URL, and append /_edge/tenant_info to the instance URL. For example, https://YOUR-INSTANCE.atlassian.net/_edge/tenant_info.
Navigate to the link to find the cloudId value. The cloudId is your Instance ID.

Set up permissions and roles

Sign in to atlassian.com with your admin account.
Click the menu icon on the top left or go to admin.atlassian.com.
On the Admin page, click Manage users and go to the Groups page.
Click Create group. Enter a name for the group and create it.
In the Group product access section of your new group's page, click Add products to group.
For Jira, select User access admin as the product role.
For Jira Admin, select Product admin as the product role and save your changes.
On the Groups page, click Add group members and add users or accounts that the connector will authenticate as.

Create a Jira Cloud connector

Console

To use the Google Cloud console to sync data from Jira Cloud to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select Jira.
Enter your authentication information and click Authenticate.
A new window appears. Enter the instance username and password. Check that the authentication succeeded before returning to the Specify the Jira source for your data store page.
Select which entities to sync, then click Continue.
Select a region for your data store.
Enter a name for your data store.
Select a synchronization frequency.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data Stores page.
To check the status of your ingestion, go to the Data Stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
See Preview results for apps with third-party access control.

Connect Salesforce

Use the following procedure to sync data from Salesforce to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Have the following authentication information ready. For information about setting up consumer IDs and secrets in Salesforce, see Configure a Connected App for the OAuth 2.0 Client Credentials Flow in the Salesforce documentation.
- Instance URL
- Consumer ID
- Consumer Secret

The following limitation applies:

To sync a user as an entity, the user must provide sharing consent.

Console

To use the Google Cloud console to sync data from Salesforce to Vertex AI Search, follow these steps:

Add Google Cloud to your Salesforce CORS allowlist. If you have already done this, skip to the next step.
1. Follow the instructions in the Salesforce documentation to configure the CORS allowlist.
2. Enter https://console.cloud.google.com/ as an origin URL and save your configuration.
In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select Salesforce.
Enter your Salesforce authentication information.
Select which entities to sync and click Continue.
Select a region for your data store.
Enter a name for your data store.
Select a synchronization frequency.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data Stores page.
To check the status of your ingestion, go to the Data Stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
See Preview results for apps with third-party access control.

Connect ServiceNow

Use the following procedure to sync data from ServiceNow to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Have the following authentication information ready:
- Instance URL in the form of https://<domain-name>.service-now.com/.
- Client ID and client secret. For information about endpoint setup and getting the client ID and client secret, see Create an endpoint for clients to access the instance in the ServiceNow documentation.
- Username and password for one of the following ServiceNow role types:
  - Administrator role. See Base system roles in the ServiceNow documentation.
  - A custom ServiceNow role. This is an alternative to using an administrator role. To use a custom ServiceNow role, create access control rules with the following fields:
    View fields
    - Type: record
    - Operation: read
    - Name (a separate access control rule is required for each):
    - Requires role: Name of the custom role
    For more information, see Create a role and Create an ACL rule in the ServiceNow documentation.

Console

To use the Google Cloud console to sync data from ServiceNow to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select ServiceNow.
Enter your ServiceNow authentication information.
Select which entities to sync and click Continue.
Select a region for your data connector.
Enter a name for your data connector.
Select a synchronization frequency.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data Stores page.
To check the status of your ingestion, go to the Data Stores page and click your data connector name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
See Preview results for apps with third-party access control.

Connect Sharepoint Online

Use the following procedure to sync data from Sharepoint Online to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Grant administrator consent. For information about how to grant consent, see Grant tenant-wide admin consent to an application in the Microsoft documentation.
Prepare the following Sharepoint Online authentication information to use during setup:
- Instance URL. In the form http://DOMAIN_OR_SERVER/[sites/]WEBSITE. For more information about URLs, see URLs and tokens in SharePoint in the Sharepoint documentation.
- Tenant ID, client ID, and client secret. To register the application, select Accounts in this organizational directory only for the sign-in audience, and then locate this authentication information. For more information, see Quickstart: Register an application with the Microsoft identity platform in the Microsoft documentation.
- When registering the application, use https://vertexaisearch.cloud.google.com/console/oauth/confluence_oauth.html as the web callback URL.
- When configuring application permissions, add the following application permissions for Graph API:
  View scopes
  - Directory.Read.All
  - Files.Read.All
  - Group.Read.All
  - GroupMember.Read.All
  - IdentityProvider.Read.All
  - Sites.FullControl.All
  - User.Read.All
- When configuring application permissions, add the following delegated permissions for Sharepoint REST API:
  View scopes
  - AllSites.FullControl

The following table describes the roles that are recommended for configuration and their limitations.

View roles

Roles	Client application registration	Can fetch content of all entities	Can fetch ACL	Can fetch identity	Notes
Site Admins	No	Yes	Yes	Yes
Site Collection Administrator	No	Yes	Yes	Yes
Global Administrator	Yes	No	Yes	No	Limitation: Can't fetch identity or the comment and attachment entities
Site Owner	No	Yes	Yes	No	Limitation: Can't fetch identity
Application Administrator	Yes	N/A	N/A	N/A	The application administrator role is related to Microsoft 365 admin center / Entra admin center. It is not specific to Sharepoint.

Console

To use the Google Cloud console to sync data from Sharepoint Online to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select Sharepoint Online.
Enter your Sharepoint Online authentication information and click Authenticate.
A new window appears. Enter the instance username and password. Check that the authentication succeeded before returning to the Specify the Sharepoint Online source for your data store page.
Select the entities to sync and click Continue.
Select a region for your data store.
Enter a name for your data store.
Select a synchronization frequency for your data store.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data Stores page.
To check the status of your ingestion, go to the Data Stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
See Preview results for apps with third-party access control.

Connect Slack

Use the following procedure to sync data from Slack to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Have the following Slack authentication information ready:
- Workspace ID. For information about getting workspace IDs, see Specify the Slack source for your data store in the Slack documentation.
- Access token. For information about creating a client app and defining scopes, see Quickstart and How to quickly get and use a Slack API token in the Slack documentation.
- When setting OAuth 2.0 permission scopes, configure the following scopes:
  View scopes
  - users:read.email
  - channels:history
  - channels:read
  - groups:history
  - groups:read
  - im:history
  - im:read
  - mpim:history
  - mpim:read
  - team:read
  - users:read
  - files:read

The following limitation applies:

Slack's default behavior restricts the crawling and syncing of content from private channels, multi-party instant messages, and 1:1 instant messages.

Console

To use the Google Cloud console to sync data from Slack to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select Slack.
Enter your Slack authentication information.
Select which entities to sync and click Continue.
Select a region for your data store.
Enter a name for your data store.
Select a synchronization frequency for your data store.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data Stores page.
To check the status of your ingestion, go to the Data Stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
See Preview results for apps with third-party access control.

Connect Dropbox

Use the following procedure to sync data from Dropbox to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before you begin

Before setting up your connection:

Set up access control for your data source. For more information, see Use data source access control.
Have the following Dropbox authentication information ready. For information about setting up these parameters, see the OAuth Guide in the Dropbox documentation.
- Client ID
- Client secret

Console

To use the Google Cloud console to sync data from Dropbox to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data Stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select Dropbox.
Enter your Dropbox authentication information and click Authenticate. A new window appears.
Authenticate your account and confirm that it succeeded before returning to the Specify the Dropbox source for your data store page.
Select which entities to sync and click Continue.
Select a location for your data store.
Enter a name for your data store.
Select a synchronization frequency for your data store.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data stores page.
To check the status of your ingestion, go to the Data stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization. Check the Documents tab to make sure your entities have been ingested correctly.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
To preview how your search results appear after your app and data store are set up, see Get search results. If you used third-party access control, see Preview results for apps with third-party access control.

Connect Box

Use the following procedure to sync data from Box to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before you begin

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Have the following Box authentication information ready. For information about setting up these parameters, see JWT Auth in the Box developer documentation.
- Enterprise ID
- Client ID
- Client secret
- Public key ID
- Private key
- Passphrase
When creating the JWT endpoint, configure the following scopes:
View scopes
- root_readonly
- root_readwrite
- manage_managed_users
- manage_app_users
- manage_groups
- manage_enterprise_properties
- as_user
- generate_access_token

The following limitations apply:

Incremental changes for comments might take longer to sync than the configured frequency interval.
If a folder containing an entity is copied or moved, then incremental changes might take longer to sync than the configured frequency interval.

Console

To use the Google Cloud console to sync data from Box to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select Box.
Enter your authentication information.
Select which entities to sync and click Continue.
Select a region for your data store.
Enter a name for your data store.
Select a synchronization frequency for your data store.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data stores page.
To check the status of your ingestion, go to the Data stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
To preview how your search results appear after your app and data store are set up, see Get search results. If you used third-party access control, see Preview results for apps with third-party access control.

Connect OneDrive

Use the following procedure to sync data from OneDrive to Vertex AI Search.

After you set up your data source and import data the first time, the data store syncs data from that source at a frequency that you select during setup.

Before you begin

Before setting up your connection:

Set up access control for your data source. For information about setting up access control, see Use data source access control.
Have the following OneDrive authentication information ready:
- Client ID, client secret, and tenant ID. For information about setting up these parameters, see Quickstart: Register an application with the Microsoft identity platform in the Microsoft documentation.
- Specify scopes for access. An administrator role is required. For more information, see Quickstart: Configure a client application to access a web API in the Microsoft documentation.
  
  Configure the following scopes:
  View scopes
  - Files.Read.All Application
  - Group.Read.All Application
  - Sites.FullControl.All Application
  - User.Read Delegated
  - User.Read.All Application

Console

To use the Google Cloud console to sync data from OneDrive to Vertex AI Search, follow these steps:

In the Google Cloud console, go to the Agent Builder page.

Agent Builder
In the navigation menu, click Data stores.
Click Create data store.
On the Select a data source page, go to the Third-party sources section and select OneDrive.
Enter your OneDrive authentication information.
Select which entities to sync and cick Continue.
Select a region for your data store.
Enter a name for your data store.
Select a synchronization frequency for your data store.
Click Create. Vertex AI Search creates your data store and displays your data stores on the Data stores page.
To check the status of your ingestion, go to the Data stores page and click your data store name to see details about it on its Data page. The Connector state changes from Creating to Running when it starts synchronizing data. When ingestion is complete, the state changes to Active to indicate that the connection to your data source is set up and awaiting the next scheduled synchronization.

Depending on the size of your data, ingestion can take several minutes or several hours.

Next steps

To attach your data store to an app, create an app and select your data store following the steps in Create a search app.
To preview how your search results appear after your app and data store are set up, see Get search results. If you used third-party access control, see Preview results for apps with third-party access control.