View analytics

This page describes how to view analytics for your media recommendations apps and search apps.

Analytics give you insight into the usage trends, search and recommendation quality, and end-user engagement of your app. The Vertex AI Agent Builder console provides an interactive dashboard experience powered by Looker.

• For media recommendations apps: Public preview. You can see per-recommendation metrics based on user events. You can filter metrics by date range and device type.
• For generic search and media search apps: You can see per-search metrics and per-search-session metrics based on search logs and user events. You can filter metrics by date range, query, and device type.

Metrics are refreshed about every 6 hours, so it can take several hours before you can use the Analytics page after you create your app.

Data sources for analytics

This section describes the data sources for analytics depending on app type.

You must complete user events integration to be able to see user event analytics.

Which user events you upload affect which analytics are generated.

Media recommendations apps require the following user events to view some metrics:

• View-item events. Required for click-through-rate.
• Media-complete events. Required for completion rate per play and completion rate per recommendation.
• Media-play events. Required for recommendation media play count, playthrough rate, completion rate per play, and average watch time per play.

Search apps require the following user events to view some metrics:

• Search events. Required for click-through-rate.
• View-item events. Required for click-through-rate.

The following information is also used for some analytics:

• Impressions. You can provide impressions (item views or clicks) that are attributable to Vertex AI Search by including the associated item in UserEvent.Documents when ingesting user events. This information is needed for attribution and to compute per-search metrics.
• Attribution tokens. Attribution tokens are unique IDs generated by Vertex AI Search and returned with each search request. Make sure to include that attribution token as UserEvent.attributionToken with any user events resulting from a search. This is needed to identify if a search is served by the API. Only user events with a Google-generated attribution token are used to compute metrics.
• User agent. Include UserInfo.userAgent with user events so that you can filter on user event metrics by device type.

Search logs

Search apps use search logs as data sources for some metrics.

Search logs are based on search requests. To allow Vertex AI Agent Builder to identify user sessions so that it can deliver per-session analytics and higher quality search results, Google strongly recommends that you set the optional field SearchRequest.UserPseudoId.

If there are not yet any search logs or user events, default values are shown for all metrics.

View media recommendations analytics

Note: Media recommendations analytics are in Public preview.

Use the following instructions to view analytics for media recommendations.

1. In the Google Cloud console, go to the Agent Builder page.

   Agent Builder

2. Click the name of the app that you want to view analytics for.

3. Click Analytics.

4. To filter your metrics, specify any of the following filters and then click the Refresh button to apply them:

   • Date range. Select a preset date range or, if available, enter a custom date range.
   • Device type. Select a device type that queries occurred on.

View search analytics

Notes:

• For data stores in US and EU multi-regions, viewing search analytics is in Public preview.
• Search analytics are GA for global data stores.
• Metrics from the widget don't appear on the Analytics page.

Use the following instructions to view analytics for generic search or media search apps.

1. In the Google Cloud console, go to the Agent Builder page.

   Agent Builder

2. Click the name of the app that you want to view analytics for.

3. Click Analytics.

4. Click a tab to view that metric group:

   • Per Search. Metrics are grouped by searches.
   • Per Session. Metrics are grouped by search sessions.
   • Compare. Baseline metrics are shown alongside metrics from a comparison period. Select date ranges for the baseline metrics (an earlier time range) and comparison metrics (a later time range). These time ranges cannot overlap.

5. To filter your metrics, specify any of the following filters and then click the Refresh button to apply them:

   • Date range. Select a preset date range or, if available, enter a custom date range.
   • Search query. Select a condition and search query value. You can add multiple search query filters. Available for Per Search and Compare metrics.
   • Device type. Select a device type that queries occurred on.

Metrics definitions

The following tables describe how metrics are defined.

Media recommendations metrics

Metric name	Metric definition	Notes
Recommendation count	Total number of recommendations
Recommendation play count	Number of times that recommended media items were played
Recommendation click count	Number of times that recommended media items were clicked
Click-through rate	Count of recommendation clicks / recommendation count	Recommendation clicks are view-item events that can be attributed to a recommendation
Recommendation media complete count	Number of times that recommended media items were played to completion
Playthrough rate per recommendation	Number of times that recommended media items were played / recommendation count
Completion rate per play	Number of times that recommended media items were played to completion / number of times that media items were played
Average watch time per play	Time in seconds that recommended media items were played / number of times that recommended media items were played
Completion rate per recommendation	Number of times that media items were played to completion / recommendation count
Average watch time per recommendation	Time in seconds that recommended media items were watched / recommendation count

Search metrics

Metric name	Metrics Definition	Notes
Search count	Count of search events	Based on search logs
No result rate	Count of search events without results / Search count	Based on search logs
Click-through rate (CTR) per search	Count of search clicks / Search count	Search clicks are view-item events that can be attributed to a previous search event. Based on user events
Search session count	Count of search sessions	A search session is a user session containing at least one search event. A user session, also called a visit, is a continuous set of user events. When there are 30 minutes of inactivity, the session ends. Based on search logs
Page view per search visit	Count of view-item events in search sessions / Search session count	This metric includes all page views in search visits regardless of whether they can be attributed to Vertex AI Search. Based on user events
Bounce rate	Count of bounces in search sessions / Search session count	A search session bounce is defined as a session with a single search, where the user left after only making one search. Based on user events