This page describes how to view analytics for your media recommendations apps and search apps.
Analytics give you insight into the usage trends, search and recommendation quality, and end-user engagement of your app. The Vertex AI Agent Builder console provides an interactive dashboard experience powered by Looker.
- For media recommendations apps: Public preview. You can see per-recommendation metrics based on user events. You can filter metrics by date range and device type.
- For generic search and media search apps: You can see per-search metrics and per-search-session metrics based on search logs and user events. You can filter metrics by date range, query, and device type.
Metrics are refreshed about every 6 hours, so it can take several hours before you can use the Analytics page after you create your app.
Data sources for analytics
This section describes the data sources for analytics depending on app type.
You must complete user events integration to be able to see user event analytics.
Which user events you upload affect which analytics are generated.
Media recommendations apps require the following user events to view some metrics:
- View-item events. Required for click-through-rate.
- Media-complete events. Required for completion rate per play and completion rate per recommendation.
- Media-play events. Required for recommendation media play count, playthrough rate, completion rate per play, and average watch time per play.
Search apps require the following user events to view some metrics:
- Search events. Required for click-through-rate.
- View-item events. Required for click-through-rate.
The following information is also used for some analytics:
- Impressions. You can provide impressions (item views or clicks) that are
attributable to Vertex AI Search by including the associated item
in
UserEvent.Documentswhen ingesting user events. This information is needed for attribution and to compute per-search metrics. - Attribution tokens. Attribution tokens are unique IDs generated by
Vertex AI Search and returned with each search request. Make sure
to include that attribution token as
UserEvent.attributionTokenwith any user events resulting from a search. This is needed to identify if a search is served by the API. Only user events with a Google-generated attribution token are used to compute metrics. - User agent. Include
UserInfo.userAgentwith user events so that you can filter on user event metrics by device type.
Search logs
Search apps use search logs as data sources for some metrics.
Search logs are based on search requests. To allow Vertex AI Agent Builder to
identify user sessions so that it can deliver per-session analytics and higher
quality search results, Google strongly recommends that you set the optional
field SearchRequest.UserPseudoId.
If there are not yet any search logs or user events, default values are shown for all metrics.
View media recommendations analytics
Use the following instructions to view analytics for media recommendations.
In the Google Cloud console, go to the Agent Builder page.
Click the name of the app that you want to view analytics for.
Click Analytics.
To filter your metrics, specify any of the following filters and then click the Refresh button to apply them:
- Date range. Select a preset date range or, if available, enter a custom date range.
- Device type. Select a device type that queries occurred on.
View search analytics
Use the following instructions to view analytics for generic search or media search apps.
In the Google Cloud console, go to the Agent Builder page.
Click the name of the app that you want to view analytics for.
Click Analytics.
Click a tab to view that metric group:
- Per Search. Metrics are grouped by searches.
- Per Session. Metrics are grouped by search sessions.
- Compare. Baseline metrics are shown alongside metrics from a comparison period. Select date ranges for the baseline metrics (an earlier time range) and comparison metrics (a later time range). These time ranges cannot overlap.
To filter your metrics, specify any of the following filters and then click the Refresh button to apply them:
- Date range. Select a preset date range or, if available, enter a custom date range.
- Search query. Select a condition and search query value. You can add multiple search query filters. Available for Per Search and Compare metrics.
- Device type. Select a device type that queries occurred on.
Metrics definitions
The following tables describe how metrics are defined.
Media recommendations metrics
| Metric name | Metric definition | Notes |
|---|---|---|
| Recommendation count | Total number of recommendations | |
| Recommendation play count | Number of times that recommended media items were played | |
| Recommendation click count | Number of times that recommended media items were clicked | |
| Click-through rate | Count of recommendation clicks / recommendation count | Recommendation clicks are view-item events that can be attributed to a recommendation |
| Recommendation media complete count | Number of times that recommended media items were played to completion | |
| Playthrough rate per recommendation | Number of times that recommended media items were played / recommendation count | |
| Completion rate per play | Number of times that recommended media items were played to completion / number of times that media items were played | |
| Average watch time per play | Time in seconds that recommended media items were played / number of times that recommended media items were played | |
| Completion rate per recommendation | Number of times that media items were played to completion / recommendation count | |
| Average watch time per recommendation | Time in seconds that recommended media items were watched / recommendation count |
Search metrics
| Metric name | Metrics Definition | Notes |
|---|---|---|
| Search count | Count of search events | Based on search logs |
| No result rate | Count of search events without results / Search count | Based on search logs |
| Click-through rate (CTR) per search | Count of search clicks / Search count | Search clicks are view-item events that can be attributed to a previous search event. Based on user events |
| Search session count | Count of search sessions | A search session is a user session containing at least one search event. A user session, also called a visit, is a continuous set of user events. When there are 30 minutes of inactivity, the session ends. Based on search logs |
| Page view per search visit | Count of view-item events in search sessions / Search session count | This metric includes all page views in search visits regardless of whether they can be attributed to Vertex AI Search. Based on user events |
| Bounce rate | Count of bounces in search sessions / Search session count | A search session bounce is defined as a session with a single search, where the user left after only making one search. Based on user events |