This page describes how to import user event data from past events in bulk. User events are required for media recommendations. If you don't use media recommendations, importing user events isn't required. However, it is recommended for media search apps.
For the user event types that you can import, see the eventType
field of the
userEvents
object. By importing user events, you can improve the
quality of your recommendations as well as the ranking of your search results.
Search results with higher click-through rates are boosted, while results with
lower click-through rates are buried. Don't import user events for documents
that you have not yet imported.
If an event has documents listed in userEvents.Documents
, then
Vertex AI Search automatically joins the events with that document.
Joining allows Vertex AI Search to attribute events such as clicks and
views to the correct document in a search result or recommendation. For generic
use case apps, joining occurs asynchronously, that is, user events are joined
to documents after the batch import is complete (generally happening within
minutes). For media apps, joining occurs synchronously,
Vertex AI Search starts joining user events to documents while
the batch import is still in progress. For how to view unjoined event metrics in
the console, see
View aggregated user event information.
To import user events in real-time, see Record real-time user events.
You can import historical events in the following ways:
For the JSON representation for user events, see
userEvents
in the API reference.
Before you begin
Before you import user events:
- Create a data store and an app.
- Review About user events for user event requirements.
Import historical user events from Cloud Storage
If you have a media app, you can import user events using the Google Cloud console or the API. For other app types, you can only import using the API.
Console
To import user events for media apps using the console, follow these steps:
In the Google Cloud console, go to the Agent Builder page.
On the Apps page, click your media app.
Go to the Data page.
For media recommendations apps, the Requirements tab displays the media documents and user events that you need to import.
Click the Events tab. If you have already imported any events, information about them is displayed on this tab.
Click Import events.
Select Cloud Storage as your data source.
Enter or select the Cloud Storage location of your user events.
Click Import.
The Activity tab displays the status of your import.
If import errors occur, in the Activity tab:
Expand the error in the Details column and click View details to see more information in the Activity log details pane.
In the Activity log details pane, click View full error logs to see the error in Logs Explorer.
REST
To import historical user events in bulk from Cloud Storage, follow these steps:
Create one or more data files for the input parameters for the import. Use the
gcsSource
object to point to your Cloud Storage bucket.{ "gcsSource": { "inputUris": ["INPUT_FILE_1", "INPUT_FILE_2"], "dataSchema": "user_event" }, "errorConfig":{ "gcsPrefix":"ERROR_DIRECTORY" } }
- INPUT_FILE: A file in Cloud Storage containing your user
event data. Make sure each user event is on its own single line, with
no line breaks. For the JSON representation for user events, see
userEvents
in the API reference. The input file fields must be in this format:gs://<bucket>/<path-to-file>/
. - ERROR_DIRECTORY: Optional. A Cloud Storage directory for
error information about the import—for example,
gs://<your-gcs-bucket>/directory/import_errors
. Google recommends leaving this field empty to let Vertex AI Agent Builder automatically create a temporary directory.
- INPUT_FILE: A file in Cloud Storage containing your user
event data. Make sure each user event is on its own single line, with
no line breaks. For the JSON representation for user events, see
Import your events by making a POST request to the
userEvents:import
method, providing the name of the data file.export GOOGLE_APPLICATION_CREDENTIALS=/tmp/my-key.json curl -X POST \ -v \ -H "Content-Type: application/json; charset=utf-8" \ -H "Authorization: Bearer "$(gcloud auth print-access-token)"" \ --data @DATA_FILE.json \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_NUMBER/locations/global/dataStores/DATA_STORE_ID/userEvents:import"
Import historical user events from BigQuery
When creating tables for user events in BigQuery, use the schemas documented in About user events.
Set up BigQuery access
If your BigQuery dataset is in a different project than your data store, follow these steps to set up access to BigQuery.
- Open the IAM & Admin page in the Google Cloud console.
- Select the project that contains the data store that you want to import user events into.
- Select the Include Google-provided role grants checkbox.
- Find the service account with the name Discovery Engine Service Account.
- If you have not previously initiated an import operation with Discovery Engine, this service account might not be listed. If you don't see this service account, return to the import task and initiate the import. When it fails due to permission errors, return here and complete this task. The Discovery Engine service account will be listed.
- Copy the identifier for the service account, which looks like an email
address—for example,
service-525@gcp-sa-discoveryengine.iam.gserviceaccount.com
. - Switch to your BigQuery project (on the same IAM & Admin page) and click Grant Access.
- For New principals, enter the identifier for the service account and select the BigQuery > BigQuery Data Viewer role.
- Click Save.
For more information about BigQuery access, see Controlling access to datasets in the BigQuery documentation.
Import events from BigQuery
If you have a media app, you can import user events using the Google Cloud console or the API. For other app types, you can only import using the API.
Console
To import user events for media apps using the console, follow these steps:
In the Google Cloud console, go to the Agent Builder page.
On the Apps page, click your media app.
Go to the Data page.
For media recommendations apps, the Requirements tab displays the media documents and user events that you need to import.
Click the Events tab. If you have already imported any events, information about them is displayed on this tab.
Click Import events.
Select BigQuery as your data source.
Enter or select the BigQuery path for your user events.
Click Import.
The Activity tab displays the status of your import.
If import errors occur, in the Activity tab:
Expand the error in the Details column and click View details to see more information in the Activity log details pane.
In the Activity log details pane, click View full error logs to see the error in Logs Explorer.
REST
To import historical user events in bulk from Cloud Storage, follow these steps:
Import your user events by making a POST request to the
userEvents:import
method, providing the name of the BigQuery project, dataset ID, and table ID.When importing your events, use the value
user_event
fordataSchema
.export GOOGLE_APPLICATION_CREDENTIALS=/tmp/my-key.json curl \ -v \ -X POST \ -H "Content-Type: application/json; charset=utf-8" \ -H "Authorization: Bearer "$(gcloud auth print-access-token)"" \ "https://discoveryengine.googleapis.com/v1beta/projects/[PROJECT_NUMBER]/locations/global/dataStores/DATA_STORE_ID/userEvents:import" \ --data '{ "bigquerySource": { "projectId":"PROJECT_ID", "datasetId": "DATASET_ID", "tableId": "TABLE_ID", "dataSchema": "user_event" } }'
Import historical user events as local JSON data
You can import user events by including local JSON data for the
events in your call to the userEvents:import
method.
To put user event data into a JSON file and specify the file in an API request, follow these instructions:
Create a JSON file containing your user event data. For the JSON representation for user events, see
userEvents
in the API reference documentation.{ "inlineSource": { "userEvents": [ { USER_EVENT_1 }, { USER_EVENT_2 } ] } }
Import your events by making a POST request to the
userEvents:import
method, providing the name of the data file.curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json; charset=utf-8" \ --data @[JSON_FILE] \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_NUMBER/locations/global/dataStores/DATA_STORE_ID/userEvents:import"