This is the documentation for Recommendations AI only. To try Retail Search and the unified Retail console in the restricted GA phase, contact Cloud sales. If you are not planning to use Retail Search, remain on the Recommendations console until further notice.

If you are using the v1beta version of Recommendations AI, migrate to the Retail API version.

Recording real-time user events

This page describes how you record real-time user events. Recommendations AI uses real-time user events to generate next-item recommendations. Recording as many types of user events as possible with valid product information increases the recommendation quality.

For details about user events, including user event types and sample JSON for all types, see User events.

You can record a user event in multiple ways:

You can find examples of recording user events of type detail-page-view for all of these methods below. For other event types, see User events.

You can also import historical user events. It can take a considerable amount of time to record sufficient user event data to train your models. You can accelerate initial model training by importing user event data from past events in bulk. See Importing historical user events.

If the user event you are recording is the first time a user is interacting with a product based on a previously provided recommendation, including a attribution token enables Recommendations AI to provide recommendation performance metrics, but the attribution token is optional. For information about how to work with attribution tokens, see Attribution tokens.

For more information about the userInfo field, see About the user information object.

Before you begin

Before recording user events, you should have:

  • A Google Cloud project created, with authentication set up.

  • A valid API key (for JavaScript Pixel or Tag Manager), or a valid service account with the Retail Editor Role assigned if using the API to write directly.

    For more information, see Before you begin.

Best practices for recording user events

Recommendations AI requires high-quality data to make high-quality predictions. If your data is incomplete or incorrect, the quality of your predictions suffers.

When you record user events, ensure that you implement the following best practices:

  • If you record user events before or while importing your catalog, rejoin any events recorded before the catalog import completed.

    You can import the catalog to Recommendations AI before, after, or at the same time you record user events. Doing these tasks in parallel can save time if the catalog is large and there are many user events. Once the catalog import is complete, you must use the API to rejoin events that were uploaded before the import completed.

    Recommendations AI attempts to join recorded user events with metadata from the product catalog when the user event is created. Only successfully joined events are used for training, so make sure to rejoin any events recorded before the catalog was completely imported. If an event refers to an item that doesn't exist in the catalog, it is discarded or not associated with the correct products. Similarly, if you import user events from the past, the catalog must include any products they reference; you can mark older products as OUT_OF_STOCK rather than removing them from the catalog.

  • Keep your catalog up to date.

    When you record user events, the catalog item included in the user event is connected with your current catalog. If you record an event for a catalog item that is not in the current catalog, it cannot be used by Recommendations AI for training your models. This is called an "unjoined" event. If you recorded events before your catalog was completely imported, you must rejoin the events that were recorded during the import. Having a few unjoined events is expected. However, if the percentage of unjoined events reaches 5% or more of your total user events, make sure your catalog is up to date, rejoin events that were recorded before the catalog was fully updated, and investigate why the unjoined events are being created.

    You can see your unjoined events by using event filtering. Learn more.

  • Make sure that you provide as much information with your user events as possible.

    Each user event type has different information that is required and accepted. For more information, see User events.

  • Set up alerts so that you will know if your user event recording processes experience any outages.

  • For a bulk user event import, limit the size of the data you are importing.

    A bulk user event import can take up to 24 hours to complete.

    The size of each file must be 2 GB or smaller. You can include at most 100 files in a single import request. One approach is import only the user events for one day at a time.

  • After a bulk import, review your error reporting to ensure that your data was imported correctly.

  • When importing user event data, include an accurate timestamp for each user event and avoid importing sequential user events with identical timestamps.

    Provide the timestamp in the eventTime field in the format specified by RFC 3339.

  • If you have imported user events that are incorrect, talk to your Recommendations AI contact about how to correct the problem.

  • When possible, keep your user event data continuous.

    Gaps in user event data can reduce model quality.

  • Use a secure form of a unique identifier to keep users anonymous to Recommendations AI and protect your users' privacy. You are responsible for redacting PII (personally identifiable information), such as email or home addresses, from your data.

Recording user events with a JavaScript pixel

The following example records a detail-page-view UserEvent using a JavaScript pixel.

<script type="text/javascript">
var user_event = {
  "eventType" : "detail-page-view",
  "visitorId": "visitor-id",
  "userInfo": {
      "userId": "user-id"
  },
  "attributionToken": "attribution-token",
  "experimentIds": "experiment-id",
  "productDetails": [
      {
        "product": {"id": "123"}
      }
  ]
};

var _gre = _gre || [];
// Credentials for project.
_gre.push(['apiKey', 'api-key']);
_gre.push(['logEvent', user_event]);
_gre.push(['projectId', 'project-id']);
_gre.push(['locationId', 'global']);
_gre.push(['catalogId', 'default_catalog']);

(function() {
  var gre = document.createElement('script'); gre.type = 'text/javascript'; gre.async = true;
  gre.src = 'https://www.gstatic.com/retail/v2_event.js';
  var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(gre, s);
})();

</script>

If you imported user events with Google Analytics 360, set visitorID to the Google Analytics client ID. Note that the Google Analytics client ID is only part of the full _ga cookie name (for example, client ID 123456789.123456789 is part of _ga cookie GA1.3.123456789.123456789).

For more about getting the client ID, see the Google Analytics documentation.

The following is an abbreviated example that shows the format for setting the client ID in a user event. Replace "UA-XXXXXX-N" with your Google Analytics tracking ID.

<script type="text/javascript">
var tracker = ga.getByName('UA-XXXXXX-N');
var user_event = {
      "visitorId": tracker.get('clientId')
};
</script>

Recording user events with the userEvents.write method

You can use the userEvents.write method to send user events directly to the API from your back-end server.

To record user events, send a POST request to the userEvents.write method and provide the appropriate request body.

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     --data "{
         'eventType': 'detail-page-view',
         'visitorId': 'visitor0',
         'eventTime': '2020-01-01T03:33:33.000001Z',
         'experimentIds': '['321'],
         'attributionToken': 'ABC',
         'attributes': {
            'example_text_attribute': {
              'text': ['text_1', 'text_2']
            },
            'example_number_attribute': {
               'numbers': [3.14, 42, 1.2345]
            }
         },
         'productDetails': [{
           'product': {
             'id': 'abc'
           }
          }],
         'userInfo': {
           'userId': 'abc@example.com',
           'ipAddress': '8.8.8.8',
           'userAgent': 'Mozilla/5.0',
           'directUserRequest': true
         },
         'uri': 'http://example',
         'referrerUri': 'http://example',
         'pageViewId': 'currentPageUri'
}"
"https://retail.googleapis.com/v2/projects/PROJECT_ID/locations/global/catalogs/default_catalog/userEvents:write"

Recording user events with Google Tag Manager

Tag Manager provides a way to manage and test multiple tags without many server-side code changes to your site.

Some decisions you make during setup depend on whether you are using Google Analytics and Enhanced Ecommerce. Neither is required; if you don't use them, you can either configure an ecommerce variable when creating the Cloud Retail tag, or manually populate your site's data layer code after creating the tag.

Enhanced Ecommerce is an additional configuration for Google Analytics that passes product titles, IDs, prices, transaction details, and other structured ecommerce data to Google Analytics. Recommendations AI can automatically use the Enhanced Ecommerce data layer, so if you have that set up already, configuration can be easier. If you don't have Enhanced Ecommerce configured for Google Analytics but want to use it, see more details and setup instructions in the Enhanced Ecommerce developer guide.

Use this one-time procedure to set up a Cloud Retail tag in Tag Manager for recording user events.

Creating a Google Tag Manager tag

Set up a tag in Tag Manager to send user event information to the Retail API.

  1. Log in to Tag Manager and select the container for your site.

  2. Go to the Tags tab and click New to add a new tag.

  3. Give your tag a name at the top of the panel (the placeholder is Untitled Variable), such as "Recommendations AI".

  4. Click Tag Configuration and choose the Cloud Retail tag to open the tag configuration panel.

  5. Enter your API key.

    Use the key you created when setting up Recommendations AI.

    Your API keys are available from the APIs & Services > Credentials page in Cloud Console. Do not use any API key that you registered with Recommendations AI for prediction calls.

  6. Enter the project number of the Google Cloud project where Recommendations AI is enabled.

    The project number is available from your Cloud Console dashboard.

  7. For the User Event Data Source field:

    • Ecommerce Data Layer: Select if you have Enhanced Ecommerce implemented via Tag Manager. This enables you to reuse the Enhanced Ecommerce data layer as your events data source instead of populating a new one.

      You can only record add-to-cart, purchase-complete, and detail-page-view events via this data source.

    • Cloud Retail Data Layer: Select if you are not using Google Analytics, and can manually populate data layer code. See the Tag Manager Developer Guide.

    • Cloud Retail User Variable: Select to populate a Tag Manager variable with the required fields for Recommendations AI. You might choose this option if you aren't using Enhanced Ecommerce, or if Enhanced Ecommerce doesn't have the data needed for Recommendations AI.

    • Ecommerce Variable: Select if you aren't using Enhanced Ecommerce in your data layer, and can't manually populate the data layer code.

      In the Read Ecommerce Data from User Variable field that appears, choose a variable. This enables Recommendations AI to read Enhanced Ecommerce user event data from a custom variable that you create.

      The variable should match the format documented in the Enhanced Ecommerce Developer Guide. To construct a variable in the correct format, you can use Enhanced Ecommerce Object Builder, a custom variable template from the Tag Manager community template gallery. Community templates are not maintained by Google. To use this template, see the Enhanced Ecommerce Object Builder gallery page for documentation and other resources.

  8. Click Save.

    Your Cloud Retail tag is created.

Next, create event triggers for your tag.

Creating event triggers for your Tag Manager tag

Create triggers for all user event types your Recommendations AI models will use.

Tag Manager tags must have triggers that control when the tag should be "fired" on the site. Triggers listen for when events occur (such as a user viewing the home page or adding an item to their cart) and prompts your tag to send that user event information to the Retail API.

Tag Manager provides some standard triggers. For example, Window Loaded is a trigger for detail-page-view events. For details about each type, see Trigger types in the Tag Manager documentation.

You'll usually set the tag to trigger when a user views any page that has the events needed for Recommendations AI (such as the home page, product detail pages, cart pages, or checkout complete page). In these cases, the tag should fire after the page has loaded, so that cookies are available and all data layer variables are populated. To accomplish this, set your triggers to fire on Window Loaded or DOM Ready.

You might need to fire the tag when an action is performed rather than at page load (such as if a user adding an item to a cart doesn't force a page to reload). In these cases, you can configure that click action on your site to simultaneously push updates to the data layer and associate the trigger with that action.

For example, if you created a trigger for add-to-cart events, you could choose the trigger type to Click - Just Links and set it to fire on the click ID (in this example, addtocart). You would then configure the addtocart link on your site to also update the data layer with new values when clicked:

  <a id="addtocart" href="javascript:void(0);"
         onclick="dataLayer.push({
                  'cloud_retail': {
                  'eventType': 'add-to-cart',
                  'visitorId': '456',
                  'cartId': 'mobile',
                  'productDetails': [{
                  'product': {
                  'id': '54321'
                  },
                  'quantity': 1
                  }]}});">Add to Cart</a>

For some user events, you must create a custom trigger. Generally, you create a custom trigger in Tag Manager using the user event name. If you can't modify your front-end code, you can create a custom trigger using JavaScript macros. For more information about custom triggers, see Custom event trigger.

Use the following procedures to create triggers in Tag Manager:

Creating new triggers for your Tag Manager tag

If you don't use Enhanced Ecommerce, create new event triggers for any user events your Recommendations AI models need. Then, associate your new triggers with the Cloud Retail tag you created in Tag Manager.

Before you start the following steps, make sure you have created a Cloud Retail tag in Tag Manager. See Creating a Tag Manager tag.

First, create the triggers. Repeat this procedure for all user events your Recommendations AI models require:

  1. On the Tag Manager, Triggers page, click New > Trigger Configuration.

  2. Choose the trigger type that applies to the user event you're creating a trigger for.

  3. Save your trigger.

Next, associate your new triggers with your Cloud Retail tag. This is a one-time procedure:

  1. On the Tag Manager, Tags page, click your Cloud Retail tag to edit it.

  2. Click Triggering, select your new triggers, and click Add.

  3. Save your tag.

Next, set up a variable that sets the Recommendations AI visitorId field to the user session ID.

Reuse Enhanced Ecommerce triggers

If you have implemented Enhanced Ecommerce via Tag Manager, reuse event triggers from your Enhanced Ecommerce for Recommendations AI.

You can only record add-to-cart, purchase-complete, and detail-page-view events via this data source.

Before you start these steps, make sure you have first:

  • Set up a tag in Tag Manager of tag type Google Analytics - Universal Analytics and enabled Enhanced Ecommerce on it. See the Tag Manager documentation and the Enhanced Ecommerce Developer Guide for details.
  • Configured your Enhanced Ecommerce tag in Tag Manager to trigger on the user events you plan to record for Recommendations AI.
  • Created a Cloud Retail tag in Tag Manager, with Enhanced Ecommerce as the user event data source (see Creating a Tag Manager tag.

To reuse Enhanced Ecommerce triggers:

  1. On the Tag Manager, Tags page, click your Enhanced Ecommerce tag (tag type Google Analytics - Universal Analytics) to edit it.

  2. Under Advanced Settings > Tag Sequencing, select Fire a tag after <Enhanced Ecommerce tag name> fires.

  3. Select your Cloud Retail tag as the Cleanup Tag.

    Select Don't fire <Cloud Retail tag name> if <Enhanced Ecommerce tag name> fails or is paused.

  4. Save your tag.

To finalize your tag setup, overwrite the visitorID field with a variable in Tag Manager so that Recommendations AI can use session IDs to track users.

Setting the visitorID field in Tag Manager

Recommendations AI uses the value visitorId to track users. visitorId is typically a session ID, and is required for all events. Set up a variable that sets session IDs as visitorId.

If you're using Google Analytics, you can use the Google Analytics visitor ID. To configure this, use the following procedure to override the visitor ID value for the Cloud Retail tag. This maps the first-party cookie "_ga" to a Tag Manager variable called "GA visitorId". You can do the same for any session ID cookie; it doesn't have to be from Google Analytics.

This procedure assumes you are using Google Analytics. If you aren't, you can use another cookie or variable, or get the visitor ID from the cloud_retail data layer.

Before starting these steps, make sure you have already created a Cloud Retail tag in Tag Manager (see Creating a Google Tag Manager tag).

To set the visitorID value to a variable for the Cloud Retail tag:

  1. In Tag Manager, go to the Variables tab and click New to create a new user-defined variable.

  2. Give the variable a name at the top of the dialog, such as "GA visitorId".

  3. Enter your variable settings:

    Google Analytics 360

    1. Set Variable Type to Custom JavaScript.

    2. Enter the following script in the Custom JavaScript field.

      Replace "UA-XXXXXX-N" with your Google Analytics tracking ID. To find your tracking ID, see What happened to my Tracking ID? For more about getting the client ID, see the Google Analytics documentation.

      function() {
       var tracker = ga.getByName('UA-XXXXXX-N');
       return tracker.get('clientID');
      }
      
    3. Click Save to save the variable.

    Google Analytics

    1. Choose 1st Party Cookie as the variable type.

    2. In the Cookie Name field, enter _ga.

    3. Click Format Value, select Convert undefined to.., and enter "" (an empty string).

    4. Click Save to save the variable.

      This maps the first party cookie "_ga" to a Tag Manager variable called "GA visitorId".

  4. Go to the Tags tab and click your Cloud Retail tag to edit it.

  5. Click Tag Configuration and, under Fields to Overwrite, click the + Overwrite a value on the UserEvent message button.

  6. Select visitorId as the field, and the variable you just created as the field value.

  7. Save your Cloud Retail tag.

Next, preview your tag, and set up monitoring of event recording errors and other issues to make sure Recommendations AI keeps receiving data successfully.

If you're using cloud_retail data layer as your user event source, make sure to also set up your data layer.

Using the cloud_retail data layer with Tag Manager

If you created your Cloud Retail tag in Tag Manager to use the cloud_retail data layer as the user event source, set up the dataLayer variable in your source HTML as described in the Tag Manager Developer Guide.

About the data layer

Most Tag Manager tags require data that changes depending on the user or the page (such as user IDs or product IDs). For the Cloud Retail tag, that data must be exposed in a structured way via a data layer so that Tag Manager can use it.

The data layer is a JavaScript object that is typically added to a page using either server-side code, or in the front end using HTML or a template. If a page is configured with the data layer, it will contain some code like the following:

dataLayer = dataLayer || [];
dataLayer.push({
  'cloud_retail': {
    'eventType': 'home-page-view',
    'visitorId': 'visitor_a',

    'userInfo': {
      'userId': '789'
    },
  }
});

This code creates a dataLayer object and assigns the cloud_retail structure to it as an array element.

Required fields in the cloud_retail data layer

User events lists all required fields and examples for event types that should be passed to the cloud_retail data layer.

Your server-side code or templates should have the correct script tags on each page that you want to send events from. After the dataLayer object is populated correctly on each page, you should be able to test the Cloud Retail tag.

Some fields like visitorId are required for the UserEvent message, but might not be available when populating the data layer. For example, visitorId might be derived from the user's cookie, or experimentIds from the A/B experimentation framework. In this case, use a variable to overwrite the field on the Tag Manager tag.

You can overwrite the following fields:

  • visitorId
  • userInfo.userId
  • attributionToken
  • experimentIds

For how to overwrite a UserEvent field in Tag Manager, see Setting the visitorID field in Tag Manager, which walks through overwriting the visitorId field value with a user-defined variable.

The following example shows the data layer that needs to be included in your page for a detail-page-view UserEvent using Tag Manager:

<script>
  dataLayer = dataLayer || [];
  dataLayer.push({
    'cloud_retail': {
      'eventType' : 'detail-page-view',
      'visitorId': 'visitor_a',
      'userInfo': {
          // The user and visitor ID fields can typically be
          // be populated from a client-side JavaScript
          // variable such as a cookie. If you set the user
          // and/or visitor ID values from the server,
          // populate the `userID`.
          'userId': 'user_a'
      },
      'attributionToken': 'attribution-token',
      // In most cases, the experiment ID field is populated from a
      // client side JavaScript variable as defined by the experiment
      // manager.
      // If you set the experiment ID value from the server,
      // populate the `experimentIds` field here.
      'productDetails': [
            {
              'product': {'id': '123'}
            }
      ],
    // You can use the 'cloud_retail' data layer element along with other
    // data layer elements.
    'ecommerce': {
      ...
    },
  }];
</script>

Previewing your Tag Manager tag

Tag Manager's Preview Mode allows you to test new tags before publishing them to your live site.

For more details about Preview Mode, see the Tag Manager documentation for Preview Mode.

Use the following procedure to confirm that your tag is firing correctly.

  1. On the Tag Manager overview page, click Preview.

    Tag Manager Preview Mode opens in a new browser tab.

  2. Enter your site information and click Start to start Tag Assistant.

    In the current browser tab, Tag Assistant starts, and your site opens in a new tab.

  3. On your site, visit any page where the Cloud Retail tag should be triggered.

  4. Confirm that the Tag Assistant lists the Cloud Retail tag in the Tags tab under the Tags Fired section.

  5. In the Tag Assistant, go to the Data Layer tab and check that the correct values from the cloud_retail or ecommerce data layer are displayed.

Checking for tag errors

If some fields are incorrect or missing when you preview your tag, the tag typically also returns an error, unless it isn't firing at all.

You can check the Errors page in Google Cloud Console short for errors. This page logs most errors except for syntax errors, which typically only appear in request results.

You can use the following steps to use Chrome DevTools to check for any errors generated, including syntax errors.

  1. Turn on Preview Mode in Tag Manager for your site in a Chrome browser and visit any page where the Cloud Retail tag should be triggered.

  2. With Preview Mode open, open DevTools and click the Network tab.

  3. Reload the page.

  4. In DevTools, search for userEvent.

    The Network tab displays the userEvent:collect event and its status code.

    • A 200 response indicates your tag is in a good state.
    • Other responses, such as a 400 error and highlighting the event in red, indicate that debugging is necessary.
  5. Double click the event name to execute the request and show a full response with more error information.

    For example, you might see a 400 error containing the message, "'visitorId' is required, and cannot be empty", indicating that visitorId was not set correctly.

  6. If no userEvent is fired, check the DevTools Console tab for syntax errors in the data layer.

Monitoring import health

Recording user events successfully is important for getting high-quality recommendations. You should monitor the event recording error rates and take action if needed. For more information, see Setting up alerts for data upload issues.

What's next