This is the unified documentation for Retail API. This includes Recommendations AI, Retail Search, and the unified Retail console (which is applicable to both Recommendations AI and Retail Search users). To use the new console or Retail Search while they are in the restricted GA phase, submit a form here to contact Cloud sales. If you are using the v1beta version of Recommendations AI, migrate to the GA version: Migrating to the Retail API from beta.

To see documentation for only Recommendations AI and the Recommendations AI-only console, go to the How-to guides for Recommendations AI and the API reference documentation for Recommendations AI.

Recording real-time user events

This page describes how you record real-time user events. The Retail API uses real-time user events to generate recommendations and search results. Recording as many types of user events as possible with valid product information increases the quality of your results.

The recording procedures on this page apply to both Recommendations AI and Retail Search. Once you record data to the Retail API, both services are able to use those events, so you don't need to upload the same data twice if you use both services.

For details about user events, including user event types and sample JSON for all types, see User events.

You can record a user event in multiple ways:

You can find examples of recording user events of type detail-page-view for all of these methods below. For other event types, see User events.

You can also import historical user events. It can take a considerable amount of time to record sufficient user event data to train your models. You can accelerate initial model training by importing user event data from past events in bulk. See Importing historical user events.

If the user event you are recording is the first time a user is interacting with a product based on a previously provided recommendation or search results, including an attribution token enables the Retail API to provide performance metrics. Including an attribution token is optional but highly recommended. For information about how to work with attribution tokens, see Attribution tokens.

For more information about about visitor and user IDs, field, see About user information.

Before you begin

Before recording user events, you should have:

  • A Google Cloud project created, with authentication set up.

  • A valid API key (for JavaScript Pixel or Tag Manager), or a valid service account with the Retail Editor Role assigned if using the API to write directly.

    For more information, see Before you begin.

Best practices for recording user events

The Retail API requires high-quality data to generate high-quality results. If your data is incomplete or incorrect, the quality of your results suffers.

When you record user events, ensure that you implement the following best practices:

  • If you record user events before or while importing your catalog, rejoin any events recorded before the catalog import completed.

    You can import the catalog to the Retail API before, after, or at the same time you record user events. Doing these tasks in parallel can save time if the catalog is large and there are many user events. Once the catalog import is complete, you must use the API to rejoin events that were uploaded before the import completed.

    Retail attempts to join recorded user events with metadata from the product catalog when the user event is created. Only successfully joined events are used for training, so make sure to rejoin any events recorded before the catalog was completely imported. If an event refers to an item that doesn't exist in the catalog, it is discarded or not associated with the correct products. Similarly, if you import user events from the past, the catalog must include any products they reference; you can mark older products as OUT_OF_STOCK rather than removing them from the catalog.

  • Keep your catalog up to date.

    When you record user events, the product included in the user event is connected with your current catalog. If you record an event for a product that is not in the current catalog, it cannot be used by the Retail API for training your models. This is called an "unjoined" event. If you recorded events before your catalog was completely imported, you must rejoin the events that were recorded during the import. Having a few unjoined events is expected. However, if the percentage of unjoined events reaches 5% or more of your total user events, make sure your catalog is up to date, rejoin events that were recorded before the catalog was fully updated, and investigate why the unjoined events are being created.

    You can see your unjoined events by using event filtering. Learn more.

  • Provide as much information with your user events as possible.

    Each user event type has different information that is required and accepted. For more information, see User events.

  • Set up alerts so that you will know if your user event recording processes experience any outages.

  • For a bulk user event import, limit the size of the data you are importing.

    A bulk user event import can take up to 24 hours to complete.

    The size of each file must be 2 GB or smaller. You can include at most 100 files in a single import request. One approach is import only the user events for one day at a time.

  • After a bulk import, review your error reporting to ensure that your data was imported correctly.

  • When importing user event data, include an accurate timestamp for each user event and avoid importing sequential user events with identical timestamps.

    Provide the timestamp in the eventTime field in the format specified by RFC 3339.

  • If you have imported user events that are incorrect, talk to your Retail contact about how to correct the problem.

  • When possible, keep your user event data continuous.

    Gaps in user event data can reduce model quality.

  • Use a secure form of a unique identifier to keep users anonymous to Retail and protect your users' privacy. You are responsible for redacting PII (personally identifiable information), such as email or home addresses, from your data.

Recording user events with a JavaScript pixel

The following example records a detail-page-view UserEvent using a JavaScript pixel.

<script type="text/javascript">
var user_event = {
  "eventType" : "detail-page-view",
  "visitorId": "visitor-id",
  "userInfo": {
      "userId": "user-id"
  },
  "attributionToken": "attribution-token",
  "experimentIds": "experiment-id",
  "productDetails": [
      {
        "product": {"id": "123"}
      }
  ]
};

var _gre = _gre || [];
// Credentials for project.
_gre.push(['apiKey', 'api-key']);
_gre.push(['logEvent', user_event]);
_gre.push(['projectId', 'project-id']);
_gre.push(['locationId', 'global']);
_gre.push(['catalogId', 'default_catalog']);

(function() {
  var gre = document.createElement('script'); gre.type = 'text/javascript'; gre.async = true;
  gre.src = 'https://www.gstatic.com/retail/v2_event.js';
  var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(gre, s);
})();

</script>

If you imported user events with Google Analytics 360, set visitorID to the Google Analytics client ID. Note that the Google Analytics client ID is only part of the full _ga cookie name (for example, client ID 123456789.123456789 is part of _ga cookie GA1.3.123456789.123456789).

For more about getting the client ID, see the Google Analytics documentation.

The following is an abbreviated example that shows the format for setting the client ID in a user event. Replace "UA-XXXXXX-N" with your Google Analytics tracking ID.

<script type="text/javascript">
var tracker = ga.getByName('UA-XXXXXX-N');
var user_event = {
      "visitorId": tracker.get('clientId')
};
</script>

Recording user events with the userEvents.write method

You can use the userEvents.write method to send user events directly to the API from your back-end server.

To record user events, send a POST request to the userEvents.write method and provide the appropriate request body.

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     --data "{
         'eventType': 'detail-page-view',
         'visitorId': 'visitor0',
         'eventTime': '2020-01-01T03:33:33.000001Z',
         'experimentIds': ['321'],
         'attributionToken': 'ABC',
         'attributes': {
            'example_text_attribute': {
              'text': ['text_1', 'text_2']
            },
            'example_number_attribute': {
               'numbers': [3.14, 42, 1.2345]
            }
         },
         'productDetails': [{
           'product': {
             'id': 'abc'
           }
          }],
         'userInfo': {
           'userId': 'abc',
           'ipAddress': '8.8.8.8',
           'userAgent': 'Mozilla/5.0',
           'directUserRequest': true
         },
         'uri': 'http://example',
         'referrerUri': 'http://example',
         'pageViewId': 'currentPageUri'
}" \
"https://retail.googleapis.com/v2/projects/[PROJECT_ID]/locations/global/catalogs/default_catalog/userEvents:write"

Recording user events with Google Tag Manager

Tag Manager provides a way to manage and test multiple tags without many server-side code changes to your site.

Some decisions you make during setup depend on whether you are using Google Analytics and Enhanced Ecommerce. Neither is required; if you don't use them, you can either configure an ecommerce variable when creating the Cloud Retail tag, or manually populate your site's data layer code after creating the tag.

Enhanced Ecommerce is an additional configuration for Google Analytics that passes product titles, IDs, prices, transaction details, and other structured ecommerce data to Google Analytics. The Retail API can automatically use the Enhanced Ecommerce data layer, so if you have that set up already, configuration can be easier. If you don't have Enhanced Ecommerce configured for Google Analytics but want to use it, see more details and setup instructions in the Enhanced Ecommerce developer guide.

Use this one-time procedure to set up a Cloud Retail tag in Tag Manager for recording user events.

Creating a visitor ID variable

The Retail API uses the value visitorId to track users. visitorId is typically a session ID, and is required for all events. Set up a variable that sets session IDs as visitorId.

If you're using Google Analytics, you can use the Google Analytics visitor ID. To configure this, use the following procedure to override the visitor ID value for the Cloud Retail tag. This maps the first-party cookie "_ga" to a Tag Manager variable called "GA visitorId". You can do the same for any session ID cookie; it doesn't have to be from Google Analytics.

This procedure assumes you are using Google Analytics. If you aren't, you can use another cookie or variable, or get the visitor ID from the cloud_retail data layer.

To set the visitorID value to a variable for the Cloud Retail tag:

  1. In Tag Manager, go to the Variables tab and click New to create a new user-defined variable.

  2. Give the variable a name at the top of the dialog, such as "GA visitorId".

  3. Enter your variable settings:

    Google Analytics 360

    1. Set Variable Type to Custom JavaScript.

    2. Enter the following script in the Custom JavaScript field.

      Replace "UA-XXXXXX-N" with your Google Analytics tracking ID. To find your tracking ID, see What happened to my Tracking ID? For more about getting the client ID, see the Google Analytics documentation.

      function() {
       var tracker = ga.getByName('UA-XXXXXX-N');
       return tracker.get('clientID');
      }
      
    3. Click Save to save the variable.

    Google Analytics

    1. Choose 1st Party Cookie as the variable type.

    2. In the Cookie Name field, enter _ga.

    3. Click Format Value, select Convert undefined to.., and enter "" (an empty string).

    4. Click Save to save the variable.

      This maps the first party cookie "_ga" to a Tag Manager variable called "GA visitorId".

Next, create a Cloud Retail tag in Tag Manager. This tag will use the visitor ID variable you just created.

Creating a Google Tag Manager tag

Set up a tag in Tag Manager to send user event information to the Retail API.

  1. Log in to Tag Manager and select the container for your site.

  2. Go to the Tags tab and click New to add a new tag.

  3. Give your tag a name at the top of the panel (the placeholder is Untitled Variable), such as "Retail".

  4. Click Tag Configuration and choose the Cloud Retail tag to open the tag configuration panel.

  5. Enter your API key.

    Use the key you created when setting up the Retail API.

    Your API keys are available from the APIs & Services > Credentials page in Cloud Console.

  6. Enter the project number of the Google Cloud project where the Retail API is enabled.

    The project number is available from your Cloud Console dashboard.

  7. For the User Event Data Source field:

    • Cloud Retail Data Layer: Select if you are not using Google Analytics, and can manually populate data layer code. See the Tag Manager Developer Guide.

    • Cloud Retail User Variable: Select to populate a Tag Manager variable with the required fields for Retail. You might choose this option if you aren't using Enhanced Ecommerce, or if Enhanced Ecommerce doesn't have the data needed for Retail.

    • Ecommerce Data Layer: Select if you have Enhanced Ecommerce implemented via Tag Manager. This enables you to reuse the Enhanced Ecommerce data layer as your events data source instead of populating a new one.

      You can only record add-to-cart, purchase-complete, and detail-page-view events via this data source.

    • Ecommerce Variable: Select if you aren't using Enhanced Ecommerce in your data layer, and can't manually populate the data layer code.

      In the Read Ecommerce Data from User Variable field that appears, choose a variable. This enables the Retail API to read Enhanced Ecommerce user event data from a custom variable that you create.

      The variable should match the format documented in the Enhanced Ecommerce Developer Guide. To construct a variable in the correct format, you can use Enhanced Ecommerce Object Builder, a custom variable template from the Tag Manager community template gallery. Community templates are not maintained by Google. To use this template, see the Enhanced Ecommerce Object Builder gallery page for documentation and other resources.

  8. Click the + Overwrite a value on the UserEvent message button.

  9. For Field Selector, select visitorId as the field, and for Field Value, select the new visitor ID variable you created in Creating a visitor ID variable.

  10. Click Save.

    Your Cloud Retail tag is created.

Next:

Creating a search query variable

If you're using Retail Search, you can create a variable in Tag Manager for search queries and attach it to your Cloud Retail tag. This allows the Retail API to get search queries from Analytics.

The types of variables you create depend on what you choose as your user event data source when you create your Cloud Retail tag.

  • Ecommerce Data Layer or Ecommerce Variable: If you chose one of these, create a URL or DOM element variable in Tag Manager and attach it to your Cloud Retail tag. In addition, enable the option for your tag to use enhanced ecommerce impressions to construct search events.
  • Cloud Retail Data Layer or Cloud Retail User Variable: If you chose one of these, create a URL or DOM element variable in Tag Manager and attach it to your Cloud Retail tag. In order for the Retail API to determine if a user event's type is search, you must also either:
    • Create a constant type variable for the search event types and attach it to your tag.
    • Set the search event type in your data layer or Cloud Retail user variable.

Creating and attaching a URL or DOM element variable

If you use Retail Search, you can create a URL or DOM element variable that will be populated with search queries entered on your site.

As an alternative to this procedure, you can configure the data layer to provide search query information. However, you might choose to use Tag Manager variables if you don't have access to the data layer, or prefer not to configure the data layer.

You can create a URL type variable or a DOM element type variable. Which one you create, and how you configure it, depends on your site's implementation:

  • A URL variable gets search queries from your site's search result URLs. Use this variable if your site includes the query string in the URL of its search results.
  • A DOM element variable gets the search query information from your site's Document Object Model (DOM). You do not need to edit the DOM to use this variable. However, you should be able to read and understand the DOM to configure this variable correctly.

First, create a URL or DOM element type variable:

  1. In Tag Manager, go to the Variables tab and click New to create a new user-defined variable.

  2. Give the variable a name at the top of the dialog, such as "search_variable".

  3. Enter your variable settings:

    URL type

    1. Set Variable Type to URL.

    2. Set Component Type to Query.

    3. If you specify a query key, set it to the key that precedes the search query in your URL.

      For example, if the URL is http://example.com/?q=shoes, the query key is q. In this example, the variable's value would get set to shoes.

    DOM element type

    1. Set Variable Type to DOM Element.

    2. Set Selection Method and enter the search query's element ID or element selector.

      This setting depends on whether your site uses an element ID or a CSS selector to identify the search query.

    3. If you specify an attribute, set it to the attribute that contains the search query term.

      For example, if the search query in your DOM is <id="search" value="shoes">, the attribute would be value. In this example, your variable's value would be set to shoes.

  4. Click Save to save the variable.

Next, attach the variable to your Cloud Retail tag:

  1. On the Tag Manager, Tags page, click your Cloud Retail tag to edit it.

  2. If your tag's user event data source is Ecommerce Data Layer or Ecommerce Variable, select the checkbox Use enhanced ecommerce impressions to construct search events.

    This allows the Retail API to determine if a user event's type is search based on search data it gets from this tag.

  3. In the User Events Data section, click the + Overwrite a value on the UserEvent message button.

  4. Select searchQuery from Field Selector and set your search query variable as the Field Value.

  5. Save your tag.

Next:

Creating and attaching a constant variable

You can use this procedure if you chose Cloud Retail Data Layer or Cloud Retail User Variable as the user event source for your Cloud Retail tag.

Creating a constant type variable for search events and setting it as a user event override on your Cloud Retail tag allows the Retail API to determine if a user event's type is search.

As an alternative to this procedure, you can specify the search user event type via the data layer or Cloud Retail user variable that you are using as your tag's event source. Otherwise, use the following steps to set the event type.

First, create a constant type variable:

  1. In Tag Manager, go to the Variables tab and click New to create a new user-defined variable.

  2. Give the variable a name at the top of the dialog, such as "search_constant".

  3. Set Variable Type to Constant.

  4. Enter search in the Value field

  5. Click Save to save the variable.

Next, attach the variable to your Cloud Retail tag:

  1. On the Tag Manager, Tags page, click your Cloud Retail tag to edit it.

  2. In the User Events Data section, click the + Overwrite a value on the UserEvent message button.

  3. Select eventType from Field Selector and set your search query variable as the Field Value.

  4. Save your tag.

Next, create event triggers for your tag.

Creating event triggers for your Tag Manager tag

Create triggers for all user event types your Retail models will use.

Tag Manager tags must have triggers that control when the tag should be "fired" on the site. Triggers listen for when events occur (such as a user viewing the home page or adding an item to their cart) and prompts your tag to send that user event information to the Retail API.

Tag Manager provides some standard triggers. For example, Window Loaded is a trigger for detail-page-view events. For details about each type, see Trigger types in the Tag Manager documentation.

You'll usually set the tag to trigger when a user views any page that has the events needed for Retail (such as the home page, product detail pages, cart pages, or checkout complete page). In these cases, the tag should fire after the page has loaded, so that cookies are available and all data layer variables are populated. To accomplish this, set your triggers to fire on Window Loaded or DOM Ready.

You might need to fire the tag when an action is performed rather than at page load (such as if a user adding an item to a cart doesn't force a page to reload). In these cases, you can configure that click action on your site to simultaneously push updates to the data layer and associate the trigger with that action.

For example, if you created a trigger for add-to-cart events, you could choose the trigger type to Click - Just Links and set it to fire on the click ID (in this example, addtocart). You would then configure the addtocart link on your site to also update the data layer with new values when clicked:

  <a id="addtocart" href="javascript:void(0);"
         onclick="dataLayer.push({
                  'cloud_retail': {
                  'eventType': 'add-to-cart',
                  'visitorId': '456',
                  'cartId': 'mobile',
                  'productDetails': [{
                  'product': {
                  'id': '54321'
                  },
                  'quantity': 1
                  }]}});">Add to Cart</a>

For some user events, you must create a custom trigger. Generally, you create a custom trigger in Tag Manager using the user event name. If you can't modify your front-end code, you can create a custom trigger using JavaScript macros. For more information about custom triggers, see Custom event trigger.

Use the following procedures to create triggers in Tag Manager:

Creating new triggers for your Tag Manager tag

If you don't use Enhanced Ecommerce, create new event triggers for any user events your Retail models need. Then, associate your new triggers with the Cloud Retail tag you created in Tag Manager.

Before you start the following steps, make sure you have created a Cloud Retail tag in Tag Manager. See Creating a Tag Manager tag.

First, create the triggers. Repeat this procedure for all user events your Retail models require:

  1. On the Tag Manager, Triggers page, click New > Trigger Configuration.

  2. Choose the trigger type that applies to the user event you're creating a trigger for.

  3. Save your trigger.

Next, associate your new triggers with your Cloud Retail tag. This is a one-time procedure:

  1. On the Tag Manager, Tags page, click your Cloud Retail tag to edit it.

  2. Click Triggering, select your new triggers, and click Add.

  3. Save your tag.

Next, preview your tag, and set up monitoring of event recording errors and other issues to make sure the Retail API keeps receiving data successfully.

If you're using cloud_retail data layer as your user event source, make sure to also set up your data layer.

Reuse Enhanced Ecommerce triggers

If you have implemented Enhanced Ecommerce via Tag Manager, reuse event triggers from your Enhanced Ecommerce for Retail.

You can only record add-to-cart, purchase-complete, and detail-page-view events via this data source.

Before you start these steps, make sure you have first:

  • Set up a tag in Tag Manager of tag type Google Analytics - Universal Analytics and enabled Enhanced Ecommerce on it. See the Tag Manager documentation and the Enhanced Ecommerce Developer Guide for details.
  • Configured your Enhanced Ecommerce tag in Tag Manager to trigger on the user events you plan to record for Retail.
  • Created a Cloud Retail tag in Tag Manager, with Enhanced Ecommerce as the user event data source (see Creating a Tag Manager tag.

To reuse Enhanced Ecommerce triggers:

  1. On the Tag Manager, Tags page, click your Enhanced Ecommerce tag (tag type Google Analytics - Universal Analytics) to edit it.

  2. Under Advanced Settings > Tag Sequencing, select Fire a tag after <Enhanced Ecommerce tag name> fires.

  3. Select your Cloud Retail tag as the Cleanup Tag.

    Select Don't fire <Cloud Retail tag name> if <Enhanced Ecommerce tag name> fails or is paused.

  4. Save your tag.

Next, preview your tag, and set up monitoring of event recording errors and other issues to make sure the Retail API keeps receiving data successfully.

If you're using cloud_retail data layer as your user event source, make sure to also set up your data layer.

Using the cloud_retail data layer with Tag Manager

If you created your Cloud Retail tag in Tag Manager to use the cloud_retail data layer as the user event source, set up the dataLayer variable in your source HTML as described in the Tag Manager Developer Guide.

About the data layer

Most Tag Manager tags require data that changes depending on the user or the page (such as user IDs or product IDs). For the Cloud Retail tag, that data must be exposed in a structured way via a data layer so that Tag Manager can use it.

The data layer is a JavaScript object that is typically added to a page using either server-side code, or in the front end using HTML or a template. If a page is configured with the data layer, it will contain some code like the following:

dataLayer = dataLayer || [];
dataLayer.push({
  'cloud_retail': {
    'eventType': 'home-page-view',
    'visitorId': 'visitor_a',

    'userInfo': {
      'userId': '789'
    },
  }
});

This code creates a dataLayer object and assigns the cloud_retail structure to it as an array element.

Required fields in the cloud_retail data layer

User events lists all required fields and examples for event types that should be passed to the cloud_retail data layer.

Your server-side code or templates should have the correct script tags on each page that you want to send events from. After the dataLayer object is populated correctly on each page, you should be able to test the Cloud Retail tag.

Some fields like visitorId are required for the UserEvent message, but might not be available when populating the data layer. For example, visitorId might be derived from the user's cookie, or experimentIds from the A/B experimentation framework. In this case, use a variable to overwrite the field on the Tag Manager tag.

You can overwrite the following fields:

  • visitorId
  • userInfo.userId
  • attributionToken
  • experimentIds

For how to overwrite a UserEvent field in Tag Manager, see Setting the visitorID field in Tag Manager, which walks through overwriting the visitorId field value with a user-defined variable.

The following example shows the data layer that needs to be included in your page for a detail-page-view UserEvent using Tag Manager:

<script>
  dataLayer = dataLayer || [];
  dataLayer.push({
    'cloud_retail': {
      'eventType' : 'detail-page-view',
      'visitorId': 'visitor_a',
      'userInfo': {
          // The user and visitor ID fields can typically be
          // be populated from a client-side JavaScript
          // variable such as a cookie. If you set the user
          // and/or visitor ID values from the server,
          // populate the `userID`.
          'userId': 'user_a'
      },
      'attributionToken': 'attribution-token',
      // In most cases, the experiment ID field is populated from a
      // client side JavaScript variable as defined by the experiment
      // manager.
      // If you set the experiment ID value from the server,
      // populate the `experimentIds` field here.
      'productDetails': [
            {
              'product': {'id': '123'}
            }
      ],
    // You can use the 'cloud_retail' data layer element along with other
    // data layer elements.
    'ecommerce': {
      ...
    },
  }];
</script>

Previewing your Tag Manager tag

Tag Manager's Preview Mode allows you to test new tags before publishing them to your live site.

For more details about Preview Mode, see the Tag Manager documentation for Preview Mode.

Use the following procedure to confirm that your tag is firing correctly.

  1. On the Tag Manager overview page, click Preview.

    Tag Manager Preview Mode opens in a new browser tab.

  2. Enter your site information and click Start to start Tag Assistant.

    In the current browser tab, Tag Assistant starts, and your site opens in a new tab.

  3. On your site, visit any page where the Cloud Retail tag should be triggered.

  4. Confirm that the Tag Assistant lists the Cloud Retail tag in the Tags tab under the Tags Fired section.

  5. In the Tag Assistant, go to the Data Layer tab and check that the correct values from the cloud_retail or ecommerce data layer are displayed.

Checking for tag errors

If some fields are incorrect or missing when you preview your tag, the tag typically also returns an error, unless it isn't firing at all.

You can check the Monitoring & Analytics page in Google Cloud Console short for errors. This page logs most errors except for syntax errors, which typically only appear in request results.

You can use the following steps to use Chrome DevTools to check for any errors generated, including syntax errors.

  1. Turn on Preview Mode in Tag Manager for your site in a Chrome browser and visit any page where the Cloud Retail tag should be triggered.

  2. With Preview Mode open, open DevTools and click the Network tab.

  3. Reload the page.

  4. In DevTools, search for userEvent.

    The Network tab displays the userEvent:collect event and its status code.

    • A 200 response indicates your tag is in a good state.
    • Other responses, such as a 400 error and highlighting the event in red, indicate that debugging is necessary.
  5. Double click the event name to execute the request and show a full response with more error information.

    For example, you might see a 400 error containing the message, "'visitorId' is required, and cannot be empty", indicating that visitorId was not set correctly.

  6. If no userEvent is fired, check the DevTools Console tab for syntax errors in the data layer.

Monitoring import health

Recording user events successfully is important for getting high-quality results. You should monitor the event recording error rates and take action if needed. For more information, see Setting up alerts for data upload issues.

What's next