Measuring recommendation performance

Recommendations AI provides metrics to help you determine how incorporating the recommendations is affecting your business.

Viewing recommendation metrics

You can view summary metrics for your project on the Business Insights page of the Recommendations AI Console. See Summary metrics for metrics definitions.

For placement-specific metrics, go to the Placement page. Click a placement name and select the Business insights tab to see its metrics. See Placement-specific metrics for metrics definitions.

Metrics definitions

Summary metrics

The table below provides definitions of the summary metrics that Recommendations AI displays on the Business Insights page.

Metric Description Details
Total revenue The total revenue from all recorded purchase events. This value includes shipping and tax.
Recommender-engaged revenue The revenue for purchase events that include at least one catalog item that was selected from a recommendation panel. This value includes shipping and tax and any discount applied.
Recommendation revenue The revenue from recommended items. This value is the sum of the original prices, as listed in the catalog, for every item that was selected from a recommendation panel and ultimately purchased. It does not include shipping, tax, or any discount applied at purchase time.
Average order value (AOV) The average value of orders from all purchase events. Total revenue divided by the number of orders.
Recommender-engaged AOV The average value of orders that include at least one item selected from a recommendation panel. Recommender-engaged revenue divided by the number of orders with at least one item that was selected from a recommendation panel.

Placement-specific metrics

You can see metrics for a specific recommendation placement on the Placement page. For metric graphs, click the placement name and select the Business insights tab.

The table below provides definitions for placement-specific metrics.

Metric Description Details
Click-through rate (CTR) The number of product detail views from a placement's recommendation panel, divided by the total number of predict queries for this placement. For example, if the recommendation placement is `shopping_cart`, then the CTR would be the number of product detail pages viewed from the shopping cart recommendation panel divided by the number of predict queries on the shopping cart page.
Conversion rate The number of add-to-cart events from a placement's recommendation panel divided by the total number of predict queries for this placement. Similar to CTR, except that instead of product detail views, conversion rate uses add to cart events.
Revenue from recommendations The total revenue from the recommendations for this placement. Similar to Recommendation revenue, except that this is for this placement only.

To track clicks from a placement's recommendation panel, Recommendations AI aligns the recommendations in the predict responses with ingested user events. If a clicked item appears in the predict responses for the same visitor ID within a one hour time window, the click/purchase is treated as a result of the recommendation.

This process is fully automatic; you do not need to set anything up. However, when you configure your prediction requests for the first time, you should confirm that:

  • Visitor IDs in the prediction request are the same as the visitor IDs you used in event ingestions.
  • The timestamp in the prediction response roughly match the timestamp for that event.

When Recommendations AI metrics are compared to the ideal expected result, or ground truth, the values might be lower, but the trends align.

A more direct alternative to the method above is to use recommendation tokens. This requires significant instrumentation and is only recommended as an advanced tracking use case.