Training a Smart Reply model & managing allowlists

This tutorial walks you through the process of training and deploying a model using the Agent Assist Console. You can use the console to train a model and test its performance, but be aware that all runtime operations must be carried out by calling the API directly. See the Agent Assist how-to guides for instructions.

Before you begin

  1. If you are using the publicly-available conversation dataset, review the documentation that guides you through using these resources.
  2. If you are using your own data, make sure that you have created one or more conversation datasets. At least one conversation dataset is required for model training.

Create & train a new model

  1. Navigate to the Agent Assist console and click on the Models menu option on the far left margin of the page:

  2. The Models menu displays all of your models. To create a new model, click on the +Create new button at the top right:

    The following page appears:

  3. Under Model type select Smart Reply.

  4. Enter a unique name for your new model in the Name field.

  5. Select the name(s) of the conversation dataset(s) that you want to use to train your model. A conversation dataset must contain at least 30,000 conversations in order to train a Smart Reply model, otherwise training will fail. When you click on the training dataset field a pop-up menu appears that contains a list of all of the datasets you've already created. You can select as many datasets as you'd like.

  6. Click Create to create your new model. The new model now appears in the list of models on the Models page. The new model's status will appear as Pending and then Creating until the model is trained.

Deploying a model

Training can take a day or more. To deploy your trained model, click on the icon made up of 3 vertical dots that's associated with your model at the far right of the page and click Deploy.

Model allowlist management

When a Smart Reply model is created using the Agent Assist console, the creation process also generates a list of potential suggestions that can be surfaced to agents during a conversation. This resource is called the allowlist. The allowlist contains all possible suggestions that were generated using your selected conversation dataset.

To access a model's allowlist, navigate to the Models menu, then click on the model's name. You will be brought to the model's Details page. There are tabs for each of the three sections in the allowlist: Unreviewed, Allowed, and Blocked. Each message in the allowlist can belong to one of these three lists. Blocked suggestions cannot be suggested to agents at runtime. Only suggestions that appear on the Allowed and Unreviewed lists are surfaced during a conversation. By default, all suggestions on the allowlist are set to Unreviewed when a model is created.

Message grouping

An allowlist might have 50,000+ messages. To make them easier to manage, Agent Assist automatically groups messages with similar meaning. You can see a message's similar messages by clicking on the Show (X) similar messages link to the right of the message:

Message searching, editing, and creation

You can search all messages in an allowlist using the search entries bar at the top of the page. You can also edit individual messages by hovering over a message and clicking the pencil icon that appears. A window containing the message's text appears:

We recommend that you edit only for spelling and grammar, and do not change the meaning of the message. The more the edited text deviates from the meaning in the model, the less likely that message is to be surfaced.

You also have the ability to create a new message by clicking the +Add a message button at the top right of the page. Similarly to edited messages, created messages are less likely to be surfaced during runtime.

Allowing/blocking messages

When you select one or more messages in the Unreviewed messages list, MOVE TO ALLOWED and MOVE TO BLOCKED buttons will appear at the top right of the page. You can also move individual messages to the allowed or blocked lists by clicking on the three vertical dots to the far right of the model's name:

What's next

Use your deployed model to create a conversation profile.