Summarization

Stay organized with collections Save and categorize content based on your preferences.

The Agent Assist Summarization feature allows you to provide conversation summaries to your agents after each conversation is completed. The summaries help agents create their conversation notes and understand end-user communication history. For example, a summary output about a conversation might look similar to the following:

Summarization training data

Summarization can be used in conjunction with any Agent Assist feature, or as a stand-alone feature. In order to implement Summarization, you must provide Agent Assist with conversation data that includes annotations. An annotation is a summary of an associated conversation transcript. Annotations are used to train a model that you can use to generate summaries for your agents at the end of each conversation with an end-user.

Agent Assist also provides sample annotated conversation data that you can use to train a model. We recommend that you choose this option if you would like to test the Summarization feature before you format your own dataset. The test dataset is located in the following Cloud Storage bucket: gs://summarization_integration_test_data/data. If you use the sample data, you can skip ahead to train a model.

Format your training data

Agent Assist Summarization models are trained using conversation datasets. A conversation dataset contains your own uploaded transcript and annotation data.

Before you can begin uploading data, you must make sure that that each conversation transcript is in JSON format, has an associated annotation, and is stored in a Google Cloud Storage bucket.

To create annotations, add expected key and value strings to the annotation field associated with each conversation in your dataset. For best results, annotation training data should adhere to the following guidelines:

  1. The recommended minimum number of training annotations is 1000. The enforced minimum number is 100.
  2. Training data should not contain PII.
  3. Annotations should not include any information about gender, race or age.
  4. Annotations should not use toxic or profane language.
  5. Annotations should not contain any information that can't be inferred from the corresponding conversation transcript.
  6. Each annotation can contain up to 3 sections. You can choose your own section names.
  7. Annotations should have correct spelling and grammar.

The following is an example demonstrating the format of a conversation transcript with associated annotation:

{
  "entries": [
    {
      "text": "How can I help?",
      "role": "AGENT"
    },
    {
      "text": "I cannot login",
      "role": "CUSTOMER"
    },
    {
      "text": "Ok, let me confirm. Are you experiencing issues accessing your account",
      "role": "AGENT"
    },
    {
      "text": "Yes",
      "role": "CUSTOMER"
    },
    {
      "text": "Got it. Do you still have access to the registered email for the account",
      "role": "AGENT"
    },
    {
      "text": "Yes",
      "role": "AGENT"
    },
    {
      "text": "I have sent an email with reset steps. You can follow the instructions in the email to reset your login password",
      "role": "AGENT"
    },
    {
      "text": "That's nice",
      "role": "CUSTOMER"
    },
    {
      "text": "Is there anything else I can help",
      "role": "AGENT"
    },
    {
      "text": "No that's all",
      "role": "CUSTOMER"
    },
    {
      "text": "Thanks for calling. You have a nice day",
      "role": "AGENT"
    }
  ],
  "conversation_info": {
    "annotations": [
      {
        "annotation": {
          "conversation_summarization_suggestion": {
            "text_sections": [
              {
                "key": "Situation",
                "value": "Customer was unable to login to account"
              },
              {
                "key": "Action",
                "value": "Agent sent an email with password reset instructions"
              },
              {
                "key": "Outcome",
                "value": "Problem was resolved"
              }
            ]
          }
        }
      }
    ]
  }
}

Train and deploy a Summarization model

Create a conversation dataset and import transcripts

Call the create method on the ConversationDataset resource to create a conversation dataset. Include the path to the Cloud Storage bucket containing your transcript data to import it into the new conversation dataset.

Example request:

{
  "parent": "projects/PROJECT_ID/locations/global",
  "conversationDataset": {
    "displayName": "CONVERSATION_DATASET_NAME",
    "inputConfig": {
      "gcsSource": {
        "uris": ["gs://PATH_NAME/*"]
      }
    }
  }
}

The response contains a conversation dataset ID. Sample response:

{
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.dialogflow.v2beta1.ConversationDataset",
    "name": "projects/PROJECT_ID/locations/global/conversationDatasets/CONVERSATION_DATASET_ID",
    "displayName": "CONVERSATION_DATASET_NAME",
    "createTime": "2022-06-16T23:13:22.627380457Z"
  }
}

Create a conversation model

Call the create method on the ConversationModel resource to create a Summarization conversation model. Each project will have up to 120 node hours for training every month and can run one training job concurrently.

Required fields:

  • datasets: Provide a single dataset that contains the transcript data you used to train the model.
  • summarizationModelMetadata: Set to an empty object, or populate the field to override the default.

Example request:

{
  "parent": "projects/PROJECT_ID/locations/global",
  "conversationModel": {
    "displayName": "CONVERSATION_DATASET_NAME",
    "datasets": [{
      "dataset": "projects/PROJECT_ID/locations/global/conversationDatasets/CONVERSATION_DATASET_ID"
    }],
    "summarizationModelMetadata": {
      "trainingModelType": "SUMMARIZATION_MODEL"
    },
    "languageCode": "en-US"
  }
}

The response is a long-running operation, which you can poll using the GetOperation API to check for completion. Training can take several hours. The response returns the status and model ID.

Example response:

{
  "name": "projects/PROJECT_ID/locations/global/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.dialogflow.v2beta1.CreateConversationModelOperationMetadata",
    "conversationModel": "projects/PROJECT_ID/locations/global/conversationModels/MODEL_ID",
    "state": "TRAINING",
    "createTime": "2022-06-16T23:27:50Z"
  }
}

Evaluate a conversation model

After the model training is completed, call the ListConversationModelEvaluations API to check the model quality.

In the ListConversationModelEvaluationsRequest, specify the name of the model to check.

{
  "parent": "projects/cloud-contact-center-quality/locations/global/conversationModels/d3f2ca9f3e2c571a"
}

The ListConversationModelEvaluationsResponse includes a Rouge-L score for evaluating automatic summarization.

{
  "conversationModelEvaluations": [{
    "name": "projects/cloud-contact-center-quality/locations/global/conversationModels/d3f2ca9f3e2c571a/evaluations/c10ac25411a23fe1",
    "displayName": "Training Auto Generated Evaluation",
    "createTime": "2022-06-04T03:38:35.151096Z",
    "evaluationConfig": {
    },
    "summarizationMetrics": {
      "rougel": 0.4474459
    }
  }]
}

Deploy a conversation model

Once the model training is completed, you can deploy the model using the DeployConversationModel API.

In the DeployConversationModelRequest, specify the name of the model to deploy. The response is a long-running operation, which you can poll using the GetOperation API to check for completion.

Configure a conversation profile

A conversation profile configures a set of parameters that control the suggestions made to an agent during a conversation. The following steps create a ConversationProfile with a HumanAgentAssistantConfig object.

Create a conversation profile

To create a conversation profile, call the create method on the ConversationProfile resource. Provide a name for the new conversation profile, your Google Cloud Platform project ID, and model ID. In the CreateConversationProfileRequest, specify the conversation model to use and the suggestion feature to be CONVERSATION_SUMMARIZATION.

The following is a JSON example:

{
  "parent": "projects/PROJECT_ID/locations/global",
  "conversationProfile": {
    "displayName": "CONVERSATION_PROFILE_NAME",
    "humanAgentAssistantConfig": {
      "humanAgentSuggestionConfig": {
        "featureConfigs": [{
          "suggestionFeature": {
            "type": "CONVERSATION_SUMMARIZATION"
          },
          "conversationModelConfig": {
            "model": "projects/PROJECT_ID/locations/global/conversationModels/MODEL_ID",
          }
        }]
      }
    },
    "languageCode": "en-US"
  }
}

The response is a ConversationProfile object containing the conversation profile name:

{
  "name": "projects/PROJECT_ID/locations/global/conversationProfiles/CONVERSATION_PROFILE_ID",
  "displayName": "CONVERSATION_PROFILE_NAME",
  "automatedAgentConfig": {
  },
  "humanAgentAssistantConfig": {
    "notificationConfig": {
    },
    "humanAgentSuggestionConfig": {
      "featureConfigs": [{
        "suggestionFeature": {
          "type": "CONVERSATION_SUMMARIZATION"
        },
        "conversationModelConfig": {
        }
      }]
    },
    "messageAnalysisConfig": {
    }
  },
  "languageCode": "en-US",
  "createTime": "2022-06-06T21:06:46.841816Z",
  "updateTime": "2022-06-06T21:06:46.841816Z",
  "projectNumber": "344549229138"
}

Handle conversations at runtime

When a dialog begins between an end-user and a human agent, you create a conversation. The following sections walk you through this process.

Create a conversation

To create a conversation, call the create method on the Conversation resource.

The response contains your new conversation ID.

Create an end-user participant

To create an end-user participant, call the create method on the Participant resource. Provide your conversation ID and END_USER for the role field.

The path segment after participants contains your new end user participant ID.

Create a human agent participant

To create a human agent participant, call the create method on the Participant resource. Provide your conversation ID and HUMAN_AGENT for the role field.

The path segment after participants contains your new human agent participant ID.

Analyze message

Option 1: During a conversation

To add and analyze a human agent message for the conversation, call the analyzeContent method on the Participant resource. Provide the conversation ID and human agent participant ID.

To add and analyze an end-user message for the conversation, call the analyzeContent method on the Participant resource. Provide the conversation ID and end-user participant ID.

Do not make duplicate calls to the analyzeContent method if it has been called for other Dialogflow features.

Option 2: After a conversation

You can use this option if you don't use the analyzeContent method during the conversation. Instead, you can use the batchCreate method on the messages resource to ingest historical messages of the conversation.

Get a suggestion

You can get a suggestion at any time for the latest message from either participant. Call the suggestConversationSummary method on the conversations.suggestions resource. Provide the conversation ID and the latest message ID from either participant.

The response contains Summarization suggestions.

Complete the conversation

To complete the conversation, call the complete method on the conversations resource. Provide the conversation ID.