Quickstart: Import conversations using Cloud Data Fusion

This guide walks you through the process of creating a pipeline in Cloud Data Fusion and successfully loading conversation data into CCAI Insights.

Prerequisites

1. Enable the CCAI Insights Accelerator on a Cloud Data Fusion instance.
2. Install the CCAI Insights plugins.
3. Make sure that your conversation data is staged in a Google Cloud Storage bucket in a valid format. See the Cloud Storage documentation for information about creating a bucket.
   1. The CCAI Insights service agent and the Compute Engine default service account must have access to the objects in your Cloud Storage bucket. The email addresses for these service accounts use the following format:
      • CCAI Insights service agent: [e-mail]
      • Compute Engine default service account: [e-mail]
4. The CCAI Insights import quickstart supports importing up to 18,000 conversations per hour. If you need to exceed this threshold, please contact your Google representative.

Create a pipeline

1. Go to the project selector page
2. Navigate to the Cloud Data Fusion Instances page.
3. Select the View instance button next to the instance you want to use. Click on the HUB tab in the new window that pops up to bring up the instance's HUB main page.

4. Click on the CCAI tab, navigate to the CCAI Import Quickstart card, and click Create. Click Finish to create the pipeline, then click the Customize Pipeline button.

   Note: Do not select the CCAI Import Quickstart (Advanced) version of the quickstart. The advanced option is intended for use cases that exceed the import limit of 18,000 conversations per hour. Please contact your Google representative if you need to exceed this threshold.

5. Hover over the CCAI Insights Import plugin card and click on the Properties button that appears:

   

6. Configure the type of conversation data you are importing by selecting either Phone call or Chat in the Conversation Medium field. The default option is Phone call. Click the x in the upper right corner of the screen to close the Properties window.

7. On the Cloud Data Fusion Studio screen, enter a name and description for your pipeline by clicking on the pipeline's name in the upper left corner:

   

8. After naming your pipeline, Save and then Deploy it. Optionally, you can set other configurations such as running the pipeline on a schedule by clicking on the Schedule and Configure buttons.

   

Import conversation data

1. When the new pipeline is ready you will be redirected to the pipeline view. Click Run and fill in the name of your input Google Cloud Storage bucket in the Value field to the right of the inputBucket key. Your bucket name must be in the format BUCKET-NAME/PREFIX-NAME. If the conversation data files you want to import are not organized in a prefix (the Cloud Storage term for a folder in a storage bucket), you can enter the bucket name only (BUCKET-NAME). In this case, CCAI Insights will import all files in the bucket. Click Run.
2. Your pipeline has successfully completed when Status is Succeeded. This process can take several minutes to hours depending on how much data you are importing. Navigate to https://ccai.cloud.google.com/insights/projects/PROJECT-NAME/conversations to see your newly created conversations in the CCAI Insights Console.

What's next

Export your conversation data to BigQuery using Cloud Data Fusion.