Synchronize Google Cloud Customer Care cases

You can synchronize your support cases between Google Cloud and your Customer Relationship Management (CRM) system (such as Jira Service Desk, Zendesk, and ServiceNow) by building a connector to integrate them.

This connector uses Customer Care's Cloud Support API (CSAPI). This document provides an example of how you can build a connector and use it. You can tweak the design to fit your use case.

Assumptions

We make some important assumptions about how your CRM works and what language you write the connector in. If your CRM has different capabilities, you could still build a connector that works perfectly fine - but you might need to implement it in a different way than we do in this guide.

This guide is based on the following assumptions:

• You're building the connector with Python and the Flask microframework.
  • We assume that you're using Flask because it's an easy framework in which to build a small app. You can also use other languages or frameworks, such as Java.
• You want to synchronize attachments, comments, priority, case metadata, and case status. You do not need to synchronize all of that data unless you want to. For example, don't synchronize attachments if you don't want to.
• Your CRM exposes endpoints capable of reading and writing the fields you want to synchronize. If you want to synchronize every field like we do in this guide, then ensure that the endpoints of your CRM support the following operations:

  Operation	CSAPI Equivalent
  Get cases using some unchanging, static ID.	cases.get
  Create cases.	cases.create
  Close cases.	cases.close
  Update the priority of a case.	cases.patch
  List attachments on a case.	cases.attachments.list
  Download an attachment on a case.	media.download
  Upload an attachment to a case.	media.upload
  List comments on a case.	cases.comments.list
  Add a new comment on a case.	cases.comments.create
  Search through cases*.	cases.search

*You must be able to filter by last update time. There must also be some way to determine which cases are meant to be synced to Customer Care. For example, if cases in your CRM can have custom fields, you can populate a custom boolean field named synchronizeWithGoogleCloudSupport and filter based on that.

High-Level design

The connector is built entirely using Google Cloud products and your CRM. It's an App Engine app running Python in the Flask microframework. It uses Cloud Tasks to periodically poll CSAPI and your CRM for new cases and updates to existing cases, and it synchronizes changes between them. Some metadata about cases are stored in Firestore, but they're deleted after they're no longer needed.

The following diagram shows the high-level design:

Connector goals

The main goal of the connector is that when a case you'd like to synchronize is created in your CRM, a corresponding case is created in Customer Care, and all subsequent updates to the case are synchronized between them. Likewise, when a case is created in Customer Care, it must sync to your CRM.

Specifically, the following aspects of cases must be synced:

• Case creation:
  • When a case is created in one system, the connector must create a corresponding case in the other system.
  • If one system is unavailable, the case must be created in it once it is available.
• Comments:
  • When a comment is added to a case in one system, it must be added to the corresponding case in the other system.
• Attachments:
  • When an attachment is added to a case in one system, it must be added to the corresponding case in the other system.
• Priority:
  • When the priority of a case in one system is updated, the priority of the corresponding case in the other system must be updated.
• Case status:
  • When a case is closed in one system, it must be closed in the other system too.

Infrastructure

Google Cloud products

The connector is an App Engine app that uses Cloud Firestore configured in Datastore mode for storing data about the synced cases. It uses Cloud Tasks to schedule jobs with automated retry logic.

To access cases in Customer Care, the connector uses a service account to call the V2 Cloud Support API. You need to grant appropriate permissions to the service account for authentication.

Your CRM

The connector accesses cases in your CRM using a mechanism provided by you. Presumably, by calling an API exposed by your CRM.

Security considerations for your organization

The connector synchronizes cases in the organization in which you build it and all the organization's child projects. This might allow users in that organization to access customer support data that you might not want them to access. Carefully consider how you structure your IAM roles to maintain security in your organization.

Detailed design

Set up CSAPI

To set up CSAPI, follow these steps:

1. Purchase a Cloud Customer Care support service for your organization.
2. For the project in which you're going to run the connector, enable the Cloud Support API.
3. Get the credentials of the default Apps Framework service account to be used by the connector .
4. Grant the following roles to the service account at the organization level:
   • Tech Support Editor
   • Organization Viewer

For more information about setting up CSAPI, see the Cloud Support API V2 user guide.

Call CSAPI

We use Python to call CSAPI. We recommend the following two ways to call CSAPI with Python:

1. Client libraries generated from its protos. These are more modern and idiomatic, but they don't support calling CSAPI's attachment endpoints. See GAPIC Generator to learn more.
2. Client libraries generated from its discovery document. These are older, but they do support attachments. See Google API Client to learn more.

Here's an example of calling CSAPI using client libraries made from its discovery document:

"""
Gets a support case using the Cloud Support API.

Before running, do the following:
- Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to
your service account credentials.
- Install the Google API Python Client: https://github.com/googleapis/google-api-python-client
- Change NAME to point to a case that your service account has permission to get. 
"""

import os
import googleapiclient.discovery

NAME = "projects/some-project/cases/43595344"


def main():
    api_version = "v2"
    supportApiService = googleapiclient.discovery.build(
        serviceName="cloudsupport",
        version=api_version,
        discoveryServiceUrl=f"https://cloudsupport.googleapis.com/$discovery/rest?version={api_version}",
    )

    request = supportApiService.cases().get(
        name=NAME,
    )
    print(request.execute())


if __name__ == "__main__":
    main()

For more examples of calling CSAPI, see this repository.

Google Resource Names, IDs, and Numbers

Let organization_id be the ID of your organization. You can create cases in Customer Care under your organization or under a project within your organization. Let project_id be the name of a project you might create cases in.

Case Name

The name of a case looks like the following:

• organizations/{organization_id}/cases/{case_number}
• projects/{project_id}/cases/{case_number}

Where case_number is the number assigned to the case. For example, 51234456.

Comment Name

The name of a comment looks like the following:

• organizations/{organization_id}/cases/{case_number}/comments/{comment_id}

Where comment_id is a number assigned to a comment. For example, 3. Also, in addition to organizations, projects are allowed parents, too.

Attachment Name

An attachment's name looks like the following:

• organizations/{organization_id}/cases/{case_number}/attachments/{attachment_id}

Where attachment_id is the ID of an attachment in a case, if any. For example, 0684M00000JvBpnQAF. Also, in addition to organizations, projects are allowed parents, too.

Firestore Entities

CaseMapping

A CaseMapping is an object defined to store metadata about a case.

It's created for every case that is synchronized and deleted when it is no longer needed. To learn more about the data types supported in Firebase, see Supported data types.

A CaseMapping has the following properties:

Property	Description	Type	Example
ID	Primary key. Automatically assigned by Firestore when the CaseMapping is created.	Integer	123456789
googleCaseName	The full name of the case, with the organization or project ID and case number.	Text string	organizations/123/cases/456
companyCaseID	The case's ID in your CRM.	Integer	789
newContentAt	The last time new content was detected on either the Google case or the case in your CRM.	Date and time	0001-01-01T00:00:00Z
resolvedAt	A timestamp for when the Google case was resolved. Used for deleting CaseMappings when they are no longer needed.	Date and time	0001-01-01T00:00:00Z
companyUpdatesSyncedAt	A timestamp of the last time the connector successfully polled your CRM for updates and synced any updates to the Google case. Used for detecting outages.	Date and time	0001-01-01T00:00:00Z
googleUpdatesSyncedAt	A timestamp of the last time the connector successfully polled Google for updates and synced any updates to your CRM's case. Used for detecting outages.	Date and time	0001-01-01T00:00:00Z
outageCommentSentToGoogle	In case an outage has been detected, whether a comment has been added to the Google case. Used to prevent multiple outage comments from being added.	Boolean	False
outageCommentSentToCompany	In case an outage has been detected, whether a comment has been added to your CRM's case. Used to prevent multiple outage comments from being added.	Boolean	False
priority	The priority level of the case.	Integer	2

Global

Global is an object that stores global variables for the connector.

Only one Global object is created and it is never deleted. It looks like this:

Property	Description	Type	Example
ID	Primary key. Automatically assigned by Firestore when this object is created.	Integer	123456789
google_last_polled_at	The time that Customer Care was last polled for updates.	Date and time	0001-01-01T00:00:00Z
company_last_polled_at	The time that your company was last polled for updates.	Date and time	0001-01-01T00:00:00Z

Tasks

PollGoogleForUpdates

This task is scheduled to run every 60 seconds. It does the following:

• Search for recently updated cases:
  • Call CSAPI.SearchCases(organization_id, page_size=100, filter="update_time>{Global.google_last_polled_at - GOOGLE_POLLING_WINDOW}")
    • Continue fetching pages as long as a nextPageToken is returned.
    • GOOGLE_POLLING_WINDOW represents the period during which a case is continually checked for updates, even after it has been synced. The larger its value, the more tolerant the connector is to changes that are added while a case is syncing. We recommend that you set GOOGLE_POLLING_WINDOW to 30 minutes to avoid any problems with comments being added out of order.
• Make a CaseMapping for any new cases:
  • If CaseMapping does not exist for case.name and case.create_time is less than 30 days ago, then create a CaseMapping with the following values:

    Property	Value
    caseMappingID	N/A
    googleCaseName	case.name
    companyCaseID	null
    newContentAt	current_time
    resolvedAt	null
    companyUpdatesSyncedAt	current_time
    googleUpdatesSyncedAt	null
    outageCommentSentToGoogle	False
    outageCommentSentToCompany	False
    priority	case.priority (converted to an integer)

• Queue tasks to sync all recently updated cases:
  • Specifically, SyncGoogleCaseToCompany(case.name).
• Update CaseMappings:
  • For each open CaseMapping not recently updated, update CaseMapping.googleUpdatesSyncedAt to the current time.
• Update last polled time:
  • Update Global.google_last_polled_at in Firestore to the current time.

Retry logic

Configure this task to retry a few times within the first minute and then expire.

PollCompanyForUpdates

This task is scheduled to run every 60 seconds. It does the following:

• Search for recently updated cases:
  • Call YOUR_CRM.SearchCases(page_size=100, filter=”update_time>{Global.company_last_polled_at - COMPANY_POLLING_WINDOW} AND synchronizeWithGoogleCloudSupport=true”).
  • COMPANY_POLLING_WINDOW can be set to whatever time duration works for you. For example, 5 minutes.
• Make a CaseMapping for any new cases:
  • For each case, if CaseMapping does not exist for case.id and case.create_time is less than 30 days ago, create a CaseMapping that looks like this:

    Property	Value
    caseMappingID	N/A
    googleCaseName	null
    companyCaseID	case.id
    newContentAt	current_time
    resolvedAt	null
    companyUpdatesSyncedAt	null
    googleUpdatesSyncedAt	current_time
    outageCommentSentToGoogle	False
    outageCommentSentToCompany	False
    priority	case.priority (converted to an integer)

• Queue tasks to sync all recently updated cases:
  • Specifically, queue SyncCompanyCaseToGoogle(case.name).
• Update CaseMappings:
  • For each open CaseMapping not recently updated, update CaseMapping.companyUpdatesSyncedAt to the current time.
• Update last polled time:
  • Update Global.company_last_polled_at in Firestore to the current time.

Retry logic

Configure this task to retry a few times within the first minute and then expire.

SyncGoogleUpdatesToCompany(case_name)

Implementation

• Get the case and case mapping:
  • Get CaseMapping for case_name.
  • Call CSAPI.GetCase(case_name).
• If necessary, update resolved time and case status:
  • If CaseMapping.resolvedAt == null and case.status == CLOSED:
    • Set CaseMapping.resolvedAt to case.update_time
    • Close the case in the CRM as well
• Try to connect to an existing case in the CRM. If unable, then make a new one:
  • If CaseMapping.companyCaseID == null:
    • Try to get your CRM case with custom_field_google_name == case_name
      • custom_field_google_name is a custom field you create on the case object in your CRM.
    • If the CRM case can't be found, call YOUR_CRM.CreateCase(case) with the following case:

      Case field name in your CRM	Value
      Summary	Case.diplay_name
      Priority	Case.priority
      Description	"CONTENT MIRRORED FROM GOOGLE SUPPORT:\n" + Case.description
      Components	"Google Cloud"
      Customer Ticket (custom_field_google_name)	case_name
      Attachments	N/A

    • Update the CaseMapping with a CaseMapping that looks like this:

      Property	Value
      companyCaseID	new_case.id
      googleUpdatesSyncedAt	current_time

    • Add comment to Google case: "This case is now syncing with Company Case: {case_id}".
• Synchronize the comments:
  • Get all comments:
    • Call CSAPI.ListComments(case_name, page_size=100). The maximum page size is 100. Continue retrieving successive pages until the oldest comment retrieved is older than googleUpdatesSyncedAt - GOOGLE_POLLING_WINDOW.
    • Call YOUR_CRM.GetComments(case_id, page_size=50). Continue retrieving successive pages until the oldest comment retrieved is older than companyUpdatesSyncedAt - COMPANY_POLLING_WINDOW.
    • Optional: If you'd like, consider caching comments in some way so you can avoid making extra calls here. We leave the implementation of that up to you.
  • Compare both lists of comments to determine if there are new comments on the Google Case.
  • For each new Google comment:
    • Call YOUR_CRM.AddComment(comment.body), starting with "[Google Comment {comment_id}by {comment.actor.display_name}]".
• Repeat for attachments.
• Update CaseMapping.googleUpdatesSyncedAt to the current time.

Note: If a task fails in the middle of adding comments or attachments, then while retrying, it recomputes which comments or attachments need to be updated, so no duplicates are added.

Retry logic

Configure this task to retry indefinitely with exponential backoff.

SyncCompanyUpdatesToGoogle(case_id)

Implementation:

• Get the case and case mapping.
  • Get CaseMapping for case.id.
  • Call YOUR_CRM.GetCase(case.id).
• If necessary, update resolved time and case status:
  • If CaseMapping.resolvedAt == null and case.status == CLOSED:
    • Set CaseMapping.resolvedAt to case.update_time
    • Close the case in CSAPI as well
• Try to connect to an existing case in CSAPI. If unable, then make a new one:
  • If CaseMapping.googleCaseName == null:
    • Search through cases in CSAPI. Try to find a case that has a comment containing “This case is now syncing with Company Case: {case_id}”. If you're able to find one, then set googleCaseName equal to its name.
    • Otherwise, call CSAPI.CreateCase(case):
• Synchronize the comments.
  • Get all comments for the case from CSAPI and the CRM:
    • Call CSAPI.ListComments(case_name, page_size=100). Continue retrieving successive pages until the oldest comment retrieved is older than googleUpdatesSyncedAt - GOOGLE_POLLING_WINDOW.
    • Call YOUR_CRM.GetComments(case_id, page_size=50). Continue retrieving successive pages until the oldest comment retrieved is older than companyUpdatesSyncedAt - COMPANY_POLLING_WINDOW.
    • NOTE: If you'd like, consider caching comments in some way so you can avoid making extra calls here. We leave the implementation of that up to you.
  • Compare both lists of comments to determine if there are new comments on the CRM case.
  • For each new Company comment:
    • Call CSAPI.AddComment, starting with "[Company Comment {comment.id} by {comment.author.displayName}]".
• Repeat for attachments.
• Update CaseMapping.companyUpdatesSyncedAt to the current time.

Retry logic

Configure this task to retry indefinitely with exponential backoff.

CleanUpCaseMappings

This task is scheduled to run daily. It deletes any CaseMapping for a case that has been closed for 30 days according to resolvedAt.

Retry logic

Configure this task to retry with exponential backoff for up to 24 hours.

DetectOutages

This task is scheduled to run once every 5 minutes. It detects outages and alerts your Google and CRM cases (when possible) if a case is not syncing within the expected latency_tolerance.

latency_tolerance is defined as follows, where Time Since New Content = currentTime - newContentAt:

Priority	Fresh (<1 hour)	Default (1 hour-1day)	Stale (>1 day)
P0	10 min	10 min	15 min
P1	10 min	15 min	60 min
P2	10 min	20 min	120 min
P3	10 min	20 min	240 min
P4	10 min	30 min	240 min

The latency that is relevant for the connector is not request latency, but rather the latency between when a change is made in one system and when it is propagated to the other. We make latency_tolerance dependent on priority and freshness to avoid spamming cases unnecessarily. If there is a short outage, such as scheduled maintenance on either system, we don't need to alert P4 cases that haven't been updated recently.

When DetectOutages runs, it does the following:

• Determine if a CaseMapping needs an outage comment, whereupon it adds one:
  • For each CaseMapping in Firestore:
    • If recently added (companyCaseId or googleUpdatesSyncedAt is not defined), then ignore.
    • If current_time > googleUpdatesSyncedAt + latency_tolerance OR current_time > companyUpdatesSyncedAt + latency_tolerance:
      • If !outageCommentSentToGoogle:
        • Try:
          • Add comment to Google that "This case has not synced properly in {duration since sync}."
          • Set outageCommentSentToGoogle = True.
      • If !outageCommentSentToCompany:
        • Try:
          • Add comment to your CRM that "This case has not synced properly in {duration since sync}."
          • Set outageCommentSentToCompany = True.
    • Else:
      • If outageCommentSentToGoogle:
        • Try:
          • Add comment to Google that "Syncing has resumed."
          • Set outageCommentSentToGoogle = False.
      • If outageCommentSentToCompany:
        • Try:
          • Add comment to your CRM that "Syncing has resumed."
          • Set outageCommentSentToCompany = False.
• Return a failing status code (4xx or 5xx) if an outage is detected. This causes any monitoring you've set up to notice that there is a problem with the task.

Retry logic

Configure this task to retry a few times within the first 5 minutes and then expire.

What's next

Your connector is now ready to use.

If you'd like, you can also implement unit tests and integration tests. Also, you can add monitoring to check that the connector is working correctly on an ongoing basis.