Configuring Pub/Sub notifications

You can receive notifications when clinical events occur in Cloud Healthcare API data stores using Pub/Sub. These notifications inform you when:

When one of these events occurs, the Cloud Healthcare API publishes a message to a named Pub/Sub resource called a topic. The message can then be received by applications subscribed to the topic.

For DICOM and HL7v2, these messages do not contain any personal information. They only include the following:

  • The project for which the message is generated
  • The message ID
  • In HL7v2 messages, the HL7v2 message type

FHIR Pub/Sub messages can contain personal information. For more information, see FHIR notifications containing FHIR resource data.

For an overview of using Pub/Sub notifications with the Cloud Healthcare API, see Pub/Sub notifications.

Prerequisites

Before using this feature, complete the following sections:

Review Pub/Sub quota

Before configuring Pub/Sub notifications, familiarize yourself with Pub/Sub quotas and limits. For information on how to view your quota, request more quota, and what happens if you run out of quota, see Working with quotas.

Enable the Pub/Sub API

To enable the Pub/Sub API, click the following button:

Enable the API

Configure Pub/Sub permissions

To allow messages to be published from the Cloud Healthcare API to Pub/Sub, you must add the pubsub.publisher role to your project's Cloud Healthcare Service Agent service account. See DICOM, FHIR, and HL7v2 store Pub/Sub permissions for steps to add the required role.

Create a Pub/Sub topic

For each data store for which you want to receive notifications, you must configure a Pub/Sub topic. Individual data stores can have their own Pub/Sub topic, or multiple data stores can share the same topic. You can create a topic using the Google Cloud console or the Google Cloud CLI.

When you create a topic, or refer to a topic in a data store's configuration, you need to use a qualified URI in the following format:

projects/PROJECT_ID/topics/TOPIC_NAME

where PROJECT_ID is your Google Cloud project ID and TOPIC_NAME is the name of the topic.

To create a topic, complete the following steps:

Console

  1. Go to the Pub/Sub Topics page in the Google Cloud console.

    Go to the Pub/Sub topics page

  2. Click Create Topic.

  3. Enter a topic name with the URI:

    projects/PROJECT_ID/topics/TOPIC_NAME

    where PROJECT_ID is your Google Cloud project ID.

  4. Click Create.

gcloud

To create a topic, run the gcloud pubsub topics create command:

gcloud pubsub topics create projects/PROJECT_ID/topics/TOPIC_NAME

If the request is successful, the command returns the following output:

Created topic [projects/PROJECT_ID/topics/TOPIC_NAME].

Create a Pub/Sub subscription

To receive messages published to a topic, you need to create a Pub/Sub subscription. Every Pub/Sub topic should have at least one Pub/Sub subscription.

The subscription connects the topic to a subscriber application that receives and processes messages published to the topic.

Subscriptions can be configured to use a push model or a pull model.

To create a subscription, complete the following steps:

Console

  1. Go to the Pub/Sub Topics page in the Google Cloud console.

    Go to the Pub/Sub topics page

  2. Click your project's topic.

  3. Click Create Subscription.

  4. Enter a subscription name:

    projects/PROJECT_ID/subscriptions/SUBSCRIPTION_NAME

    Leave Delivery Type set to Pull.

  5. Click Create.

gcloud

To create a topic, run the gcloud pubsub subscriptions create command:

gcloud pubsub subscriptions create SUBSCRIPTION_NAME --topic=TOPIC_NAME

If the request is successful, the command returns the following output:

Created subscription [projects/PROJECT_ID/subscriptions/TOPIC_NAME].

View DICOM notifications

The DicomStore resource contains a notificationConfig object where you can specify a Pub/Sub topic. Optionally, the notificationConfig object also allows you to indicate whether or not Pub/Sub notifications should be sent during bulk import. This field is only available in the v1beta1 version.

When you store a new DICOM instance in a DICOM store, the Cloud Healthcare API publishes a message to the DICOM store's Pub/Sub topic.

To view a notification for a stored DICOM instance, complete the following steps:

  1. Create or edit a DICOM store and configure it with a Pub/Sub topic. Optionally, indicate whether Pub/Sub notifications should be sent during bulk import (this feature is only available in the v1beta1 API).
  2. Add the required pubsub.publisher role to your project's service account.
  3. Store an instance in the DICOM store. This triggers the Cloud Healthcare API to publish a message to the configured Pub/Sub topic.
  4. Run the gcloud pubsub subscriptions pull command to view the message published to the Pub/Sub topic:

    gcloud pubsub subscriptions pull --auto-ack projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION
    

    The command returns the following output about the DICOM instance that was stored:

    ┌----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------┐
    |                                                                          DATA                                                                           |    MESSAGE_ID   | ATTRIBUTES |
    ├---------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------|------------|
    | projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/dicomStores/DICOM_STORE_ID/dicomWeb/studies/STUDY_UID/series/SERIES_UID/instances/INSTANCE_UID | 123456789012345 |            |
    └----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------┘
    

View FHIR notifications

The following options are available when configuring FHIR Pub/Sub notifications:

The following table shows the information contained in the Pub/Sub message for each option after creating a Patient FHIR resource:

NotificationConfig (without FHIR resource data)
{
  "receivedMessages": [
    {
      "ackId": "UAYWLF1GSFE3GQhoUQ5PXiM_NSAoRRICB08CKF15MFQqQV92Dj4NGXJ9YXRtDEVTUUJWd1gIEQ1iXE5EB0nh1PDfV1dKXhACA0FVfFxfHQlrWFtzBXmgo-iEzsKGbQk9Oqme_P1tO-KSgaREZiI9XhJLLD5-PTJFQV5AEkw2B0RJUytDCypYEU4EISE-MD5FU0Q",
      "message": {
        "data": "cHJvamVjdHMvbm9lcm8taGVhbHRoY2FyZS9sb2NhdGlvbnMvdXMtY2VudHJhbDEvZGF0YXNldHMvbXlkYXRhc2V0L2ZoaXJTdG9yZXMvbm90aWZpY2F0aW9uZmhpci9maGlyL1BhdGllbnQvNmU3NDIxNTEtNGNlNi00N2UwLTk2MjUtZGUzODc5M2RkMjQ4",
        "attributes": {
          "action": "CreateResource",
          "payloadType": "NameOnly",
          "resourceType": "Patient"
        },
        "messageId": "4331462346434762",
        "publishTime": "PUBLISH_TIME"
      }
    }
  ]
}
FhirNotificationConfig (with FHIR resource data)
{
  "receivedMessages": [
    {
      "ackId": "U0RQBhYsXUZIUTcZCGhRDk9eIz81IChFGwIIFAV8fXFBWXVaVBoHUQ0ZcnxmIz9bG1AHQlZ2VVsRDXptXG3z7KW7RF9BcWlaEwELQFB9X10TDWhVWl3Euqi8lfyEQnBmK_W_7fFIf4WI7M9vZiA9XxJLLD5-PTJFQV5AEkw2B0RJUytDCypYEU4EISE-MD5F",
      "message": {
        "data": "ewogICJiaXJ0aERhdGUiOiAiMTk3MC0wMS0wMSIsCiAgImdlbmRlciI6ICJmZW1hbGUiLAogICJpZCI6ICIzZDQzMjMwMi1mZWYyLTRlYTUtYWJlNi0wNDAzYzlhMmY1NGIiLAogICJtZXRhIjogewogICAgImxhc3RVcGRhdGVkIjogIjIwMjItMDMtMzFUMTk6NTM6MzMuNzg3ODE3KzAwOjAwIiwKICAgICJ2ZXJzaW9uSWQiOiAiTVRZME9EYzFOalF4TXpjNE56Z3hOekF3TUEiCiAgfSwKICAibmFtZSI6IFsKICAgIHsKICAgICAgImZhbWlseSI6ICJTbWl0aCIsCiAgICAgICJnaXZlbiI6IFsKICAgICAgICAiRGFyY3kiCiAgICAgIF0sCiAgICAgICJ1c2UiOiAib2ZmaWNpYWwiCiAgICB9CiAgXSwKICAicmVzb3VyY2VUeXBlIjogIlBhdGllbnQiCn0=",
        "attributes": {
          "storeName": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/fhirStores/FHIR_STORE_ID",
          "lastUpdatedTime": "LAST_UPDATED_TIME",
          "action": "CreateResource",
          "payloadType": "FullResource",
          "resourceType": "Patient"
        },
        "messageId": "4300095330680096",
        "publishTime": "PUBLISH_TIME"
      }
    }
  ]
}

FHIR notifications without FHIR resource data

The FhirStore resource contains a NotificationConfig object where you can specify a Pub/Sub topic.

When a FHIR resource is created, updated, or deleted in a FHIR store, the Cloud Healthcare API publishes a message to the FHIR store's Pub/Sub topic.

To view a notification for a created FHIR resource, complete the following steps:

  1. Create or edit a FHIR store and configure it with a Pub/Sub topic.
  2. Add the required pubsub.publisher role to your project's service account.
  3. Create a FHIR resource in the FHIR store. This triggers the Cloud Healthcare API to publish a message to the configured Pub/Sub topic.
  4. Run the gcloud pubsub subscriptions pull command to view the message published to the Pub/Sub topic:

    gcloud pubsub subscriptions pull --auto-ack projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION
    

    The command returns the following output about the created FHIR resource:

    ┌----------------------------------------------------------------------------------------------------------------|-----------------|-----------------------┐
    |                                                      DATA                                                      |    MESSAGE_ID   |       ATTRIBUTES      |
    ├----------------------------------------------------------------------------------------------------------------|-----------------|-----------------------|
    | projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/fhirStores/FHIR_STORE_ID/resources/Patient/PATIENT_ID | 123456789012345 | action=CreateResource |
    |                                                                                                                |                 | resourceType=Patient  |
    └----------------------------------------------------------------------------------------------------------------|-----------------|-----------------------┘
    

FHIR notifications containing FHIR resource data

In the v1beta1 version of the Cloud Healthcare API, the FhirStore resource contains a FhirNotificationConfig object where you can specify the following information:

  • The Pub/Sub topic to attach to the FHIR store
  • Whether to send the full contents of the changed FHIR resource to the Pub/Sub topic

By sending the full contents of the FHIR resource, you can retrieve all of the information about the FHIR resource directly from the Pub/Sub message. This eliminates having to query for Pub/Sub messages and then separately query the Cloud Healthcare API to get details about the FHIR resource.

FHIR Pub/Sub messages contain the following information in the attributes field:

  • The type of FHIR resource (resourceType)
  • The action that caused the message (action)
  • The payload type of the message (payloadType), one of nameOnly or fullResource
  • The full resource name of the FHIR store where the action occurred (storeName)
  • A timestamp of the most recent time the FHIR resource was modified (lastUpdatedTime) using the RCF1123 format

To view a notification for a created FHIR resource and view the full contents of the FHIR resource, complete the following steps:

  1. Create or edit a FHIR store and configure it with a notificationConfigs object. In the following sample, you create a FHIR store.

    Before using any of the request data, make the following replacements:

    • PROJECT_ID: your Google Cloud project ID
    • LOCATION: the location of the parent dataset
    • DATASET_ID: the FHIR store's parent dataset
    • FHIR_STORE_ID: the FHIR store ID
    • PUBSUB_TOPIC_ID: the Pub/Sub topic ID

    Request JSON body:

    {
      "notificationConfigs": [
        {
          "pubsubTopic": "projects/PROJECT_ID/topics/PUBSUB_TOPIC_ID",
          "sendFullResource": true
        }
      ],
      "version": "R4"
    }
    

    To send your request, choose one of these options:

    curl

    1. Save the request body in a file called request.json. Copy the following command and run it in the terminal to create this file.
      cat > request.json << 'EOF'
      {
        "notificationConfigs": [
          {
            "pubsubTopic": "projects/PROJECT_ID/topics/PUBSUB_TOPIC_ID",
            "sendFullResource": true
          }
        ],
        "version": "R4"
      }
      EOF
    2. Execute the following command in the terminal. It references the request.json file you just created.
      curl -X POST \
      -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
      -H "Content-Type: application/fhir+json" \
      -d @request.json \
      "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/fhirStores/?fhirStoreId=FHIR_STORE_ID"

    PowerShell

    1. Save the request body in a file called request.json. Copy the following command and run it in the terminal to create this file.
      @'
      {
        "notificationConfigs": [
          {
            "pubsubTopic": "projects/PROJECT_ID/topics/PUBSUB_TOPIC_ID",
            "sendFullResource": true
          }
        ],
        "version": "R4"
      }
      '@  | Out-File -FilePath request.json -Encoding utf8
    2. Execute the following command in the terminal. It references the request.json file you just created.
      $cred = gcloud auth application-default print-access-token
      $headers = @{ "Authorization" = "Bearer $cred" }

      Invoke-WebRequest `
      -Method POST `
      -Headers $headers `
      -ContentType: "application/fhir+json" `
      -InFile request.json `
      -Uri "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/fhirStores/?fhirStoreId=FHIR_STORE_ID" | Select-Object -Expand Content

    You should receive a JSON response similar to the following:

  2. Add the required pubsub.publisher role to your project's service account.

  3. Create a FHIR resource in the FHIR store. This triggers the Cloud Healthcare API to publish a message to the configured Pub/Sub topic.

  4. View the message published to the Pub/Sub topic. The following message was generated when a Patient resource was created in a FHIR store:

    REST & CMD LINE

    Before using any of the request data, make the following replacements:

    • PROJECT_ID: your Google Cloud project ID
    • PUBSUB_SUBSCRIPTION_ID: the ID of the subscription attached to the Pub/Sub topic configured in the FHIR store

    To send your request, choose one of these options:

    curl

    Execute the following command:

    curl -X POST \
    -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
    -H "Content-Type: application/json; charset=utf-8" \
    -d "" \
    "https://pubsub.googleapis.com/v1beta1/projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION_ID:pull?maxMessages=1"

    PowerShell

    Execute the following command:

    $cred = gcloud auth application-default print-access-token
    $headers = @{ "Authorization" = "Bearer $cred" }

    Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -Uri "https://pubsub.googleapis.com/v1beta1/projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION_ID:pull?maxMessages=1" | Select-Object -Expand Content

    You should receive a JSON response similar to the following:

    gcloud

    Before using any of the command data below, make the following replacements:

    • PROJECT_ID: your Google Cloud project ID
    • PUBSUB_SUBSCRIPTION_ID: the ID of the subscription attached to the Pub/Sub topic configured in the FHIR store

    Execute the following command:

    Linux, macOS, or Cloud Shell

    gcloud pubsub subscriptions pull --auto-ack projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION

    Windows (PowerShell)

    gcloud pubsub subscriptions pull --auto-ack projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION

    Windows (cmd.exe)

    gcloud pubsub subscriptions pull --auto-ack projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION

    You should receive a response similar to the following:

    ┌────────────────────────────────────────────────────────┬──────────────────┬──────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────┬──────────────────┐
    │                          DATA                          │    MESSAGE_ID    │ ORDERING_KEY │                                                   ATTRIBUTES                                                  │ DELIVERY_ATTEMPT │
    ├────────────────────────────────────────────────────────┼──────────────────┼──────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────┼──────────────────┤
    │ {                                                      │ 4300088465623331 │              │ action=CreateResource                                                                                         │                  │
    │   "birthDate": "1970-01-01",                           │                  │              │ lastUpdatedTime=LAST_UPDATED_TIME                                                                             │                  │
    │   "gender": "female",                                  │                  │              │ payloadType=FullResource                                                                                      │                  │
    │   "id": "3fbfac80-6357-44ad-b886-80f92a64bf7b",        │                  │              │ resourceType=Patient                                                                                          │                  │
    │   "meta": {                                            │                  │              │ storeName=projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/fhirStores/FHIR_STORE_ID                 │                  │
    │     "lastUpdated": "LAST_UPDATED",                     │                  │              │                                                                                                               │                  │
    │     "versionId": "MTY0ODc1NTk3MDMzODU3MzAwMA"          │                  │              │                                                                                                               │                  │
    │   },                                                   │                  │              │                                                                                                               │                  │
    │   "name": [                                            │                  │              │                                                                                                               │                  │
    │     {                                                  │                  │              │                                                                                                               │                  │
    │       "family": "Smith",                               │                  │              │                                                                                                               │                  │
    │       "given": [                                       │                  │              │                                                                                                               │                  │
    │         "Darcy"                                        │                  │              │                                                                                                               │                  │
    │       ],                                               │                  │              │                                                                                                               │                  │
    │       "use": "official"                                │                  │              │                                                                                                               │                  │
    │     }                                                  │                  │              │                                                                                                               │                  │
    │   ],                                                   │                  │              │                                                                                                               │                  │
    │   "resourceType": "Patient"                            │                  │              │                                                                                                               │                  │
    │ }                                                      │                  │              │                                                                                                               │                  │
    └────────────────────────────────────────────────────────┴──────────────────┴──────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────┘
    

Behavior when FHIR resource is too large or traffic is high

In the following cases, the attributes field might only contain the resource name (in the format projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/fhirStores/FHIR_STORE_ID/fhir/FHIR_RESOURCE/FHIR_RESOURCE_ID) even if sendFullResource is set to true:

  • The size of a FHIR resource is too large
  • The Cloud Healthcare API servers are experiencing high traffic

Ensure that you check the payloadType field in the response from viewing the Pub/Sub notification. If payloadType is set to nameOnly, then the attributes field did not fully populate the FHIR resource data, and you must get the contents of the FHIR resource manually from the FHIR store instead of from the Pub/Sub message.

View HL7v2 notifications

The Hl7V2Store resource contains an array notificationConfigs where you can specify Pub/Sub topics and filtering criteria.

When an HL7v2 message is ingested or created in an HL7v2 store, the Cloud Healthcare API publishes a message to the Pub/Sub topics that have a filter that matches the HL7v2 message.

To view a notification for an ingested HL7v2 message, complete the following steps:

  1. Create or edit an HL7v2 store and configure it with a Pub/Sub topic.
  2. Add the required pubsub.publisher role to your project's service account.
  3. Ingest an HL7v2 message into the HL7v2 store. This triggers the Cloud Healthcare API to publish a message to the configured Pub/Sub topic.
  4. To view the message published to the Pub/Sub topic, run the gcloud pubsub subscriptions pull command:

    gcloud pubsub subscriptions pull --auto-ack projects/PROJECT_ID/subscriptions/PUBSUB_SUBSCRIPTION
    

    The command returns the following output about the ingested HL7v2 message:

    ┌--------------------------------------------------------------------------------------------------------------------|-----------------|---------------┐
    |                                                                 DATA                                               |    MESSAGE_ID   |   ATTRIBUTES  |
    ├--------------------------------------------------------------------------------------------------------------------|-----------------|---------------|
    | projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID/hl7V2Stores/HL7V2_STORE_ID/messages/HL7V2_MESSAGE_ID | 123456789012345 | msgType=TYPE  |
    └--------------------------------------------------------------------------------------------------------------------|-----------------|---------------┘
    

Cloud Healthcare API and Pub/Sub message storage policy

To ensure that your Cloud Healthcare API data and the associated data in Pub/Sub messages reside in the same region, you must set a Pub/Sub message storage policy.

You must explicitly set the message storage policy on the Pub/Sub topic configured on the FHIR store to ensure that the data stays in the same region. For example, if your Cloud Healthcare API dataset and FHIR store are in us-central1, the message storage policy must only allow the us-central1 region.

To configure a message storage policy, see Configuring message store policies.

Troubleshooting missed Pub/Sub messages

If a notification can't be published to Pub/Sub, an error is logged to Cloud Logging. For more information, see Viewing error logs in Cloud Logging.

If the rate of error generation exceeds a limit, errors in excess of this limit aren't submitted to Cloud Logging.

What's next

Learn how to configure a Pub/Sub topic in: