Importing and exporting HL7v2 messages using Cloud Storage

This page describes how to export and import HL7v2 messages to and from Cloud Storage using the projects.locations.datasets.hl7V2Stores.import and projects.locations.datasets.hl7V2Stores.export methods.

Setting Cloud Storage permissions

Before exporting and importing HL7v2 messages to and from Cloud Storage, you must grant additional permissions to the Cloud Healthcare Service Agent service account. For more information, see HL7v2 store Cloud Storage permissions.

Importing HL7v2 messages

To import HL7v2 messages, you must first create one or more newline-delimited JSON (.ndjson) files in Cloud Storage containing one or more messages. Each line of the file is a single Message resource which contains a base64-encoded HL7v2 message. The Message resource can also include optional labels.

For example, the following file, named messages.ndjson, contains two HL7v2 messages. A label is defined in the second message.

{"data" :"TVNIfF5+XCZ8QXxTRU5EX0ZBQ0lMSVRZXzF8QXxBfDIwMTgwMTAxMDAwMDAwfHxUWVBFXkF8MjAxODAxMDEwMDAwMDB8VHwwLjB8fHxBQXx8MDB8QVNDSUkNRVZOfEEwMHwyMDE4MDEwMTA0MDAwMA1QSUR8fDE0ATExMV5eXl5NUk58MTExMTExMTFeXl5eTVJOfjExMTExMTExMTFeXl5eT1JHTk1CUg=="}
{"data" :"TVNIfF5+XCZ8QXxTRU5EX0ZBQ0lMSVRZXzJ8QXxBfDIwMTgwMTAxMDAwMDAwfHxUWVBFXkF8MjAxODAxMDEwMDAwMDB8VHwwLjB8fHxBQXx8MDB8QVNDSUkNRVZOfEEwMHwyMDE4MDEwMTA0MDAwMA1QSUR8fDE0ATExMV5eXl5NUk58MTExMTExMTFeXl5eTVJOfjExMTExMTExMTFeXl5eT1JHTk1CUg==","labels":{"foo":"bar"}}

The following samples show how to import HL7v2 messages from Cloud Storage using the projects.locations.datasets.hl7V2Stores.import method.

Note the following when calling the import operation:

  • The location of the file within the bucket is arbitrary and does not have to adhere exactly to the format specified in the following samples.
  • When specifying the location of the HL7v2 messages in Cloud Storage, you can use wildcards to import multiple files from one or more directories. The following wildcards are supported:
    • Use * to match 0 or more non-separator characters. For example, gs://BUCKET/DIRECTORY/Example*.ndjson matches Example.ndjson and Example22.ndjson in DIRECTORY.
    • Use ** to match 0 or more characters (including separators). Must be used at the end of a path and with no other wildcards in the path. Can also be used with a filename extension (such as .ndjson), which imports all files with the filename extension in the specified directory and its subdirectories. For example, gs://BUCKET/DIRECTORY/**.ndjson imports all files with the .ndjson filename extension in DIRECTORY and its subdirectories.
    • Use ? to match 1 character. For example, gs://BUCKET/DIRECTORY/Example?.ndjson matches Example1.ndjson but does not match Example.ndjson or Example01.ndjson.

curl

To import HL7v2 messages into an HL7v2 store, make a POST request and specify the following information:

  • The name of the parent dataset
  • The name of the HL7v2 store
  • The location of the object in a Cloud Storage bucket
  • An access token

The following sample shows how to import a single file using a POST request using curl.

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    --data "{
      'gcsSource': {
        'uri': 'gs://BUCKET/DIRECTORY/HL7V2_MESSAGE_FILE'
      }
    }" "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/hl7V2Stores/HL7V2_STORE_ID:import"

If the request is successful, the server returns the response in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"
}

The response contains an operation name. To track the status of the operation, you can use the Operation get method:

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"

If the request is successful, the server returns a response with the status of the operation in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1beta1.hl7v2.Hl7V2Service.ImportMessages",
    "createTime": "CREATE_TIME",
    "endTime": "END_TIME"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.hl7v2.ImportMessagesResponse"
  }
}

PowerShell

To import HL7v2 messages into an HL7v2 store, make a POST request and specify the following information:

  • The name of the parent dataset
  • The name of the HL7v2 store
  • The location of the object in a Cloud Storage bucket
  • An access token

The following sample shows how to import a single file using a POST request using Windows PowerShell.

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Post `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -Body "{
    'gcsSource': {
      'uri': 'gs://BUCKET/DIRECTORY/HL7V2_MESSAGE_FILE'
    }
  }" `
  -Uri "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/hl7V2Stores/HL7V2_STORE_ID:import" | Select-Object -Expand Content

If the request is successful, the server returns the response in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"
}

The response contains an operation name. To track the status of the operation, you can use the Operation get method:

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Get `
  -Headers $headers `
  -Uri "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID" | Select-Object -Expand Content

If the request is successful, the server returns a response with the status of the operation in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1beta1.hl7v2.Hl7V2Service.ImportMessages",
    "createTime": "CREATE_TIME",
    "endTime": "END_TIME"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.hl7v2.ImportMessagesResponse"
  }
}

Exporting HL7v2 messages

When you export HL7v2 messages from an HL7v2 store, all messages in the HL7v2 store are exported. To export only a subset of these messages, you can define startTime and endTime parameters to include only messages sent within the defined time span. To export only specific parts of the Message resource, set the MessageView.

When messages are exported, the Cloud Healthcare API creates an object for each HL7v2 message. Each object consists of an .ndjson file containing a Message resource on each line. Messages are exported in ascending order based on the sendTime (MSH.7).

The following samples show how to export HL7v2 messages to Cloud Storage using the projects.locations.datasets.hl7V2Stores.messages.export method.

Note the following when calling the export operation:

  • Write to a Cloud Storage bucket or directory, rather than an object, because the Cloud Healthcare API might create multiple newline-delimited JSON files when there are many messages. In each JSON file, each line is an HL7v2 message.
  • If the command specifies a directory that does not exist, the directory is created.

curl

To export HL7v2 messages, make a POST request and specify the following information:

  • The name of the parent dataset
  • The name of the HL7v2 store
  • The destination Cloud Storage bucket or directory
  • An access token

The following sample shows a POST request using curl.

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    --data "{
      'gcsDestination': {
        'uriPrefix': 'gs://BUCKET/DIRECTORY'
      }
    }" "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/hl7V2Stores/HL7V2_STORE_ID/messages:export"

If the request is successful, the server returns the response in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"
}

The response contains an operation name. To track the status of the operation, you can use the Operation get method:

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"

If the request is successful, the server returns a response with the status of the operation in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1beta1.hl7v2.Hl7V2Service.ExportMessages",
    "createTime": "CREATE_TIME",
    "endTime": "END_TIME",
    "counter": {
      "success": "RESOURCE_COUNT"
    }
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.hl7v2.ExportMessagesResponse"
  }
}

PowerShell

To export HL7v2 messages, make a POST request and specify the following information:

  • The name of the parent dataset
  • The name of the HL7v2 store
  • The destination Cloud Storage bucket or directory
  • An access token

The following sample shows a POST request using Windows PowerShell.

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Post `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -Body "{
    'gcsDestination': {
      'uriPrefix': 'gs://BUCKET/DIRECTORY'
    }
  }" `
  -Uri "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/hl7V2Stores/HL7V2_STORE_ID/messages:export" | Select-Object -Expand Content

If the request is successful, the server returns the response in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"
}

The response contains an operation name. To track the status of the operation, you can use the Operation get method:

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Get `
  -Headers $headers `
  -Uri "https://healthcare.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID" | Select-Object -Expand Content

If the request is successful, the server returns a response with the status of the operation in JSON format:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1beta1.hl7v2.Hl7V2Service.ExportMessages",
    "createTime": "CREATE_TIME",
    "endTime": "END_TIME",
    "counter": {
      "success": "RESOURCE_COUNT"
    }
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.hl7v2.ExportMessagesResponse"
  }
}

Troubleshooting HL7v2 import and export requests

If errors occur during an HL7v2 import or export request, the errors are logged to Cloud Logging. For more information, see Viewing error logs in Cloud Logging.