Managing long-running operations

This page describes how to manage the lifecycle of a Cloud Healthcare API long-running operation.

Long-running operations (LROs) are returned when method calls might take a long time to complete. The Cloud Healthcare API creates LROs for several methods. An example is the method to create a dataset, projects.locations.datasets.create. When projects.locations.datasets.create is called, the Cloud Healthcare API creates an LRO to track the dataset creation status. The Cloud Healthcare API provides operations APIs that let you check the status of LROs. You can also list LROs.

LROs are managed at the dataset level. If you are calling a method that returns an LRO, like projects.locations.datasets.dicomStores.import, then to view the LRO's status, make a request to the DICOM store's parent dataset.

In addition to the REST API, the following sources generate long-running operations when you call the methods listed in Methods that return a long-running operation:

• The Google Cloud CLI
• The Cloud Healthcare API page in the Google Cloud console

You can manage your Cloud Healthcare API LROs using the Google Cloud console, the Google Cloud CLI, or the REST API.

Note that the record of an LRO is kept for approximately 30 days after the LRO finishes, meaning that you cannot view or list an LRO after that point.

Methods that return a long-running operation

The following methods return an LRO:

Dataset operations:

• projects.locations.datasets.create
• projects.locations.datasets.deidentify

DICOM operations:

• projects.locations.datasets.dicomStores.import
• projects.locations.datasets.dicomStores.export
• projects.locations.datasets.dicomStores.studies.delete
• projects.locations.datasets.dicomStores.studies.series.delete

FHIR operations:

• projects.locations.datasets.fhirStores.import
• projects.locations.datasets.fhirStores.export

Getting details about a long-running operation

The following samples show how to get details about an LRO.

Note that the version of the Cloud Healthcare API shown in the response when getting details about an LRO will be the same as the API version of the method that initiated the LRO.

Console

After calling a method using either the gcloud CLI or the API that returns an LRO, you can view details about the LRO in the Google Cloud console.

1. In the Google Cloud console, go to the Datasets page.

   Go to Datasets

2. Click the name of the dataset containing the long-running operation you want to view.
3. Click Operations.
4. To view error logs for the operation in Cloud Logging, click Actions and then click View details in Cloud Logging. For more information, see Viewing error logs in Cloud Logging.
5. To view more details about the operation, click a running operation ID. The Long-running Operation Details page is displayed. This page shows the following information:
   • The operation's progress
   • The operation's details, such as operation ID and method
   • All log entries

gcloud

Suppose that you receive the following response after calling gcloud healthcare datasets deidentify:

Request issued for: [DATASET_ID]
Waiting for operation [OPERATION_ID] to complete...

The response shows that the Cloud Healthcare API created an LRO with an operation ID. The command continues to run until it finishes, at which point it outputs the following:

Request issued for: [DATASET_ID]
Waiting for operation [OPERATION_ID] to complete...done
name: projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID

To get details about an LRO, run the gcloud healthcare operations describe command, specifying the dataset ID and the operation ID.

gcloud healthcare operations describe OPERATION_ID --dataset=DATASET_ID

You can also retrieve the operation ID by listing long-running database operations.

If the request is successful, the command prompt displays the operation details:

done: true
metadata:
'@type': type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata
apiMethodName: google.cloud.healthcare.v1.dataset.DatasetService.DeidentifyDataset
createTime: 'CREATE_TIME'
endTime: 'END_TIME'
name: projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID
response:
'@type': type.googleapis.com/google.cloud.healthcare.v1.dataset.DeidentifySummary

API

To get the status of and view details about an LRO, use the projects.locations.datasets.operations.get method.

curl

To get details about an LRO, make a GET request and provide the name of the dataset, the operation ID, and an access token.

Suppose that you receive the following response after calling projects.locations.datasets.deidentify:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"
}

The name value in the response shows that the Cloud Healthcare API created an LRO named projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID.

The following sample shows how to use a GET request to get details about an LRO using curl.

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"

You can also retrieve the LRO name by listing long-running database operations.

If the request is successful, the server returns the response in JSON format. When an LRO finishes, the done field will be set to true.

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1.dataset.DatasetService.DeidentifyDataset",
    "createTime": "CREATE_TIME",
    "endTime": "END_TIME",
    "logsUrl": "https://console.cloud.google.com/logs/query/CLOUD_LOGGING_URL"
  },
  "done": true,
  "response": {
    "@type": "...",
    "successStoreCount": "SUCCESS_STORE_COUNT"
  }
}

To poll an LRO, repeatedly invoke the projects.locations.datasets.operations.get method until the operation finishes. Use a backoff between each poll request, such as 10 seconds.

PowerShell

To get details about an LRO, make a GET request and provide the name of the dataset, the operation ID, and an access token.

Suppose that you receive the following response after calling projects.locations.datasets.deidentify:

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"
}

The name value in the response shows that the Cloud Healthcare API created an LRO named projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID.

The following sample shows how to use a GET request to get details about an LRO using Windows PowerShell.

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Get `
  -Headers $headers `
  -Uri "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID" | Select-Object -Expand Content

You can also retrieve the LRO name by listing long-running database operations.

If the request is successful, the server returns the response in JSON format. When an LRO finishes, the done field will be set to true.

{
  "name": "projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1.dataset.DatasetService.DeidentifyDataset",
    "createTime": "CREATE_TIME",
    "endTime": "END_TIME",
    "logsUrl": "https://console.cloud.google.com/logs/query/CLOUD_LOGGING_URL"
  },
  "done": true,
  "response": {
    "@type": "...",
    "successStoreCount": "SUCCESS_STORE_COUNT"
  }
}

To poll an LRO, repeatedly invoke the projects.locations.datasets.operations.get method until the operation finishes. Use a backoff between each poll request, such as 10 seconds.

Listing long-running operations

The following samples show how to list the long-running operations in a dataset.

Console

After calling methods using gcloud CLI or the API that return an LRO, you can view a list of LROs in the Google Cloud console.

1. In the Google Cloud console, go to the Datasets page.

   Go to Datasets

2. Select the dataset containing the long-running operation you want to view.
3. Click Operations.

A list of LROs in the dataset and their status displays. To view the error logs in Cloud Logging, click the more information icon in the last column and then click View details in Cloud Logging. For more information, see Viewing error logs in Cloud Logging.

gcloud

To list the LROs in a dataset, run the gcloud healthcare operations list command.

gcloud healthcare operations list \
  --dataset=DATASET_ID \
  --location=LOCATION

If the request is successful, the command prompt lists the LROs:

ID                    LOCATION     DONE
OPERATION_ID          LOCATION       {TRUE|FALSE}
...

API

To list the LROs in a dataset, use the projects.locations.datasets.operations.list method.

curl

To list the LROs in a dataset, make a GET request and provide the dataset name and an access token. The following sample shows a GET request using curl.

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations"

If the request is successful, the server returns the response in JSON format:

{
  "operations": [
    {
      "name": "PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
      "metadata": {
        "@type": "type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata",
        "apiMethodName": "google.cloud.healthcare.v1.dataset.DatasetService.DeidentifyDataset",
        "createTime": "CREATE_TIME",
        "endTime": "END_TIME",
        "logsUrl": "https://console.cloud.google.com/logs/query/CLOUD_LOGGING_URL"
      },
      "done": true,
      "response": {
        "@type": "...",
        "successStoreCount": "SUCCESS_STORE_COUNT"
      }
    },
    ...
  ]
}

PowerShell

To list the LROs in a dataset, make a GET request and provide the dataset name and an access token. The following sample shows a GET request using Windows PowerShell.

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Get `
  -Headers $headers `
  -Uri "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations" | Select-Object -Expand Content

If the request is successful, the server returns the response in JSON format:

{
  "operations": [
    {
      "name": "PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
      "metadata": {
        "@type": "type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata",
        "apiMethodName": "google.cloud.healthcare.v1.dataset.DatasetService.DeidentifyDataset",
        "createTime": "CREATE_TIME",
        "endTime": "END_TIME",
        "logsUrl": "https://console.cloud.google.com/logs/query/CLOUD_LOGGING_URL"
      },
      "done": true,
      "response": {
        "@type": "...",
        "successStoreCount": "SUCCESS_STORE_COUNT"
      }
    },
    ...
  ]
}

Cancelling long-running operations

The following samples show how to cancel an LRO in a dataset.

Important: Cancelling an LRO does not roll back any part of the operation that has already completed.

Console

After calling a method using either the gcloud CLI or the API that returns an LRO, you can cancel it in the Google Cloud console while it is still being processed.

1. In the Google Cloud console, go to the Datasets page.

   Go to Datasets

2. Select the dataset containing the long-running operation you want to cancel.
3. Click Operations.
4. For the relevant LRO, select Stop Operation from the more information menu in the last column.

API

To cancel the LROs in a dataset, use the projects.locations.datasets.operations.cancel method.

curl

To cancel an LRO in a dataset, make a POST request and specify the following information:

• The name of the dataset
• An access token
• The operation ID
• The cancel parameter

The following sample shows a POST request using curl.

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID:cancel"

Important: A cancelled operation can stay in a cancel requested state for a period of time before it is completely cancelled.

To view the status of a cancel request, make a GET request and specify the following information:

• The name of the dataset
• An access token
• The operation ID

The following sample shows a GET request using curl.

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID"

If the request is successful, the server returns the response in JSON format:

{
  "name": "PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata",
    "apiMethodName": "API_METHOD_NAME",
    "createTime": "OPERATION_START_TIME",
    "endTime": "OPERATION_END_TIME",
    "cancelRequested": true,
    "logsUrl": "https://console.cloud.google.com/logs/query/CLOUD_LOGGING_URL",
    "counter": {
      "success": "SUCCESS_COUNT"
    }
  },
  "done": true,
  "error": {
    "code": 1,
    "message": "Operation cancelled by user."
  }
}

PowerShell

To cancel an LRO in a dataset, make a POST request and specify the following information:

• The name of the dataset
• An access token
• The operation ID
• The cancel parameter

The following sample shows a POST request using Windows PowerShell.

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Post `
  -Headers $headers `
  -Uri "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID:cancel" | Select-Object -Expand Content

Important: A cancelled operation can stay in a cancel requested state for a period of time before it is completely cancelled.

To view the status of a cancel request, make a GET request and specify the following information:

• The name of the dataset
• An access token
• The operation ID

The following sample shows a GET request using Windows PowerShell.

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Get `
  -Headers $headers `
  -Uri "https://healthcare.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID" | Select-Object -Expand Content

If the request is successful, the server returns the response in JSON format:

{
  "name": "PROJECT_ID/locations/LOCATION/datasets/DATASET_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1.OperationMetadata",
    "apiMethodName": "API_METHOD_NAME",
    "createTime": "OPERATION_START_TIME",
    "endTime": "OPERATION_END_TIME",
    "cancelRequested": true,
    "logsUrl": "https://console.cloud.google.com/logs/query/CLOUD_LOGGING_URL",
    "counter": {
      "success": "SUCCESS_COUNT"
    }
  },
  "done": true,
  "error": {
    "code": 1,
    "message": "Operation cancelled by user."
  }
}

Cancelling multiple long-running operations

To cancel multiple long-running operations, complete the following steps:

1. Call the operations.list method to get the names of the operations in a dataset.
2. Call the operations.cancel method on each operation.

Google provides a Python script that you can use to cancel all operations for a given dataset.

Troubleshooting long-running operations

When a long-running operation fails, its response includes a gRPC code. The following table provides an explanation of the cause for each code:

Code	Status	Description
0	OK	Returned on Success.
1	CANCELLED	The operation was cancelled, typically by the caller.
2	UNKNOWN	This error might be returned when a Status value received from another address space belongs to an error space that is not known in this address space. If an API's error does not return enough information, the error might be converted to this error.
3	INVALID_ARGUMENT	The client specified an invalid argument. This error differs from FAILED_PRECONDITION. INVALID_ARGUMENT indicates arguments that are problematic regardless of the state of the system, such as a malformed file name.
4	DEADLINE_EXCEEDED	The deadline expired before the operation could complete. For operations that change the state of the system, this error may be returned even if the operation has completed successfully. For example, a successful response from a server could have been delayed long enough for the deadline to expire.
5	NOT_FOUND	Some requested entity, such as a file or directory, was not found.
6	ALREADY_EXISTS	The entity that a client attempted to create, such as a file or directory, already exists.
7	PERMISSION_DENIED	The caller does not have permission to execute the specified operation. This error code does not imply the request is valid or the requested entity exists or satisfies other pre-conditions.
8	RESOURCE_EXHAUSTED	Some resource has been exhausted, perhaps a per-user quota, or perhaps the entire file system is out of space.
9	FAILED_PRECONDITION	The operation was rejected because the system is not in a state required for the operation's execution. For example, the directory to be deleted is non-empty, or an rmdir operation is applied to a non-directory.
10	ABORTED	The operation was aborted, typically due to a concurrency issue such as a sequencer check failure or transaction abort.
11	OUT_OF_RANGE	The operation was attempted past the valid range, such as seeking or reading past end-of-file. Unlike INVALID_ARGUMENT, this error indicates a problem that may be fixed if the system state changes.
12	UNIMPLEMENTED	The operation isn't implemented or isn't supported/enabled in this service.
13	INTERNAL	Internal errors. This indicates that an unexpected error was encountered in processing on the underlying system.
14	UNAVAILABLE	The service is currently unavailable. This is most likely a transient condition, which can be corrected by retrying with a backoff. Note that it is not always safe to retry non-idempotent operations.
15	DATA_LOSS	Unrecoverable data loss or corruption.
16	UNAUTHENTICATED	The request doesn't have valid authentication credentials for the operation.