Refresh healthcare data

After the initial import of data into your Vertex AI Search healthcare data store, you might have performed any of the following updates in your source FHIR store:

  • Added new FHIR resources
  • Updated existing FHIR resources
  • Deleted FHIR resources

In such cases, you can reconcile the changes from your source FHIR store into your Vertex AI Search healthcare data store.

Reconciliation overview

You can reconcile the changes either incrementally or fully. The two modes are compared in the following table.

Changes to the source FHIR store Incremental mode Full mode
New FHIR resources Adds new documents to the Vertex AI Search data store Adds new documents to the Vertex AI Search data store
Updated FHIR resources Replaces the existing documents in the Vertex AI Search data store while retaining the document ID Replaces the existing documents in the Vertex AI Search data store while retaining the document ID
Deleted FHIR resources Doesn't reconcile Removes the corresponding documents from your Vertex AI Search data store

Before you begin

Review the quotas and limits for your Google Cloud project. Your Vertex AI Search healthcare data store can contain a maximum of 1 million documents per project. If this quota is reached during the import, the import process stops.

Perform an incremental import

The following sample shows how to import incremental changes from a Cloud Healthcare API FHIR store using the documents.import method.

REST

  1. Perform an incremental import.

    curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-H "X-Goog-User-Project: PROJECT_ID" \
"https://us-discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/us/dataStores/DATA_STORE_ID/branches/0/documents:import" \
-d '{
   "reconciliation_mode": "INCREMENTAL",
   "fhir_store_source": {"fhir_store": "projects/PROJECT_ID/locations/CLOUD_HEALTHCARE_DATASET_LOCATION/datasets/CLOUD_HEALTHCARE_DATASET_ID/fhirStores/FHIR_STORE_ID"}
}'

    Replace the following:

    • PROJECT_ID: the ID of your Google Cloud project.
    • DATA_STORE_ID: the ID of the Vertex AI Search data store.
    • CLOUD_HEALTHCARE_DATASET_ID: the ID of the Cloud Healthcare API dataset that contains the source FHIR store.
    • CLOUD_HEALTHCARE_DATASET_LOCATION: the location of the Cloud Healthcare API dataset that contains the source FHIR store.
    • FHIR_STORE_ID: the ID of the Cloud Healthcare API FHIR R4 store.

  2. Verify whether the FHIR data import operation is complete.

    curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://us-discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/us/collections/default_collection/dataStores/DATA_STORE_ID/branches/0/operations/IMPORT_OPERATION_ID"

    Replace the following:

    • PROJECT_ID: the ID of your Google Cloud project.
    • DATA_STORE_ID: the ID of the Vertex AI Search data store.
    • IMPORT_OPERATION_ID: the operation ID of the long-running operation that's returned when you call the import method.

Python

For more information, see the Vertex AI Agent Builder Python API reference documentation.

To authenticate to Vertex AI Agent Builder, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

from google.api_core.client_options import ClientOptions
from google.cloud import discoveryengine

# TODO(developer): Uncomment these variables before running the sample.
# project_id = "YOUR_PROJECT_ID"
# location = "YOUR_LOCATION" # Values: "us"
# data_store_id = "YOUR_DATA_STORE_ID"
# healthcare_project_id = "YOUR_HEALTHCARE_PROJECT_ID"
# healthcare_location = "YOUR_HEALTHCARE_LOCATION"
# healthcare_dataset_id = "YOUR_HEALTHCARE_DATASET_ID"
# healthcare_fihr_store_id = "YOUR_HEALTHCARE_FHIR_STORE_ID"

#  For more information, refer to:
# https://cloud.google.com/generative-ai-app-builder/docs/locations#specify_a_multi-region_for_your_data_store
client_options = (
    ClientOptions(api_endpoint=f"{location}-discoveryengine.googleapis.com")
    if location != "global"
    else None
)

# Create a client
client = discoveryengine.DocumentServiceClient(client_options=client_options)

# The full resource name of the search engine branch.
# e.g. projects/{project}/locations/{location}/dataStores/{data_store_id}/branches/{branch}
parent = client.branch_path(
    project=project_id,
    location=location,
    data_store=data_store_id,
    branch="default_branch",
)

request = discoveryengine.ImportDocumentsRequest(
    parent=parent,
    fhir_store_source=discoveryengine.FhirStoreSource(
        fhir_store=f"projects/{healthcare_project_id}/locations/{healthcare_location}/datasets/{healthcare_dataset_id}/fhirStores/{healthcare_fihr_store_id}",
    ),
    # Options: `FULL`, `INCREMENTAL`
    reconciliation_mode=discoveryengine.ImportDocumentsRequest.ReconciliationMode.INCREMENTAL,
)

# Make the request
operation = client.import_documents(request=request)

print(f"Waiting for operation to complete: {operation.operation.name}")
response = operation.result()

# Once the operation is complete,
# get information from operation metadata
metadata = discoveryengine.ImportDocumentsMetadata(operation.metadata)

# Handle the response
print(response)
print(metadata)