Monitor long-running operations

This page describes how to manage the lifecycle of a long-running operation (LRO) in Vertex AI Agent Builder.

A long-running operation object is returned when a call to a method might take a long time to complete. For example, the Vertex AI Agent Builder API creates a long-running operation when you call documents.import through the API or Client Libraries. The operation tracks the status of the processing job.

You can use the long-running operations methods that the Vertex AI Agent Builder API provides to check the status of the operations. You can also list or poll operations.

The record of an operation is kept for approximately 30 days after the operation finishes, meaning that you can't view or list an operation after that time.

Get details about a long-running operation

The following show how to get details about an operation.

REST

To get the status of and view details about a long-running operation, follow these steps:

  1. Find the name of the operation in one of two ways:

    • After you've made a call to a method that returns a long-running operation, review the response.

      For example, if you call documents.import, the start of the response looks something like this:

      {
  "name": "projects/12345/locations/global/collections/default_collection/dataStores/my-datastore_4321/branches/0/operations/import-documents-56789",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.discoveryengine.v1beta.ImportDocumentsMetadata"
  }
}

      The name value in the response provides the operation name, which can be used to query for the operation status. Don't include the quotes when you copy the operation name.

    • Get the operation name by listing long-running operations.

  2. Call the operations.get method on the resource that created the operation:

    curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://discoveryengine.googleapis.com/v1beta/OPERATION_NAME"

    OPERATION_NAME: The name of the operation from the previous step.

    The first lines of the response from the GET command look something like this:

    {
  "operations": [
    {
      "name": "projects/12345/locations/global/collections/default_collection/dataStores/my-datastore_4321/branches/0/operations/import-documents-56789",
      "metadata": {
        "@type": "type.googleapis.com/google.cloud.discoveryengine.v1alpha.ImportDocumentsMetadata",
      }
    }
  ]
}

Python

For more information, see the Vertex AI Agent Builder Python API reference documentation.

To authenticate to Vertex AI Agent Builder, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

from google.cloud import discoveryengine
from google.longrunning import operations_pb2

# TODO(developer): Uncomment these variables before running the sample.
# Example: `projects/{project}/locations/{location}/collections/{default_collection}/dataStores/{search_engine_id}/branches/{0}/operations/{operation_id}`
# operation_name = "YOUR_OPERATION_NAME"


def get_operation_sample(operation_name: str) -> operations_pb2.Operation:
    # Create a client
    client = discoveryengine.DocumentServiceClient()

    # Make GetOperation request
    request = operations_pb2.GetOperationRequest(name=operation_name)
    operation = client.get_operation(request=request)

    # Print the Operation Information
    print(operation)

    return operation

List long-running operations

The following show how to list the operations for a Google Cloud resource.

REST

To list the long-running operations for a Google Cloud resource, follow this step:

  • Call the operations.list method:

    curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/operations"

    DATA_STORE_ID: The ID of the Vertex AI Agent Builder data store that was created with your engine. In the Google Cloud console URL, the data store ID appears after engines/ and before /data.

Python

For more information, see the Vertex AI Agent Builder Python API reference documentation.

To authenticate to Vertex AI Agent Builder, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

from typing import Optional

from google.cloud import discoveryengine
from google.longrunning import operations_pb2

# TODO(developer): Uncomment these variables before running the sample.
# project_id = "YOUR_PROJECT_ID"
# location = "YOUR_PROCESSOR_LOCATION"  # Options: "global"
# search_engine_id = "YOUR_SEARCH_ENGINE_ID"

# Create filter in https://google.aip.dev/160 syntax
# operations_filter = "YOUR_FILTER"


def list_operations_sample(
    project_id: str,
    location: str,
    search_engine_id: str,
    operations_filter: Optional[str] = None,
) -> operations_pb2.ListOperationsResponse:
    # Create a client
    client = discoveryengine.DocumentServiceClient()

    # The full resource name of the search engine branch.
    name = f"projects/{project_id}/locations/{location}/collections/default_collection/dataStores/{search_engine_id}"

    # Make ListOperations request
    request = operations_pb2.ListOperationsRequest(
        name=name,
        filter=operations_filter,
    )

    # Make ListOperations request
    response = client.list_operations(request=request)

    # Print the Operation Information
    for operation in response.operations:
        print(operation)

    return response

Poll a long-running operation

The following show how to poll the status of an operation.

REST

To poll the long-running operation until it finishes, follow these steps:

  1. Run the following command, which calls the operations.get method repeatedly, using a backoff of 10 seconds between each request:

    while true; \
    do curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://discoveryengine.googleapis.com/v1beta/OPERATION_NAME"; \
    sleep 10; \
    done

    OPERATION_NAME: The operation name, found in Get details about a long-running operation. For example: projects/12345/locations/global/collections/default_collection/dataStores/my-datastore_4321/branches/0/operations/import-documents-56789

  2. Stop the polling job (Control+Z) after the status shows "done": true.

Python

For more information, see the Vertex AI Agent Builder Python API reference documentation.

To authenticate to Vertex AI Agent Builder, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

from time import sleep

from google.cloud import discoveryengine
from google.longrunning import operations_pb2

# TODO(developer): Uncomment these variables before running the sample.
# Example: `projects/{project}/locations/{location}/collections/{default_collection}/dataStores/{search_engine_id}/branches/{0}/operations/{operation_id}`
# operation_name = "YOUR_OPERATION_NAME"


def poll_operation_sample(
    operation_name: str, limit: int = 10
) -> operations_pb2.Operation:
    # Create a client
    client = discoveryengine.DocumentServiceClient()

    # Make GetOperation request
    request = operations_pb2.GetOperationRequest(name=operation_name)

    for _ in range(limit):
        operation = client.get_operation(request=request)
        # Print the Operation Information
        print(operation)

        # Stop polling when Operation is no longer running
        if operation.done:
            break

        # Wait 10 seconds before polling again
        sleep(10)

    return operation