Understand connectors

Workflows publishes connectors to make it easier to access other Google Cloud products within a workflow. For a detailed reference of available connectors, see the Connectors reference.

Connectors can be used to connect to other Google Cloud APIs within a workflow, and to integrate your workflows with those Google Cloud products. For example, you can use connectors to publish Pub/Sub messages, read or write data to a Firestore database, or retrieve authentication keys from Secret Manager.

Connectors simplify calling services because they handle the formatting of requests for you, providing methods and arguments so that you don't need to know the details of a Google Cloud API.

Connectors have built-in behavior for handling retries and long-running operations.

Authentication

Connectors use built-in Identity and Access Management authentication to interact with other Google Cloud products. Note that a workflow using a connector still needs to have proper permissions assigned to its service account to properly authenticate to a given API. For example, to publish a message to Pub/Sub, the service account associated with your workflow needs to be granted the roles/pubsub.publisher role. To learn more about authenticating to other Google Cloud products, see Grant a workflow permission to access Google Cloud resources.

Retries

All connectors apply the following retry policy and error handling configurations:

  • Workflows uses an idempotent retry policy for GET requests.

  • Workflows uses a non-idempotent retry policy for all other HTTP methods.

  • By default, requests time out after 30 minutes. Note that for long-running operations, this applies to individual requests (for example, starting the operation or checking its status) within the long-running operation. The timeout for the entire operation can be configured using the connector_params field. See the Connectors reference.

Long-running operations

For calls performing a long-running operation, the following behaviors apply:

  • Workflows blocks the workflow execution until the operation completes successfully or fails.

  • By default, calls time out after 30 minutes. The timeout of the call can be changed using the connector_params field (to a maximum of 1 year). See the Connectors reference.

  • Connectors poll long-running operations periodically. By default, Workflows uses an exponential backoff of 1.25 when polling, starting with 1 second and slowing to 60 seconds between polls. Each polling attempt counts as a billable step. The polling parameters can be changed using the connector_params field. See the Connectors reference.

  • The preceding retry mechanism applies to a failed polling attempt.

  • When an error occurs during a long-running operation, Workflows raises an OperationError error. The error includes an additional attribute, error, which contains operation-specific details.

If a long-running operation fails, the connector call throws an exception with the following attributes:

  • message: Human-readable error message.
  • tags: A list of tags. For a long-running operation connector call, the list includes OperationError.
  • operation: The failed long-running operation object.

For a detailed list of supported long-running operation types, see the Connectors reference.

What's next