Workflows is a fully-managed orchestration platform that executes services in an order that you define: a workflow. These workflows can combine services including custom services hosted on Cloud Run or Cloud Functions, Google Cloud services such as Cloud Vision AI and BigQuery, and any HTTP-based API.
By incorporating Workflows into solutions, you can make service dependencies explicit and observable end-to-end. A workflow that specifies an application, operational, or business process provides a source-of-truth or canonical narrative for the process.
Workflows is serverless, scaling up as needed, and no charges are incurred while idle. Because a workflow contains no code or library dependencies, it does not require security patches. Once you deploy a workflow, you can expect it to reliably execute without maintenance. A workflow can hold state, retry, poll, or wait for up to a year.
Workflows is compliant with these certifications and standards.
The following diagram shows an example of using Workflows to orchestrate services:
Key use cases
Workflows supports many use cases. Some examples are:
|Service orchestration||Create solutions by combining services—Perform a sequence of
operations across multiple systems, waiting for all operations to complete.
Can be event-driven. For example:
|Batch jobs||Operate on multiple items—Perform operations on a set of items or
batch data. Often scheduled. For example:
|Business processes||Automate line-of-business workflows—Encode the steps in a
business process, including conditions, actions, and human-in-the-loop
events. For example:
|IT process automation||Managed execution of service operations—Easily script sequences
of Google Cloud service operations. For example:
A workflow consists of a series of steps described using the Workflows syntax, and can be written in either YAML or JSON. This is the workflow's definition. For a detailed explanation of the Workflows syntax, see the Syntax reference.
After a workflow is created, it is deployed, which makes the workflow ready for execution. Learn how to create and update a workflow in the Google Cloud console or by using the Google Cloud CLI. You can also create your workflow by using a preferred IDE or source code editor and setting up autocompletion and syntax validation.
An execution is a single run of the logic contained in a workflow's
definition. A workflow that hasn't been executed generates no charges. All
workflow executions are independent, and the product's rapid scaling allows for
a high number of concurrent executions. You can execute a workflow using the
client libraries, in the Google Cloud console, using the Google Cloud CLI,
or by sending an HTTP
POST request to the workflow's invocation URL using
the Workflows REST API. For details, see
Execute a workflow.
The following are some of the key capabilities of Workflows.
- To create a workflow, you define the steps you want and their order of execution using the Workflows syntax. Every workflow must have at least one step. By default, Workflows treats steps as if they are in an ordered list and executes them one at a time until all the steps have run. For details, see Steps.
- You can use a
switchblock as a selection mechanism that allows the value of an expression to control the flow of a workflow's execution. For details, see Conditions.
- You can use a
forloop to iterate over a sequence of numbers or through a collection of data, such as a list or map. For details, see Iteration.
- Parallel steps
- You can use a
parallelstep to define a part of your workflow (a branch or a loop) that can execute concurrently. For details, see Parallel steps.
- A subworkflow works similarly to a routine or function in a programming language, allowing you to encapsulate a step or set of steps that your workflow will repeat multiple times. For details, see Subworkflows.
- You can manage workflows from either the Google Cloud console or from the command line using the Google Cloud CLI. Visualization support while editing the Workflows syntax is also available through the Google Cloud console.
- The Cloud Client Libraries for the Workflows API, or the REST API, can be used to manage workflows. For details, see Workflows APIs and reference.
- You can use Cloud Scheduler to run a workflow on a particular schedule, such as every Monday at 9 AM or every 15 minutes. For details, see Schedule a workflow using Cloud Scheduler.
- Runtime arguments
- Data passed at runtime can be accessed by adding a
paramsfield to your main workflow (placed in a
mainblock accepts a single argument that is any valid JSON data type. The
paramsfield names the variable that the workflow uses to store the data you pass in. For details, see Runtime arguments.
- HTTP APIs
- You can define a workflow step that makes an HTTP call and assign the response from the call to a variable. For example, you can invoke a Google Cloud service such as Cloud Functions or Cloud Run through an HTTP request. Both HTTP and HTTPS requests are supported. For details, see Make an HTTP request and Invoke Cloud Functions or Cloud Run.
- You can also invoke private on‑premises, Compute Engine, Google Kubernetes Engine (GKE), or other Google Cloud endpoints. For details, see Invoke private on‑prem, Compute Engine, GKE, or other endpoint.
- Workflows publishes connectors that can be used to connect to other Google Cloud APIs within a workflow, and to integrate your workflows with those Google Cloud products. They simplify calling services because they handle the formatting of requests for you, and provide methods and arguments so that you don't need to know the details of a Google Cloud API. For details, see Understand connectors.
- Standard library and environment variables
- The Workflows standard library and built-in environment variables allow you to efficiently construct arguments for services and process responses.
- The standard library includes modules and frequently used functions, such as for data type and format conversions. There is no need to import or load libraries in a workflow—library functions work immediately. For details, see the Standard library overview.
- You can access a workflow's environment information (such as its location or project identifier) using built-in environment variables. Built-in environment variables require no declaration and are available in every workflow execution. For details, see Built-in environment variables.
You can make your workflows resilient and customize their behavior when a failure occurs by using Workflows' exception handling, including automated HTTP call retries with exponential back-offs, custom error handlers, and other advanced features. For details, see Workflow errors.
Callbacks allow workflow executions to wait for another service to make a request to the callback endpoint; that request resumes the execution of the workflow. With callbacks, you can signal to your workflow that a specified event has occurred, and wait on that event without polling. For details, see Wait using callbacks.
You can pause the execution of a workflow by adding a sleep step to your
workflow's definition. You can then use
sys.sleep to poll for data over a
given interval. For details, see Wait using polling.
Authentication and access control
Since every workflow execution requires an authenticated call, you can mitigate the risk of accidental or malicious calls by using Workflows. You can also simplify interactions with other Google Cloud APIs by using IAM-based service accounts, and you can securely store keys and passwords for authenticating to external APIs by using the Secret Manager connector. For details, learn more about authentication and access control.
Workflows automatically generates execution logs for workflow executions in Cloud Logging. You can also control when logs are sent to Logging during a workflow execution through call logging or custom logs. For details, see Send logs to Cloud Logging.
Audit logging information is available through Cloud Audit Logs. For details, see Workflows audit logging information.
You can find many useful Workflows code samples on the samples page.
Learn more about creating and updating workflows.
Learn how to control the order in which a workflow's steps are run.