You can create and manage workflows in the Google Cloud console, or by using the Google Cloud CLI in either your terminal or Cloud Shell. You can also manage workflows through the Workflows API.
Before you begin
Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, see Develop applications in a constrained Google Cloud environment.
Console
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Workflows API.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Workflows API.
gcloud
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
Create or select a Google Cloud project.
-
Create a Google Cloud project:
gcloud projects create PROJECT_ID
Replace
PROJECT_ID
with a name for the Google Cloud project you are creating. -
Select the Google Cloud project that you created:
gcloud config set project PROJECT_ID
Replace
PROJECT_ID
with your Google Cloud project name.
-
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Workflows API:
gcloud services enable workflows.googleapis.com
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
Create or select a Google Cloud project.
-
Create a Google Cloud project:
gcloud projects create PROJECT_ID
Replace
PROJECT_ID
with a name for the Google Cloud project you are creating. -
Select the Google Cloud project that you created:
gcloud config set project PROJECT_ID
Replace
PROJECT_ID
with your Google Cloud project name.
-
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Workflows API:
gcloud services enable workflows.googleapis.com
REST
To manage workflows using the Workflows API, we recommend
that you use the Google-provided
client libraries to call the
workflows.googleapis.com
service. For more information, see
Workflows API.
Create a service account
A service account represents the identity of a workflow and determines what permissions the workflow has and which Google Cloud resources it can access. Create a service account if you don't already have one; then grant it the roles necessary for managing workflows and creating logs.
If you don't specify a service account during the workflow's creation, the workflow uses the default Compute Engine service account for its identity. For more information, see Grant a workflow permission to access Google Cloud resources.
We strongly recommend using a service account with the least privileges necessary to access the required resources.
Note that to create a resource and attach a service account, you need permissions to create that resource and to impersonate the service account that you will attach to the resource. For more information, see Service account permissions.
Console
In the Google Cloud console, go to the Service Accounts page.
Select a project and then click Create service account.
In the Service account name field, enter a name.
The name must be between 6 and 30 characters, and can contain lowercase alphanumeric characters and dashes. After you create a service account, you can't change its name.
Click Create and continue.
Click Select a role.
Select the following roles, clicking Add another role as required:
- To create, update, and execute workflows, select Workflows > Workflows Editor.
- To send logs to Cloud Logging, select Logging > Logs Writer.
Click Done to finish creating the service account.
gcloud
Create the service account.
gcloud iam service-accounts create SERVICE_ACCOUNT
Grant permissions to the service account by assigning roles.
To create, update, and execute workflows, assign the
roles/workflows.editor
role:gcloud projects add-iam-policy-binding PROJECT_ID \ --member "serviceAccount:SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com" \ --role "roles/workflows.editor"
To send logs to Cloud Logging, assign the
roles/logging.logWriter
role:gcloud projects add-iam-policy-binding PROJECT_ID \ --member "serviceAccount:SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com" \ --role "roles/logging.logWriter"
Replace the following:
SERVICE_ACCOUNT
: the name of the service account. It must be between 6 and 30 characters, and can contain lowercase alphanumeric characters and dashes. After you create a service account, you can't change its name.PROJECT_ID
: your project's ID.
REST
You can use the
serviceAccounts.create
method to create a service account. For more information, see
Create service accounts.
You can programmatically grant multiple roles by modifying and setting the
allow policy for a resource using the
setIamPolicy
method. For more information, see
Grant or revoke multiple roles programmatically.
Create a workflow
A workflow definition is made up of a series of steps described using the Workflows syntax, which can be written in either YAML or JSON format. After defining a workflow, you deploy it to make it available for execution. The deploy step also validates that the source file can be executed. It fails if the source file doesn't contain a valid workflow definition.
Editing YAML files can be error-prone. You can create your workflow by using a preferred IDE or source code editor and setting up autocompletion and syntax validation to reduce errors.
Console
In the Google Cloud console, go to the Workflows page.
Click
Create.Enter a name for the workflow, such as
myFirstWorkflow
. The name can contain letters, numbers, underscores, and hyphens. It must start with a letter and end with a number or letter.Optionally, add a description of the workflow.
In the Region list, select an appropriate location to deploy your workflow in; for example, us-central1.
In the Service account list, select a service account that your workflow will use to access other Google Cloud services. For more information, in this document, see Create a service account.
Optionally, do any of the following:
Specify the level of call logging that you want to apply to the workflow definition. In the Call log level list, select one of:
- Not specified: no logging level is specified. This is the default. An execution log level takes precedence over any workflow log level, unless the execution log level is not specified (the default); in that case, the workflow log level applies.
- Errors only: log all caught exceptions; or when a call is stopped due to an exception.
- All calls: log all calls to subworkflows or library functions and their results.
- No logs: no call logging.
Specify a Cloud Key Management Service key that the workflow should use for data encryption: select Customer-managed encryption key (CMEK). For more information, see Use customer-managed encryption keys.
Specify an environment variable that is accessible by your workflow at runtime. For more information, see Use environment variables.
Add a label: labels are key-value pairs that help you organize your Google Cloud instances. For more information, see What are labels? Workflow labels are inherited by executions of the workflow. To view execution labels, use the
workflows.executions.list
method to list and filter workflow executions.Schedule your workflow: select Add new trigger > Cloud Scheduler. For more information, see Schedule a workflow using Cloud Scheduler.
Trigger your workflow through an event or Pub/Sub message: select Add new trigger > Eventarc. For more information, see Trigger a workflow with events or Pub/Sub messages.
Click Next.
In the workflow editor, enter the definition for your workflow. See an example workflow.
Click Deploy.
gcloud
Make sure your workflow's source code is saved in a YAML or JSON file, such as
MY_WORKFLOW.YAML
orMY_WORKFLOW.JSON
. See an example workflow.Open a terminal.
You can create and deploy a workflow using the
gcloud workflows deploy
command:gcloud workflows deploy WORKFLOW_NAME \ --location=LOCATION \ --call-log-level=CALL_LOGGING_LEVEL \ --description="DESCRIPTION" \ --labels="LABEL_KEY=LABEL_VALUE" \ --set-env-vars="ENV_KEY=ENV_VALUE" \ --kms-key=ENCRYPT_KEY \ --service-account=SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com \ --source=YAML_OR_JSON_SOURCE_FILE
Replace the following:
WORKFLOW_NAME
: the name of your workflow—for example,myFirstWorkflow
. The name can contain letters, numbers, underscores, and hyphens. It must start with a letter and end with a number or letter.LOCATION
: the region to deploy your workflow in; for example,us-central1
.CALL_LOGGING_LEVEL
: optional. Level of call logging to apply during execution. Can be one of:none
: no logging level is specified. This is the default. An execution log level takes precedence over any workflow log level, unless the execution log level is not specified (the default); in that case, the workflow log level applies.log-errors-only
: log all caught exceptions; or when a call is stopped due to an exception.log-all-calls
: log all calls to subworkflows or library functions and their results.log-none
: no call logging.
DESCRIPTION
: optional. A description of the workflow.LABEL_KEY=LABEL_VALUE
: optional. List of label key-value pairs that helps you organize your Google Cloud instances; for example,name=wrench
. For more information, see What are labels? Workflow labels are inherited by executions of the workflow. To view execution labels, use theworkflows.executions.list
method to list and filter workflow executions.ENV_KEY=ENV_VALUE
: optional. List of environment variable key-value pairs; for example,MONTH=January
. For more information, see Use environment variables.ENCRYPT_KEY
: optional. A Cloud KMS key that the workflow should use for data encryption in the formatprojects/PROJECT_NAME/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME
. For more information, see Use customer-managed encryption keys.SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com
: optional. The service account your workflow will use to access other Google Cloud services. For more information, in this document, see Create a service account.YAML_OR_JSON_SOURCE_FILE
: the source file for the workflow definition. For example:myFirstWorkflow.yaml
.
REST
To create a new workflow with a specified name, use the
projects.locations.workflows.create
method, and use the workflowId
query parameter to specify an ID for the
workflow.
Before using any of the request data, make the following replacements:
WORKFLOW_NAME
: the name of your workflow—for example,myFirstWorkflow
. The name can contain letters, numbers, underscores, and hyphens. It must start with a letter and end with a number or letter.DESCRIPTION
: optional. A description of your workflow. Must be no longer than 1000 Unicode characters.LABEL_KEY
andLABEL_VALUE
: optional. A map of label key and value pairs that help you organize your Google Cloud instances. For example:{"name": "wrench", "mass": "1kg", "count": "3"}
For more information, see What are labels? Workflow labels are inherited by executions of the workflow. To view execution labels, use theworkflows.executions.list
method to list and filter workflow executions.SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com
: optional. The service account your workflow will use to access other Google Cloud services. The project ID is your Google Cloud project ID. For more information, in this document, see Create a service account.ENCRYPT_KEY
: optional. A Cloud KMS key that the workflow should use for data encryption in the formatprojects/PROJECT_NAME/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME
. For more information, see Use customer-managed encryption keys.CALL_LOGGING_LEVEL
: optional. The call logging level to apply during execution. The default is that no logging level is specified and the workflow log level applies instead. For more information, see Send Logs to Logging. One of the following:CALL_LOG_LEVEL_UNSPECIFIED
: no logging level is specified and the workflow log level applies instead. This is the default. Otherwise, the execution log level applies and takes precedence over the workflow log level.LOG_ERRORS_ONLY
: log all caught exceptions; or when a call is stopped due to an exception.LOG_ALL_CALLS
: log all calls to subworkflows or library functions and their results.LOG_NONE
: no call logging.
ENV_KEY
andENV_VALUE
: optional. A map of environment variable key and value pairs; for example,{ "month": "January", "day": "Monday"}
. For more information, see Use environment variables.SOURCE_CODE
: the workflow definition. You must escape new lines in YAML. For example:main:\n params:\n - input\n steps:\n - returnOutput:\n return: Hello
You must escape quotation marks in JSON. For example:
{\"main\":{\"params\":[\"input\"],\"steps\":[{\"returnOutput\":{\"return\":\"Hello\"}}]}}
LOCATION
: the region in which the workflow is to be deployed—for example,us-central1
.
Request JSON body:
{ "name": "WORKFLOW_NAME", "description": "DESCRIPTION", "labels": {"LABEL_KEY":"LABEL_VALUE"}, "serviceAccount": "SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com", "cryptoKeyName": "ENCRYPT_KEY", "callLogLevel": "CALL_LOGGING_LEVEL", "userEnvVars": {"ENV_KEY":"ENV_VALUE"}, "sourceContents": "SOURCE_CODE" }
To send your request, expand one of these options:
If successful, the response body contains a newly created instance of
Operation
:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.workflows.v1.OperationMetadata", "createTime": "2023-12-05T14:06:06.338390918Z", "target": "projects/PROJECT_ID/locations/LOCATION/workflows/WORKFLOW_NAME", "verb": "create", "apiVersion": "v1" }, "done": false }
If the value of "done"
is false
, the operation
is still in progress.
List workflows
You can list workflows or use filters to retrieve a specific workflow.
Console
In the Google Cloud console, go to the Workflows page.
This page lists your workflows in all locations, and includes details such as names, regions, latest revisions, and more.
To filter your workflows:
- Click Filter or the Filter workflows field.
- In the Properties list, select an option to filter the workflows by.
You can select a single property or use the logical operator
OR
to add more properties.To sort your workflows, beside any supported column heading, click
Sort.
gcloud
List workflows using the
gcloud workflows list
command:
gcloud workflows list --location=LOCATION
Replace LOCATION
with the ID or fully qualified
identifier for the location of your workflow.
This command lists your workflows in the specified location, and includes
details such as a workflow's NAME
, STATE
, REVISION_ID
, and
UPDATE_TIME
.
REST
To list workflows in a given project and location, use the
projects.locations.workflows.list
method.
Alternatively, to retrieve the details of a single workflow, use the
projects.locations.workflows.get
method.
Before using any of the request data, make the following replacements:
PROJECT_ID
: your Google Cloud project ID.LOCATION
: the region in which the workflows are deployed—for example,us-central1
.
To send your request, expand one of these options:
If successful, the response body contains instances of
Workflow
and the response should be similar to the following:
{ "workflows": [ { "name": "projects/PROJECT_ID/locations/LOCATION/workflows/WORKFLOW_NAME", "state": "ACTIVE", "revisionId": "000001-0ce", "createTime": "2023-12-08T13:56:59.306770745Z", "updateTime": "2023-12-08T13:56:59.547021939Z", "revisionCreateTime": "2023-12-08T13:56:59.340161044Z", "serviceAccount": "projects/PROJECT_ID/serviceAccounts/PROJECT_NUMBER-compute@developer.gserviceaccount.com", "sourceContents": "main:\n params: [input]\n [...] return: '${wikiResult.body[1]}'\n" }, { object (Workflow) } ], "nextPageToken": string, "unreachable": [ string ] }
Update a workflow
You can update an existing workflow. You must specify the name of the workflow you want to update and its source. Note that you can't change a workflow's name or location.
Updating a workflow does not affect in-progress executions. Only future executions of the workflow will use the updated configuration.
Each time you update a workflow, its versionID
is updated. The versionID
consists of two parts, separated by a hyphen:
A number, starting at one, that increments each time you update a workflow.
A random, three-character, alphanumeric string.
For example, 000001-27f
indicates the initial version of a workflow and
000002-d52
indicates a workflow that has been updated once.
Console
In the Google Cloud console, go to the Workflows page.
Click the name of the workflow you want to update. Note that you can't change the workflow's name.
The Workflow details page appears.
You can edit the workflow in the following ways:
To edit the source:
- Click the Source tab.
- Click Edit.
- To save your changes, click Save. The updated workflow is deployed.
To update the description, the service account the workflow uses for authentication, the call log level, environment variables, labels, or the encryption key:
- Click the Details tab.
- Click the appropriate icon.
- If you are updating the call log level, select one of:
- Not specified: no logging level is specified. This is the default. An execution log level takes precedence over any workflow log level, unless the execution log level is not specified (the default); in that case, the workflow log level applies.
- Errors only: log all caught exceptions; or when a call is stopped due to an exception.
- All calls: log all calls to subworkflows or library functions and their results.
- No logs: no call logging.
- If you update an existing label, or add a new one, it can take up
to 10 minutes for the new label to take effect. Workflow labels are
inherited by executions of the workflow. To view execution labels,
use the
workflows.executions.list
method to list and filter workflow executions. - To save your changes, click Save. The updated workflow is deployed.
To edit the previous fields at the same time, or add or update a trigger:
- Click Edit.
- To edit the source, click Next.
- To save your changes and deploy the updated workflow, click Deploy.
If you want to update the roles for your service account, click the Permissions tab.
Principals are users, groups, domains, or service accounts. To update an existing principal:
- Find a row containing the principal.
- Click Edit principal in that row.
- Click Add another role or click Delete role.
If you are adding a role, in the Select a role list, select an appropriate role.
To add another role, click Add another role.
Click Save.
gcloud
Open a terminal.
Find the name of the workflow you want to update. If you don't know the workflow's name, you can enter the following command to list all your workflows:
gcloud workflows list
Locate the YAML or JSON file where your workflow source is saved, such as
WORKFLOW_NAME.YAML
orWORKFLOW_NAME.JSON
.You can update an existing workflow to change its source, description, labels, environment variables, the call log level, encryption key, or its associated service account using the
gcloud workflows deploy
command.You must specify the name of the workflow you want to update and its source; however, the rest of the flags are optional. To remove a customer-managed encryption key, use the
--clear-kms-key
flag.gcloud workflows deploy WORKFLOW_NAME \ --call-log-level=CALL_LOGGING_LEVEL \ --description="DESCRIPTION" \ --labels="LABEL_KEY=LABEL_VALUE" \ --set-env-vars="ENV_KEY=ENV_VALUE" \ --kms-key=ENCRYPT_KEY \ --service-account=SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com \ --source=YAML_OR_JSON_SOURCE_FILE
Replace the following:
WORKFLOW_NAME
: required. The name of your workflow.CALL_LOGGING_LEVEL
: optional. Level of call logging to apply to the workflow. Can be one of:none
: no logging level is specified. This is the default. An execution log level takes precedence over any workflow log level, unless the execution log level is not specified (the default); in that case, the workflow log level applies.log-errors-only
: log all caught exceptions; or when a call is stopped due to an exception.log-all-calls
: log all calls to subworkflows or library functions and their results.log-none
: no call logging.
DESCRIPTION
: optional. A description of the workflow.LABEL_KEY=LABEL_VALUE
: optional. List of label key-value pairs that helps you organize your Google Cloud instances; for example,name=wrench
. For more information, see What are labels? Workflow labels are inherited by executions of the workflow. To view execution labels, use theworkflows.executions.list
method to list and filter workflow executions.ENV_KEY=ENV_VALUE
: optional. List of environment variable key-value pairs; for example,MONTH=January
. For more information, see Use environment variables.ENCRYPT_KEY
: optional. A Cloud KMS key that the workflow should use for data encryption in the formatprojects/PROJECT_NAME/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME
. For more information, see Use customer-managed encryption keys.SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com
: optional. The service account your workflow will use to access other Google Cloud services. If you want to update the roles for your service account, see Grant a workflow permission to access Google Cloud resources and Manage access to projects, folders, and organizations.YAML_OR_JSON_SOURCE_FILE
: required. The source file for the workflow in either YAML or JSON format. For example:myFirstWorkflow.yaml
.
REST
To update an existing workflow, use the
projects.locations.workflows.patch
method and optionally use the updateMask
query parameter to specify a list
of fields to be updated.
Before using any of the request data, make the following replacements:
WORKFLOW_NAME
: the name of your workflow—for example,myFirstWorkflow
.DESCRIPTION
: optional. A description of your workflow. Must be no longer than 1000 Unicode characters.LABEL_KEY
andLABEL_VALUE
: optional. A map of label key and value pairs that help you organize your Google Cloud instances. For example:{"name": "wrench", "mass": "1kg", "count": "3"}
For more information, see What are labels? Workflow labels are inherited by executions of the workflow. To view execution labels, use theworkflows.executions.list
method to list and filter workflow executions.SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com
: optional. The service account your workflow will use to access other Google Cloud services. The project ID is your Google Cloud project ID. For more information, in this document, see Create a service account.ENCRYPT_KEY
: optional. A Cloud KMS key that the workflow should use for data encryption in the formatprojects/PROJECT_NAME/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME
. For more information, see Use customer-managed encryption keys.CALL_LOGGING_LEVEL
: optional. The call logging level to apply during execution. The default is that no logging level is specified and the workflow log level applies instead. For more information, see Send Logs to Logging. One of the following:CALL_LOG_LEVEL_UNSPECIFIED
: no logging level is specified and the workflow log level applies instead. This is the default. Otherwise, the execution log level applies and takes precedence over the workflow log level.LOG_ERRORS_ONLY
: log all caught exceptions; or when a call is stopped due to an exception.LOG_ALL_CALLS
: log all calls to subworkflows or library functions and their results.LOG_NONE
: no call logging.
ENV_KEY
andENV_VALUE
: optional. A map of environment variable key and value pairs; for example,{ "month": "January", "day": "Monday"}
. For more information, see Use environment variables.SOURCE_CODE
: the minified workflow definition. For example:{\"main\":{\"params\":[\"input\"],\"steps\":[{\"returnOutput\":{\"return\":\"Hello\"}}]}}
LOCATION
: the region in which the workflow is deployed—for example,us-central1
.UPDATE_FIELDS
: optional. A comma-separated list of fields to be updated. If not provided, the entire workflow is updated. For exampledescription,callLogLevel
.
Request JSON body:
{ "name": "WORKFLOW_NAME", "description": "DESCRIPTION", "labels": {"LABEL_KEY":"LABEL_VALUE"}, "serviceAccount": "SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com", "cryptoKeyName": "ENCRYPT_KEY", "callLogLevel": "CALL_LOGGING_LEVEL", "userEnvVars": {"ENV_KEY":"ENV_VALUE"}, "sourceContents": "SOURCE_CODE" }
To send your request, expand one of these options:
If successful, the response body contains a newly created instance of
Operation
:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.workflows.v1.OperationMetadata", "createTime": "2023-12-05T14:06:06.338390918Z", "target": "projects/PROJECT_ID/locations/LOCATION/workflows/WORKFLOW_NAME", "verb": "update", "apiVersion": "v1" }, "done": false }
If the value of "done"
is false
, the operation
is still in progress.
Delete a workflow
You can delete an existing workflow. Deleting a workflow also deletes its executions and cancels any active executions for the workflow.
Console
In the Google Cloud console, go to the Workflows page.
Click the name of the workflow you want to delete and then click
Delete.At the prompt to confirm the deletion, enter the workflow name.
Click Confirm.
gcloud
Open a terminal.
Find the name of the workflow you want to delete. If you don't know the workflow's name, you can enter the following command to list all your workflows:
gcloud workflows list
Delete a workflow using the
gcloud workflows delete
command:gcloud workflows delete WORKFLOW_NAME
Replace
WORKFLOW_NAME
with the name of your workflow.
REST
To delete a workflow with a specified name, use the
projects.locations.workflows.delete
method.
Before using any of the request data, make the following replacements:
WORKFLOW_NAME
: the name of the workflow you want to delete.PROJECT_ID
: your Google Cloud project ID.LOCATION
: the region in which the workflow is deployed—for example,us-central1
.
To send your request, expand one of these options:
If successful, the response body contains an instance of
Operation
:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.workflows.v1.OperationMetadata", "createTime": "2023-12-05T14:06:06.338390918Z", "target": "projects/PROJECT_ID/locations/LOCATION/workflows/WORKFLOW_NAME", "verb": "delete", "apiVersion": "v1" }, "done": false }
If the value of "done"
is false
, the operation
is still in progress.