Create and manage workflows

This page shows you how to create and manage workflows in the Google Cloud console, or by using the Google Cloud CLI in either your terminal or Cloud Shell. You can also manage workflows through the Workflows API.

Before you begin

Some of the steps in this document might not work correctly if your organization applies constraints to your Google Cloud environment. In that case, you might not be able to complete tasks like creating public IP addresses or service account keys. If you make a request that returns an error about constraints, see how to Develop applications in a constrained Google Cloud environment.

Console

1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

2. Make sure that billing is enabled for your Google Cloud project. Learn how to check if billing is enabled on a project.

3. Enable the Workflows API.

   Enable the API

gcloud

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. Make sure that billing is enabled for your Google Cloud project. Learn how to check if billing is enabled on a project.

3. Enable the Workflows API.

   gcloud services enable workflows.googleapis.com
   

API

To manage workflows using the Workflows API, we recommend that you use the Google-provided client libraries to call the workflows.googleapis.com service. For more information, see Workflows API.

Create a service account

Workflows uses service accounts to give workflows access to Google Cloud resources. Create a service account if you don't already have one; then grant it the roles necessary for managing workflows and creating logs. If you don't specify a service account during the workflow's creation, the workflow uses the default Compute Engine service account for its identity. Learn more about deploying a workflow with a service account.

Note that to create a resource and attach a service account, you need permissions to create that resource and to impersonate the service account that you will attach to the resource. For more information, see Service account permissions.

Console

1. In the Google Cloud console, go to the Service Accounts page.

   Go to Service Accounts

2. Select a project and then click Create service account.

3. In the Service account name field, enter a name.

   The name must be between 6 and 30 characters, and can contain lowercase alphanumeric characters and dashes. After you create a service account, you cannot change its name.

4. Click Create and continue.

5. Click Select a role.

6. Select the following roles, clicking Add another role as required:

   • To create, update, and execute workflows, select Workflows > Workflows Editor.
   • To send logs to Cloud Logging, select Logging > Logs Writer.
   Important: For production environments, Google strongly recommends creating a new service account and granting it one or more IAM roles that contain the minimum permissions required for managing workflows. To learn more about which permissions and roles are available in Workflows, see Access control.

7. Click Done to finish creating the service account.

gcloud

1. Create the service account.

   gcloud iam service-accounts create NAME

2. Grant permissions to the service account by assigning a role.

   gcloud projects add-iam-policy-binding PROJECT_ID \
       --member "serviceAccount:NAME@PROJECT_ID.iam.gserviceaccount.com" \
       --role "roles/logging.logWriter"
   

   Replace the following:

   • NAME: the name of the service account. It must be between 6 and 30 characters, and can contain lowercase alphanumeric characters and dashes. After you create a service account, you cannot change its name.
   • PROJECT_ID: your project's ID.

   The --role flag is used to authorizes your service account to access resources.

   • To create, update, and execute workflows, assign the roles/workflows.editor role.
   • To send logs to Cloud Logging, assign the roles/logging.logWriter role.
   Important: For production environments, Google strongly recommends creating a new service account and granting it one or more IAM roles that contain the minimum permissions required for managing workflows. To learn more about which permissions and roles are available in Workflows, see Access control.

Create a workflow

A workflow definition is made up of a series of steps described using the Workflows syntax, which can be written in either YAML or JSON format. After creating a workflow, you deploy it to make it available for execution. The deploy step also validates that the source file can be executed. It fails if the source file doesn't contain a valid workflow definition.

Note: YAML indentation is meaningful and should be at least two spaces per indentation level. Insufficient indentation can cause errors, and a new level should be at least two spaces in from the start of the text in the previous line. For more details, see Troubleshoot issues.

Console

1. In the Google Cloud console, go to the Workflows page.

   Go to Workflows

2. Click Create.

3. Enter a name for the new workflow, such as myFirstWorkflow. The name can contain letters, numbers, underscores, and hyphens. It must start with a letter and end with a number or letter.

4. In the Region list, select an appropriate region; for example, us-central1.

5. In the Service account list, select a service account for your workflow to use for authentication with other Google Cloud services. We strongly recommend using a service account with the least privileges necessary to access the required resources. For more information, in this document, see Create a service account.

   We strongly recommend using a service account with the least privileges necessary to access the required resources.

6. Optionally, do any of the following:

   1. Add a label: labels are key-value pairs that help you organize your Google Cloud instances. For more information, see Create and manage labels.

      Note: It can take up to 10 minutes for the new label to take effect.

   2. Schedule your workflow: select Add new trigger > Cloud Scheduler. For more information, see Schedule a workflow using Cloud Scheduler.

   3. Trigger your workflow through an event or Pub/Sub message: select Add new trigger > Eventarc. For more information, see Trigger a workflow with events or Pub/Sub messages.

7. Click Next.

8. In the workflow editor, enter the definition for your workflow. See an example workflow.

9. Click Deploy.

gcloud

1. Make sure your workflow's source code is saved in a YAML or JSON file, such as MY_WORKFLOW.YAML or MY_WORKFLOW.JSON. See an example workflow.

2. Open a terminal.

3. Deploy the workflow by entering the following command:

   gcloud workflows deploy MY_WORKFLOW \
       --source=YAML_OR_JSON_SOURCE_FILE \
       --service-account=MY_SERVICE_ACCOUNT@MY_PROJECT.IAM.GSERVICEACCOUNT.COM
   

   Replace the following:

   • MY_WORKFLOW: the name of your workflow.

   • YAML_OR_JSON_SOURCE_FILE: the source file to use for the workflow. For example: myFirstWorkflow.yaml.

   • MY_SERVICE_ACCOUNT@MY_PROJECT.IAM.GSERVICEACCOUNT.COM: optional. The service account your workflow will use to access other Google Cloud services. We strongly recommend using a service account with the least privileges necessary to access the required resources. If left blank, the default service account is used. For more information, in this document, see Create a service account.

API

Call the projects.locations.workflows.create method and use the workflowId parameter to specify an ID for the workflow.

List workflows

You can list workflows or use filters to retrieve a specific workflow.

Console

1. In the Google Cloud console, go to the Workflows page.

   Go to Workflows

   This page lists your workflows in all locations, and includes details such as names, regions, latest revisions, and more.

2. To filter your workflows:

   1. Click Filter or the Filter workflows field.
   2. In the Properties list, select an option to filter the workflows by.

   You can select a single property or use the logical operator OR to add more properties.

3. To sort your workflows, beside any supported column heading, click Sort.

gcloud

List workflows using the gcloud workflows list command:

gcloud workflows list --location=WORKFLOW_NAME

Replace LOCATION with the ID or fully qualified identifier for the location of your workflows

This command lists your workflows in the specified location, and includes details such as a workflow's NAME, STATE, REVISION_ID, and UPDATE_TIME.

API

Call the projects.locations.workflows.list method to list workflows in a given project and location. Or, call the projects.locations.workflows.get method to retrieve the details of a single workflow.

Note: When listing triggers, paging might be applied before or after filtering. The service might return fewer resources than the number requested through the pageSize parameter, even if not at the end of the collection.

Update a workflow

You can update an existing workflow to change its source, description, labels, triggers, or its associated service account. Updating a workflow does not affect in-progress executions. Only future executions of the workflow will use the updated configuration.

Console

1. In the Google Cloud console, go to the Workflows page.

   Go to Workflows

2. Click the name of the workflow you want to update. Note that you can't change the workflow's name.

   The Workflows details page appears.

3. You can edit the workflow in the following ways:

   • To edit the source:

     1. Click the Source tab.
     2. Click Edit.
     3. To save your changes, click Save. The updated workflow is deployed.

   • To edit the description, update the service account the workflow uses for authentication, or add or update a label:

     1. Click the Details tab.
     2. Click the appropriate .
     3. To save your changes, click Save. The updated workflow is deployed.

   • To edit the previous fields at the same time, or add or update a trigger:

     1. Click Edit.
     2. To edit the source, click Next.
     3. To save your changes and deploy the updated workflow, click Deploy.
   Note: If you update an existing label, or add a new one, it can take up to 10 minutes for the new label to take effect.

4. If you want to update the roles for your service account, click the Permissions tab.

   1. Principals are users, groups, domains, or service accounts. To update an existing principal:

      1. Find a row containing the principal.
      2. Click Edit principal in that row.
      3. Click Add another role or click Delete role.

   2. If you are adding a role, in the Select a role list, select an appropriate role.

   3. To add another role, click Add another role.

   4. Click Save.

   Note: To change the service account the workflow uses for authentication, you must still edit the workflow and modify the service account as detailed in the previous step. For more information, see Grant a workflow permission to access Google Cloud resources.

gcloud

1. Open a terminal.

2. Find the name of the workflow you want to update. If you don't know the workflow's name, you can enter the following command to list all your workflows:

   gcloud workflows list
   

3. Locate the YAML or JSON file where your workflow source is saved, such as WORKFLOW_NAME.YAML or WORKFLOW_NAME.JSON.

4. You can update a workflow's source, associated service account, description, or labels using the gcloud workflows deploy command:

   gcloud workflows deploy WORKFLOW_NAME \
       --source=YAML_OR_JSON_SOURCE_FILE \
       --service-account=MY_SERVICE_ACCOUNT@MY_PROJECT.IAM.GSERVICEACCOUNT.COM \
       --labels=KEY=VALUE... \
       --description='WORKFLOW_DESCRIPTION'
   

   Replace the following:

   • WORKFLOW_NAME: required. The name of your workflow.

   • YAML_OR_JSON_SOURCE_FILE: required. The source file for the workflow in either YAML or JSON format. For example: myFirstWorkflow.yaml.

   • MY_SERVICE_ACCOUNT@MY_PROJECT.IAM.GSERVICEACCOUNT.COM: optional. The service account your workflow will use to access other Google Cloud services. If you want to update the roles for your service account, see Grant a workflow permission to access Google Cloud resources and Manage access to projects, folders, and organizations.

   • KEY=VALUE: optional. KEY=VALUE pairs of labels, in a comma-separated list if you add more than one. Labels help you organize your Google Cloud instances. For more information, see Create and manage labels.

     Note: If you update an existing label, or add a new one, it can take up to 10 minutes for the new label to take effect.

   • WORKFLOW_DESCRIPTION: optional. The description you want to give your workflow.

   You must specify the name of the workflow you want to update and its source; however, the rest of the flags are optional.

API

Call the projects.locations.workflows.patch method to update an existing workflow.

Each time you update a workflow, its versionID is updated. The versionID consists of two parts, separated by a hyphen:

• A number, starting at one, that is incremented each time you update a workflow.

• A random, three-character, alphanumeric string.

For example, 000002-d52 indicates a workflow that has been updated once. 000001-27f indicates a workflow that is on its initial version.

Delete a workflow

You can delete an existing workflow. Deleting a workflow also deletes its executions and cancels any active executions for the workflow.

Console

1. In the Google Cloud console, go to the Workflows page.

   Go to Workflows

2. Click the name of the workflow you want to delete and then click Delete.

3. At the prompt to confirm the deletion, enter the workflow name.

4. Click Confirm.

gcloud

1. Open a terminal.

2. Find the name of the workflow you want to delete. If you don't know the workflow's name, you can enter the following command to list all your workflows:

   gcloud workflows list
   

3. Delete a workflow using the gcloud workflows delete command:

   gcloud workflows delete WORKFLOW_NAME
   

   Replace WORKFLOW_NAME with the name of your workflow.

API

Call the projects.locations.workflows.delete method to delete a workflow with a specified name.

What's next

• Execute a workflow

• Control the order of execution in a workflow

• Workflows syntax reference