Running services with Workflows

You can use Workflows to combine and arrange Cloud Run services, linking and running the services in a customizable sequence.

Possible use cases include:

  • Combining multiple Cloud Run services easily.

  • Defining retries that extend the time beyond Cloud Run's own request limits.

  • Orchestrating batch processing so that the failure of any individual service is isolated.

This page describes deploying a Cloud Run service, creating a service account with the necessary permissions, and creating a workflow.

Before you start

  1. If you haven't done so already, set up your environment as described in the setup page for Cloud Run. You'll need to use the gcloud command line and a Google Cloud project to deploy your Cloud Run service to.

  2. Enable the Workflows API for your project:

    Enable the Workflows API

Deploying a Cloud Run service

To deploy a Cloud Run service to use in a workflow, deploy the service in the same way as any other Cloud Run service, keeping in mind the following best practices:

  • Create Cloud Run services to be stateless with minimal external dependencies. Use Workflows to provide state through other services and perform operations in other services.

  • When processing a list of items, create one service to return the list, and another service to process an individual item; use Workflows to sequence the retrieval of the list and the processing of each item.

  • If a service requires more than 64 KB of data (the maximum size of a request passed to a workflow), consider storing the data (for example, in Firestore or Cloud Storage), and passing a reference. Or, retrieve the data in the service rather than passing it with Workflows.

Creating a service account

You need to create a service account to associate with your workflow, and give it the permission to invoke your Cloud Run service.

Console

  1. In the Cloud Console, go to the Service Accounts page.

    Go to Service Accounts

  2. Select a project.

  3. Enter a service account name to display in the Cloud Console.

    The Cloud Console generates a service account ID based on this name. Edit the ID if necessary. You cannot change the ID later.

  4. Optional: Enter a description of the service account.

  5. Click Create.

  6. Click the Select a role field.

  7. Under All roles, select Cloud Run > Cloud Run Invoker.

  8. Click Done.

Command line

  1. Create the service account:

    gcloud iam service-accounts create SERVICE_ACCOUNT_NAME \
       --display-name "DISPLAYED_SERVICE_ACCOUNT_NAME"

    Replace

    • SERVICE_ACCOUNT_NAME with a lower case name unique within your Google Cloud project, for example my-invoker-service-account-name.
    • DISPLAYED_SERVICE_ACCOUNT_NAME with the name you want to display for this service account, for example, in the console, for example, My Invoker Service Account.
  2. For Cloud Run, give your service account permission to invoke your service:

    gcloud run services add-iam-policy-binding SERVICE \
       --member=serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \
       --role=roles/run.invoker

    Replace

    • SERVICE with the name of the service you want to be invoked by Workflows.
    • SERVICE_ACCOUNT_NAME with the name of the service account.
    • PROJECT_ID with your Google Cloud project ID.

Creating a workflow

A workflow definition is made up of a series of steps described using the Workflows syntax, which can be written in either YAML or JSON format. After creating a workflow, you deploy it to make it available for execution. The deploy step also validates that the source file can be executed. It fails if the source file doesn't contain a valid workflow definition.

Console

  1. In the Cloud Console, go to the Workflows page:

    Go to Workflows

  2. Select Create.

  3. Enter a name for the new workflow, such as myFirstWorkflow. The name can contain letters, numbers, underscores, and hyphens. It must start with a letter and end with a number or letter.

  4. Choose an appropriate region; for example, us-central1.

  5. Select the service account you want your workflow to use for authentication with other Google Cloud services. We strongly recommend using a service account with the least privileges necessary to access the required resources. To learn more about service accounts, see Create and manage service accounts.

  6. Select Next.

  7. In the workflow editor, enter the definition for your workflow. You can find an example workflow in Quickstart: Create a workflow using the Cloud Console page.

  8. Select Deploy.

gcloud

  1. Make sure your workflow's source code is saved in a YAML or JSON file, such as MY_WORKFLOW.YAML or MY_WORKFLOW.JSON. You can find an example workflow in Quickstart: Create a workflow using the gcloud tool page.

  2. Open a terminal.

  3. Deploy the workflow by entering the following command:

    gcloud workflows deploy MY_WORKFLOW \
    --source=MY_WORKFLOW.YAML \
    [--service-account=MY_SERVICE_ACCOUNT@MY_PROJECT.IAM.GSERVICEACCOUNT.COM]
    

    Replace the following:

    • MY_WORKFLOW: The name of your workflow.

    • MY_WORKFLOW.YAML: The source file to use for the workflow.

    • MY_SERVICE_ACCOUNT@MY_PROJECT.IAM.GSERVICEACCOUNT.COM: Optional. The service account your workflow will use to access other Google Cloud services. We strongly recommend using a service account with the least privileges necessary to access the required resources. If left blank, the default service account is used. To learn more about service accounts, see Create and manage service accounts.

Invoking Cloud Run with authentication

To make an authenticated request within a workflow, use OpenID Connect (OIDC) to connect with Cloud Run.

To make an HTTP request using OIDC, add an auth value of type: OIDC in the args key of the call step.

An audience parameter in the auth key is optional, but can be used to specify the OIDC audience for the token. By default, it's set to the same value as url.

This sample invokes a Cloud Run workload that requires authentication.

YAML

- call_my_function:
    call: http.get
    args:
      url: https://example-12345-ew.a.run.app
      auth:
        type: OIDC
      query:
        some_val: "Hello World"
        another_val: 123
    result: the_message
- return_value:
    return: ${the_message.body}

JSON

[
  {
    "call_my_function": {
      "call": "http.get",
      "args": {
        "url": "https://example-12345-ew.a.run.app",
        "auth": {
          "type": "OIDC"
        },
        "query": {
          "some_val": "Hello World",
          "another_val": 123
        }
      },
      "result": "the_message"
    }
  },
  {
    "return_value": {
      "return": "${the_message.body}"
    }
  }
]

For more information about authenticating to a resource from within a workflow, see making authenticated requests from within a workflow.

What's next