Executing asynchronous tasks

You can use Cloud Tasks to securely enqueue a task to be asynchronously processed by a Cloud Run service. Typical use cases include:

Preserving requests through unexpected production incidents
Smoothing traffic spikes by delaying work that is not user-facing
Speeding user response time by delegating slow background operations to be handled by another service, such as database updates or batch processing
Limiting the call rate to backing services like databases and third-party APIs

This page shows how to enqueue tasks that are securely pushed via the HTTPS protocol to a private Cloud Run service. It describes required behavior for the private Cloud Run service, required service account permissions, task queue creation, and task creation.

Before you start

Enable the Cloud Tasks API on the project you are using.

Deploying a Cloud Run service to handle tasks

To deploy a service that accepts tasks sent to the task queue, deploy the service in the same way as any other Cloud Run service. The Cloud Run service must return an HTTP 200 code to confirm proper processing of the task.

Tasks will be pushed to this Cloud Run service as HTTPS requests by Cloud Tasks.

Creating a task queue

To create a task queue, use the command

gcloud tasks queues create QUEUE-ID

replacing QUEUE-ID with the name you want to give to your task queue: it must be unique in your project. If you are prompted to create an App Engine app in your project, respond y to create it. Cloud Tasks uses this for the queue: make sure you choose the same location as you are using for your Cloud Run service.

The default task queue configuration should work in most cases. However, you can optionally set different rate limits and retry parameters if you want.

Creating a service account to associate with the tasks

You must create a service account that will be associated with the enqueued tasks. This service account must have the Cloud Run Invoker IAM role to allow the task queue to push tasks to the Cloud Run (fully managed) service. .

Console

Visit the Create service account key page in the Cloud Console.

Create service account page
From the Service account list, select New service account.
In the Service account name field, enter the name you want to use for the service account.
Click Create.
Copy the service account email to use in the following steps.
Click Continue if prompted to specify permissions.
Visit the Cloud Run Services page in the Cloud Console.

Go to the Services page
Select your service in the displayed list.
If necessary, click the Show Info Panel/Hide Info Panel toggle in the far right of the page to show information.
Locate the Permissions tab, and in that tab, click Add Member.
Paste your service account email into the New members field.
From the Role dropdown menu, select Cloud Run > Cloud Run Invoker.
Click Save.

Command line

Create the service account:
```
gcloud iam service-accounts create SERVICE-ACCOUNT_NAME \
   --display-name "DISPLAYED-SERVICE-ACCOUNT_NAME"
```
Replace
- SERVICE-ACCOUNT_NAME with a lower case name unique within your Google Cloud project, for example my-invoker-service-account-name.
- DISPLAYED-SERVICE-ACCOUNT-NAME with the name you want to display for this service account, for example, in the console, for example, My Invoker Service Account.
For Cloud Run, give your service account permission to invoke your service:
```
gcloud run services add-iam-policy-binding SERVICE \
   --member=serviceAccount:SERVICE-ACCOUNT_NAME@PROJECT-ID.iam.gserviceaccount.com \
   --role=roles/run.invoker
```
Replace
- SERVICE with the name of the service you want to be invoked by Cloud Tasks.
- SERVICE-ACCOUNT_NAME with the name of the service account.
- PROJECT-ID with your Google Cloud project ID.

Creating HTTP tasks with authentication tokens

When you create a task to send to the task queue, you specify the project, the location, queue name, the email of the previously created service account to associate with tasks, the URL of the private Cloud Run service that will run the task, and any other data you need to send.

Refer to the Cloud Tasks API documentation for details on the task request body. Note that requests that contain data payloads must use the HTTP PUT or POST method.

The code that enqueues the tasks must have the necessary IAM permissions to do so, such as the Cloud Tasks Enqueuer role. Your code will have the necessary IAM permissions if you use the default service account on Cloud Run (fully managed).

The following examples create task requests that also include the creation of a header token. OIDC tokens are used in the examples. To use an OAuth token, replace the OIDC parameter with the language appropriate OAuth parameter in constructing the request.

Python

samples/snippets/create_http_task_with_token.py

View on GitHub Feedback

"""Create a task for a given queue with an arbitrary payload."""

from google.cloud import tasks_v2

# Create a client.
client = tasks_v2.CloudTasksClient()

# TODO(developer): Uncomment these lines and replace with your values.
# project = 'my-project-id'
# queue = 'my-queue'
# location = 'us-central1'
# url = 'https://example.com/task_handler'
# service_account_email = 'service-account@my-project-id.iam.gserviceaccount.com';
# payload = 'hello'

# Construct the fully qualified queue name.
parent = client.queue_path(project, location, queue)

# Construct the request body.
task = {
    "http_request": {  # Specify the type of request.
        "http_method": tasks_v2.HttpMethod.POST,
        "url": url,  # The full url path that the task will be sent to.
        "oidc_token": {"service_account_email": service_account_email},
    }
}

if payload is not None:
    # The API expects a payload of type bytes.
    converted_payload = payload.encode()

    # Add the payload to the request.
    task["http_request"]["body"] = converted_payload

# Use the client to build and send the task.
response = client.create_task(request={"parent": parent, "task": task})

print("Created task {}".format(response.name))
return response

Note the requirements.txt file:

samples/snippets/requirements.txt

View on GitHub Feedback

google-cloud-tasks==2.2.0

Java

samples/snippets/src/main/java/com/example/task/CreateHttpTaskWithToken.java

View on GitHub

import com.google.cloud.tasks.v2.CloudTasksClient;
import com.google.cloud.tasks.v2.HttpMethod;
import com.google.cloud.tasks.v2.HttpRequest;
import com.google.cloud.tasks.v2.OidcToken;
import com.google.cloud.tasks.v2.QueueName;
import com.google.cloud.tasks.v2.Task;
import com.google.protobuf.ByteString;
import java.io.IOException;
import java.nio.charset.Charset;

public class CreateHttpTaskWithToken {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project-id";
    String locationId = "us-central1";
    String queueId = "my-queue";
    String serviceAccountEmail = 
        "java-docs-samples-testing@java-docs-samples-testing.iam.gserviceaccount.com";
    createTask(projectId, locationId, queueId, serviceAccountEmail);
  }

  // Create a task with a HTTP target and authorization token using the Cloud Tasks client.
  public static void createTask(
      String projectId, String locationId, String queueId, String serviceAccountEmail)
      throws IOException {

    // Instantiates a client.
    try (CloudTasksClient client = CloudTasksClient.create()) {
      String url =
          "https://example.com/taskhandler"; // The full url path that the request will be sent to
      String payload = "Hello, World!"; // The task HTTP request body

      // Construct the fully qualified queue name.
      String queuePath = QueueName.of(projectId, locationId, queueId).toString();

      // Add your service account email to construct the OIDC token.
      // in order to add an authentication header to the request.
      OidcToken.Builder oidcTokenBuilder =
          OidcToken.newBuilder().setServiceAccountEmail(serviceAccountEmail);

      // Construct the task body.
      Task.Builder taskBuilder =
          Task.newBuilder()
              .setHttpRequest(
                  HttpRequest.newBuilder()
                      .setBody(ByteString.copyFrom(payload, Charset.defaultCharset()))
                      .setHttpMethod(HttpMethod.POST)
                      .setUrl(url)
                      .setOidcToken(oidcTokenBuilder)
                      .build());

      // Send create task request.
      Task task = client.createTask(queuePath, taskBuilder.build());
      System.out.println("Task created: " + task.getName());
    }
  }
}

Note the pom.xml file:

samples/snippets/pom.xml

View on GitHub

<?xml version='1.0' encoding='UTF-8'?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.google.cloud</groupId>
  <artifactId>cloudtasks-snippets</artifactId>
  <packaging>jar</packaging>
  <name>Google Cloud Tasks Snippets</name>
  <url>https://github.com/googleapis/java-tasks</url>

  <!--
    The parent pom defines common style checks and testing strategies for our samples.
    Removing or replacing it should not affect the execution of the samples in anyway.
  -->
  <parent>
    <groupId>com.google.cloud.samples</groupId>
    <artifactId>shared-configuration</artifactId>
    <version>1.0.21</version>
  </parent>

  <properties>
    <maven.compiler.target>1.8</maven.compiler.target>
    <maven.compiler.source>1.8</maven.compiler.source>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>


  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>libraries-bom</artifactId>
        <version>19.0.0</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-tasks</artifactId>
    </dependency>

    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.13.2</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>com.google.truth</groupId>
      <artifactId>truth</artifactId>
      <version>1.1.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

Go

tasks/token/create_http_task_with_token.go

View on GitHub

import (
	"context"
	"fmt"

	cloudtasks "cloud.google.com/go/cloudtasks/apiv2"
	taskspb "google.golang.org/genproto/googleapis/cloud/tasks/v2"
)

// createHTTPTaskWithToken constructs a task with a authorization token
// and HTTP target then adds it to a Queue.
func createHTTPTaskWithToken(projectID, locationID, queueID, url, email, message string) (*taskspb.Task, error) {
	// Create a new Cloud Tasks client instance.
	// See https://godoc.org/cloud.google.com/go/cloudtasks/apiv2
	ctx := context.Background()
	client, err := cloudtasks.NewClient(ctx)
	if err != nil {
		return nil, fmt.Errorf("NewClient: %v", err)
	}

	// Build the Task queue path.
	queuePath := fmt.Sprintf("projects/%s/locations/%s/queues/%s", projectID, locationID, queueID)

	// Build the Task payload.
	// https://godoc.org/google.golang.org/genproto/googleapis/cloud/tasks/v2#CreateTaskRequest
	req := &taskspb.CreateTaskRequest{
		Parent: queuePath,
		Task: &taskspb.Task{
			// https://godoc.org/google.golang.org/genproto/googleapis/cloud/tasks/v2#HttpRequest
			MessageType: &taskspb.Task_HttpRequest{
				HttpRequest: &taskspb.HttpRequest{
					HttpMethod: taskspb.HttpMethod_POST,
					Url:        url,
					AuthorizationHeader: &taskspb.HttpRequest_OidcToken{
						OidcToken: &taskspb.OidcToken{
							ServiceAccountEmail: email,
						},
					},
				},
			},
		},
	}

	// Add a payload message if one is present.
	req.Task.GetHttpRequest().Body = []byte(message)

	createdTask, err := client.CreateTask(ctx, req)
	if err != nil {
		return nil, fmt.Errorf("cloudtasks.CreateTask: %v", err)
	}

	return createdTask, nil
}

Node.js

samples/createHttpTaskWithToken.js

View on GitHub

// Imports the Google Cloud Tasks library.
const {CloudTasksClient} = require('@google-cloud/tasks');

// Instantiates a client.
const client = new CloudTasksClient();

async function createHttpTaskWithToken() {
  // TODO(developer): Uncomment these lines and replace with your values.
  // const project = 'my-project-id';
  // const queue = 'my-queue';
  // const location = 'us-central1';
  // const url = 'https://example.com/taskhandler';
  // const serviceAccountEmail = 'client@<project-id>.iam.gserviceaccount.com';
  // const payload = 'Hello, World!';

  // Construct the fully qualified queue name.
  const parent = client.queuePath(project, location, queue);

  const task = {
    httpRequest: {
      httpMethod: 'POST',
      url,
      oidcToken: {
        serviceAccountEmail,
      },
    },
  };

  if (payload) {
    task.httpRequest.body = Buffer.from(payload).toString('base64');
  }

  console.log('Sending task:');
  console.log(task);
  // Send create task request.
  const request = {parent, task};
  const [response] = await client.createTask(request);
  const name = response.name;
  console.log(`Created task ${name}`);
}
createHttpTaskWithToken();

Note the package.json file:

samples/package.json

View on GitHub

{
  "name": "appengine-cloudtasks",
  "description": "Google App Engine Cloud Tasks example.",
  "license": "Apache-2.0",
  "author": "Google Inc.",
  "private": true,
  "engines": {
    "node": ">=10"
  },
  "files": [
    "*.js"
  ],
  "scripts": {
    "test": "mocha",
    "start": "node server.js"
  },
  "dependencies": {
    "@google-cloud/tasks": "^2.3.0",
    "body-parser": "^1.18.3",
    "express": "^4.16.3"
  },
  "devDependencies": {
    "chai": "^4.2.0",
    "mocha": "^8.0.0",
    "uuid": "^8.0.0"
  }
}

Executing asynchronous tasks

Before you start

Deploying a Cloud Run service to handle tasks

Creating a task queue

Creating a service account to associate with the tasks

Console

Command line

Creating HTTP tasks with authentication tokens

Python

Java

Go

Node.js

What's next