This page describes what Cloud Tasks tasks and queues are, and when and how to use them. Cloud Tasks lets you separate out pieces of work that can be performed independently, outside of your main application flow, and send them off to be processed, asynchronously, using handlers that you create. These independent pieces of work are called tasks. For example, you need to update a database as part of processing a user request, but updates can be time-consuming. Offloading that detail as a task allows you to return from the request more quickly.
The offloaded task is added to a queue, which persists the task until it is successfully executed. Based on your initial configuration, the queue can also act as a dispatch flow control. You create and configure the queue which is then managed by the Cloud Tasks service. Once tasks are added, the queue dispatches them and makes sure they are reliably processed by your workers. Complexities associated with that process, such as user-facing latency costs, server crashes, resource consumption limitations, and retry management, are handled by the service.
The tasks themselves are made up of a unique name and configuration information and, optionally, any data from the initial request, called the payload, necessary to process the request. As the payload is sent in the request body, tasks that include payloads must use POST or PUT as their HTTP method.
Users of App Engine standard environment first generation runtimes should access the Cloud Tasks service via the App Engine Task Queue API. For more information on using this method, see the Java 8, Python 2.7, Go 1.11, or PHP 5.5 documentation. Users of App Engine standard environment second generation runtimes, flexible environment, and those on other platforms can now use Cloud Tasks API.
To access the Cloud Tasks service using Cloud Tasks API , you must have a Google Cloud project.
Typical use cases include:
- Speeding user response times by delegating potentially slow background operations like database updates to a worker
- Preserving requests in the context of unexpected production incidents
- Helping smooth traffic spikes by removing non-user-facing tasks from the main user flow
- Managing third-party API call rates
Cloud Tasks queues with HTTP targets
In the case of generic HTTP Targets, the Cloud Tasks service forwards the task request to the worker, located at any generic HTTP endpoint, based on how the task is configured. This endpoint could be on Cloud Functions, Cloud Run, GKE, Compute Engine, or even an on-prem web server, based on how the task is configured. These queues dispatch requests at a reliable, configurable rate. They guarantee reliable task execution - upon success, all workers must send an HTTP response code (200-299) to the Cloud Tasks service before the default timeout deadline of 10 minutes, with a maximum of 30 minutes. If a different response is sent, or no response, the task is retried.
The target must manage scaling workers and cleaning up tasks once they are complete.
If your target requires authentication, you must set up two service accounts, one for your application, the client, and one for the queue itself. Both accounts must have been granted the requisite permissions, and an identifier for the client service account must be included in the task request. See Create HTTP Target tasks for more information.
Cloud Tasks queues with App Engine targets
In the case of App Engine targets, the Cloud Tasks service also forwards the task request to the handler, but this worker is located within App Engine. So all queues that target App Engine handlers must have an App Engine app. The handlers must run in the region where the App Engine app runs. This region also serves as the LOCATION_ID parameter for your Cloud Tasks requests.
The tasks are routed based on how the task (or, less commonly, the queue itself) is configured. The queues dispatch requests at a reliable, configurable rate. They guarantee reliable task execution - upon success, all workers must send an HTTP response code (200-299) to the Cloud Tasks service, in this instance before a deadline based on the instance scaling type of the service: 10 minutes for automatic scaling or up to 24 hours for manual scaling. If a different response is sent, or no response, the task is retried.
Because the handlers are part of App Engine, the Cloud Tasks service itself can do much of the process management for the task, scaling workers up and down in relation to traffic and deleting tasks when they are completed.
Supported regions by target
If your target is an HTTP/S endpoint, Cloud Tasks is available in all supported Google Cloud regions for Cloud Tasks.
If your target is an App Engine application located within your current project:
A task targeting App Engine can only be created in the project's App Engine region.
A Cloud project can contain only one App Engine app, and the region where the App Engine app is located cannot be changed once the app is created.
App Engine is regional which means the infrastructure that runs your app is located in a specific region. If you want to distribute compute and queues across multiple regions, you should target an HTTP/S endpoint instead.
If you are not using App Engine as a target, you do not need to deploy an App Engine app, and you can disable any existing App Engine app.
The general workflow is as follows:
- You create a worker to process the tasks.
- You create a queue.
- You create tasks programmatically and add them to the queue.
- The Cloud Tasks service returns an OK to the originating application. This indicates that the task has been successfully written to Cloud Tasks storage, making the create task request both highly available and durable.
- The tasks are passed to the worker.
- The worker processes the task.
- To complete the sequence, the worker returns a 2xx success status code to the Cloud Tasks service.
Once the task has been handed off to the queue, no data is made available to the initial request.
Using Cloud Tasks, you can dispatch asynchronous work items with the following controls:
- Schedule specific delivery times
- Manage delivery rates
- Configure retry behavior
- Access and manage individual tasks in a queue
- Enable task deduplication
The following table shows key terms that describe aspects of Cloud Tasks behavior.
|queue||A set of tasks with the same target type managed by a single configuration.|
|target type||Where and how a task is processed.|
|worker||A service which processes tasks.|
|attempt||An attempt to run a task.|
|attempt dispatch||The moment when Cloud Tasks has sent the task to its target.|
|attempt response||A response from a worker which indicates that the work associated with the task completed successfully or failed.|
|retry||Multiple attempts to run a task. The number of retries is set via the RetryConfig.|
|rate limits||The rate limits for a queue.|
The following predefined Tasks metrics are available using Cloud Monitoring.
Kind, Type, Unit
Count of Cloud Tasks API calls.
BETA Queue depth
|The number of tasks in the queue. Sampled every 60 seconds. After sampling, data is not visible for up to 120 seconds.|
Task attempt count
Count of task attempts broken down by response code.
Task attempt delays
|Delay between each scheduled attempt time and actual attempt time.|