Applications define task queues in a configuration file called
queue.yaml. For Java apps, this file is stored in the application's
WEB-INF directory. You can use
queue.yaml to configure both push queues and pull queues. This configuration file is optional for push queues, which have a default queue. Pull queues must be specifically configured in
You define task queues for an app using a file named
, in the app's `WEB-INF/` directory in the WAR.
queue.yaml, under the
queue directive, you specify and configure
one or more task queues.
There is only one
queue.yaml for an app, and it applies to all versions of
that app. Note that the task handler used to process a task by default is the
task handler of the app version that enqueued the task. But you have control
over this behavior using the
An app can only add tasks to queues defined in
queue.yaml and the default
queue. You can remove a queue by removing its definition from
uploading the changed
Note that if you upload a new
queue.yaml file that removes a queue that still
contains tasks, the queue has its rate set to 0, which disables it, but the
queue is not actually deleted. You can re-enable a disabled queue by
uploading a new
queue.yaml file with the queue defined with a rate greater
Setting the storage limit for all queues
Task queues contain task-related data, and the storage of this data counts
towards your billable storage quota. The default task storage limit for all of
an application's queues is
500M (500 megabytes) for free apps. For billed apps
there is no limit unless you explicitly set one.
However, you may want to specify a total storage limit to protect your app from errors such as a fork bomb programming error in which each task adds multiple other tasks during its execution. If you do use storage limits, we recommend setting a limit sufficient for several days' worth of tasks.
Note that the total storage limit applies to all the storage used by all of the app's queues. If adding a task would result in the exceeding of the storage limit, the task add call will fail.
To specify a total storage limit, include the directive
queue line in your app
queue.yaml like this:
# Set the total storage limit for all queues to 120MB total_storage_limit: 120M queue: - name: foo rate: 35/s
total_storage_limit value is a number followed by a unit:
Defining push queues and processing rates
You can define any number of individual queues by providing a queue
queue.yaml. You can control the rate at which tasks are processed in each
queue by defining other directives, such as
You can read more about these directives in the Queue Definitions section.
The task queue uses token buckets to
control the rate of task execution. Each named queue has a token bucket that
holds a certain number of tokens, defined by the
bucket_size directive. Each
time your application executes a task, it uses a token. Your app continues
processing tasks in the queue until the queue's bucket runs out of tokens.
App Engine refills the bucket with new tokens continuously based on the
that you specified for the queue.
Configuring the default queue
All apps have a push queue named
default. This queue has a preset rate of
5 tasks per second, but you can change this rate by defining a default queue in
queue.yaml. If you do not configure a default queue in
default queue doesn't display in the Cloud Platform Console until the first
time it is used. You can customize the settings for this queue by defining a
queue: # Change the refresh rate of the default queue from 5/s to 1/s - name: default rate: 1/s
Configuring the processing rate
If your queue contains tasks to process, and the queue's bucket contains tokens, App Engine processes as many tasks as there are tokens remaining in the bucket. This can lead to bursts of processing, consuming system resources and competing with user-serving requests.
In most cases, undesirable process bursts are controlled by the default
max_concurrent_requests settings, which limits the number of tasks that can
execute simultaneously. You should not set
unless you are experiencing problems such as high error rates driven by
concurrent requests, or unless your application is extremely sensitive to
The following samples shows how to set
max_concurrent_requests to limit tasks
and also shows how to adjust the bucket size and rate based on your
application's needs and available resources:
queue: - name: optimize-queue rate: 20/s bucket_size: 40 max_concurrent_requests: 10
Configuring the maximum number of concurrent requests
If using the default
max_concurrent_requests settings are not sufficient, you
can change the settings for
max_concurrent_requests, as shown in the following
If your application queue has a rate of 20/s and a bucket size of 40, tasks in that queue execute at a rate of 20/s and can burst up to 40/s briefly. These settings work fine if task latency is relatively low; however, if latency increases significantly, you'll end up processing significantly more concurrent tasks. This extra processing load can consume extra instances and slow down your application.
For example, let's assume that your normal task latency is 0.3 seconds. At this latency, you'll process at most around 40 tasks simultaneously. But if your task latency increases to 5 seconds, you could easily have over 100 tasks processing at once. This increase forces your application to consume more instances to process the extra tasks, potentially slowing down the entire application and interfering with user requests.
You can avoid this possibility by setting
max_concurrent_requests to a lower
value. For example, if you set
max_concurrent_requests to 10, our example
queue maintains about 20 tasks/second when latency is 0.3 seconds. However, when
the latency increases over 0.5 seconds, this setting throttles the processing
rate to ensure that no more than 10 tasks run simultaneously.
queue: # Set the max number of concurrent requests to 50 - name: optimize-queue rate: 20/s bucket_size: 40 max_concurrent_requests: 10
Defining pull queues
You can specify any named queue as a pull queue by adding the
mode: pull directive to
If you are using the Task Queue REST API, you also need to create an access control list (ACL) using the acl directive. This directive allows you to restrict access to user email addresses corresponding to an account hosted by Google.
acl element has two available parameters:
user_email: enables the user to list, get, lease, delete, and update tasks.
writer_email: enables the user to insert tasks.
In order to access all functions of the API, a developer's email address must be
specified both as a
user_email and a
writer_email. The following code
snippet creates a pull queue named pull-queue with two users in the ACL. The
email@example.com can access all API calls:
queue: - name: pull-queue mode: pull acl: # Can list, get, lease, delete, and update tasks: - user_email: firstname.lastname@example.org # Can insert tasks: - writer_email: email@example.com # Can insert tasks, in addition to rights granted by being a user_email above: - writer_email: firstname.lastname@example.org
Configuring retry attempts for failed tasks
Tasks executing in the task queue can fail for many reasons. If a task fails to execute by returning any HTTP status code outside of the range 200–299, App Engine retries the task until it succeeds. By default, the system gradually reduces the retry rate to avoid flooding your application with too many requests, but schedules retry attempts to recur at a maximum of once per hour until the task succeeds.
Push queues and pull queues differ in how they retry tasks, as described in the following sections.
Retrying tasks in push queues
In push queues, you can specify your own scheme for task retries by adding the
retry_parameters directive in
queue.yaml. This addition
allows you to specify the maximum number of times to retry failed tasks in a
specific queue. You can also set a time limit for retry attempts and control the
interval between attempts.
The following example demonstrates various retry scenarios:
fooqueue, tasks are retried at least seven times and for up to two days from the first execution attempt. After both limits are passed, it fails permanently.
barqueue, App Engine attempts to retry tasks, increasing the interval linearly between each retry until reaching the maximum backoff and retrying indefinitely at the maximum interval. So the intervals between requests are 10s, 20s, 30s, ..., 190s, 200s, 200s, ...
bazqueue, the interval increases to twice the minimum backoff and retries indefinitely at the maximum interval. So the intervals between requests are 10s, 20s, 40s, 80s, 120s, 160s, 200s, 200s, ...
queue: - name: fooqueue rate: 1/s retry_parameters: task_retry_limit: 7 task_age_limit: 2d - name: barqueue rate: 1/s retry_parameters: min_backoff_seconds: 10 max_backoff_seconds: 200 max_doublings: 0 - name: bazqueue rate: 1/s retry_parameters: min_backoff_seconds: 10 max_backoff_seconds: 200 max_doublings: 3
Retrying tasks in pull queues
In pull queues, you can specify the number of times to retry a task using the
retry_parameters directive with the
task_retry_limit field. The system
counts each time you lease a task using
When the count exceeds the
task_retry_limit, the system deletes the task
automatically. If you don't specify a
task_retry_limit, the system never
deletes a task automatically.
The following code sample shows how to specify a pull queue limited to seven retry attempts:
queue: - name: pull-queue mode: pull retry_parameters: task_retry_limit: 7
queue.yaml file is a YAML file whose root directive is
This directive contains zero or more
queue directives, one for each named
queue directive contains configuration for a queue. It can contain the
acl (pull queues only)
- Experimental. Creates an access control list (ACL) for Pull Queues using the
Task Queue REST API. The ACL is composed of the specified email addresses.
Accepts email addresses only from a
Enter each email address on its own line as follows
- user_email: email@example.com. For more information, see Defining Pull Queues.
bucket_size (push queues only)
Task queues use a "token bucket" algorithm for dequeueing tasks. The bucket size limits how fast the queue is processed when many tasks are in the queue and the rate is high. The maximum value for bucket size is 100. This allows you to have a high rate so processing starts shortly after a task is enqueued, but still limit resource usage when many tasks are enqueued in a short period of time.
bucket_sizeis specified for a queue, the default value is 5.
For more information on the algorithm, see the Wikipedia article on token buckets.
max_concurrent_requests (push queues only)
Sets the maximum number of tasks that can be executed simultaneously from the specified queue. The value is an integer. By default, the limit is 1000 tasks per queue.
Restricting the number of concurrent tasks gives you more control over the queue's rate of execution and can prevent too many tasks from running at once. It can also prevent datastore contention and make resources available for other queues or online processing.
- Identifies the queue mode. This setting defaults to
push, which identifies a queue as a push queue. If you wish to use pull queues, set the mode to
The name of the queue. This is the name you specify when you call
A queue name can contain uppercase and lowercase letters, numbers, and hyphens. The maximum length for a queue name is 100 characters.
rate (push queues only)
How often tasks are processed on this queue. The value is a number followed by a slash and a unit of time, where the unit is
hfor hours, or
dfor days. For example, the value
5/msays tasks will be processed at a rate of 5 times per minute.
If the number is
0/s), the queue is considered "paused," and no tasks are processed.
Configures retry attempts for failed tasks. This addition allows you to specify the maximum number of times to retry failed tasks in a specific queue. You can also set a time limit for retry attempts and control the interval between attempts.
- The maximum number of retry attempts for a failed task. If specified
task_age_limit, App Engine retries the task until both limits are reached.
task_age_limit(push queues only)
- The time limit for retrying a failed task, measured from when the task
was first run. The value is a number followed by a unit of time, where
the unit is
hfor hours, or
dfor days. For example, the value
5dspecifies a limit of five days after the task's first execution attempt. If specified with
task_retry_limit, App Engine retries the task until both limits are reached.
min_backoff_seconds(push queues only)
- The minimum number of seconds to wait before retrying a task after it fails.
max_backoff_seconds(push queues only)
- The maximum number of seconds to wait before retrying a task after it fails.
max_doublings(push queues only)
- The maximum number of times that the interval between failed task retries will be doubled before the increase becomes constant. The constant is: 2**(max_doublings - 1) * min_backoff_seconds.
target (push queues only)
A string naming a service/version, a frontend version, or a backend, on which to execute all of the tasks enqueued onto this queue.
The string is prepended to the domain name of your app when constructing the HTTP request for a task. For example, if your app ID is
my-appand you set the target to
my-version.my-service, the URL hostname will be set to
If target is unspecified, then tasks are invoked on the same version of the application where they were enqueued. So, if you enqueued a task from the default application version without specifying a target on the queue, the task is invoked in the default application version. Note that if the default application version changes between the time that the task is enqueued and the time that it executes, then the task will run in the new default version.
Updating task queue configuration
The task queue configuration for the app is updated when you upload the
application using the
appcfg.sh update command. You also
have the option to update just the task queue configuration
without uploading the full application by running the
./appengine-java-sdk/bin/appcfg.sh update_queues myapp/war
See Deploying a Java App for more information.