With the Task Queue API, applications can perform work outside of a user request, initiated by a user request. If an app needs to execute some background work, it can use the Task Queue API to organize that work into small, discrete units, called tasks. The app adds tasks to task queues to be executed later.
- Task Queue concepts
- Task concepts
Task Queue concepts
Tasks queues are an efficient and powerful tool for background processing; they allow your application to define tasks, add them to a queue, and then use the queue to process them in aggregate. You name queues and configure their properties in a configuration file named
queue.yaml. A queue can be one of two types—push or pull—based on your needs.
Push queues function only within the App Engine environment. These queues are the best choice for applications whose tasks work only with App Engine tools and services. With push queues, you simply configure a queue and add tasks to it. App Engine handles the rest. Push queues are easier to implement, but are restricted to use within App Engine. For more information about push queues and examples of how to use them, see Using Push Queues.
In summary, push queues allow you to process tasks within App Engine at a steady rate and App Engine scales computing resources according to the number of tasks in your queue.
The default queue
For convenience, App Engine provides a default push queue for each application (there is no default pull queue). If you do not name a queue for a task, App Engine automatically inserts it into the default queue. You can use this queue immediately without any additional configuration. All modules and versions of the same application share the same default task queue.
The default queue is preconfigured with a throughput rate of 5 task invocations per second. If you want to change the preconfigured settings, simply define a queue named
queue.yaml. Code may always insert new tasks into the default queue, but if you wish to disable execution of these tasks, you may do so by clicking the Pause Queue button in the Cloud Platform Console Task queues page .
While the default queue makes it easy to enqueue tasks with no configuration, you can also create custom queues by defining them in
queue.yaml. Custom queues allow you to more effectively handle task processing by grouping similar types of tasks. You can control the processing rate—and a number of other settings—based specifically on the type of task in each queue.
All versions of an application share the same named task queues. A queue name can contain uppercase and lowercase letters, numbers, and hyphens. The maximum length for a queue name is 100 characters.
For more information about configuring queues in
queue.yaml, please see Task Queue Configuration.
Task queues in the Cloud Platform Console
You can manage task queues for an application using the Cloud Platform Console Task queues page. The Task Queue tab lists all of the queues in the application. Clicking on a queue name brings up the Task Queue Details page where you can see all of the tasks scheduled to run in a queue and you can manually delete individual tasks or purge every task from a queue. This is useful if a task in a push queue cannot be completed successfully and is stuck waiting to be retried. You can also pause and resume a queue on this page.
You can view details of individual tasks by clicking the task name from the list of tasks on the Task Queue Details page. This page allows you to debug why a task did not run successfully. You can also see information about the previous run of the task as well as the task's body.
A task is a unit of work to be performed by the application. Each task is an object of the
PushTask class. Each Task object contains an endpoint (with a request handler for the task and an optional data payload that parameterizes the task). You can enqueue push tasks to a queue defined in
You may optionally assign a name to a task. Task names must be unique within a queue. Once a named task is added to a queue, any subsequent attempt to insert another task with the same name into the same queue will fail. If a task successfully executes or is deleted, you must wait for a period of 10 days before you can insert a new task with that same name.
Note that task names do not provide an absolute guarantee of once-only semantics. In extremely rare cases, multiple calls to create a task of the same name may succeed, but in this event, only one of the tasks would be executed. It's also possible in exceptional cases for a task to run more than once.
A task name can contain uppercase and lowercase letters, numbers, underscores, and hyphens. The maximum length for a task name is 500 characters.
A task is deleted immediately upon successful execution or deletion, or after a maximum number of failures. The task name can then be re-used after 10 days. Attempting to create another task with that same name during this 10-day period will result in an "item exists" error. To avoid issues with task name re-use, we recommend that you let App Engine generate the task name automatically.