This page explains the differences between using the
Cloud Tasks API to manage queues
and using the upload of a Cloud Tasks
file to accomplish the same ends. It also discusses some of the
pitfalls of mixing mechanisms and how to deal with common problems.
The Cloud Tasks API provides an App Engine-independent interface to
the App Engine Task Queue
service. As part of that interface, it provides the ability to manage queues,
including doing so through the console or the
gcloud command. Queues that are
created by Cloud Tasks API are accessible from the App Engine SDK and vice
versa. To maintain compatibility, it is possible to use the configuration
file used by the App Engine SDK,
queue.yaml, to also create and configure
queues to be used through the Cloud Tasks API. However, mixing configuration
via file with configuration through the Cloud Tasks API can produce unexpected consequences.
Pitfalls of mixing
queue.yaml with Cloud Tasks queue management methods
For the underlying service,
queue.yaml files are definitive. Uploading a
queue.yaml that omits existing queues in your project, no matter how they were created, causes those queues to be disabled, or paused. Thus if you use the Cloud Tasks API to call
UpdateQueue and then upload a
queue.yaml file that omits them, the queues that were created in the Cloud Tasks calls are disabled.
Consider the following scenario:
CreateQueueto create a queue named "cloud-tasks-queue".
queue.yamlfile with the following contents:
queue: - name: queue-yaml-queue
What is the current state of queues in this project? The queue named
"cloud-tasks-queue" and any other queues that existed prior are in
DISABLED state, and the queue named "queue-yaml-queue" is in
This behavior might be surprising if you create queues through the Cloud Tasks API. The instructions below explain how to resume a disabled queue.
Similarly, if a queue is disabled in the Cloud Tasks API but later appears in an
queue.yaml file, that queue is resumed.
If a queue is deleted with the
DeleteQueue method and later appears in a
queue.yaml file, the
queue.yaml upload can fail because queue names are not
allowed to be reused for several days after deletion.
If you are new to Cloud Tasks or App Engine, use the Cloud Tasks API
exclusively to manage your queues and avoid the use of
altogether. Cloud Tasks queue management methods give users more choice in creating, updating, and deleting queues.
If, however, you are an existing
queue.yaml user, you should only consider switching to queue management methods if you understand the
pitfalls of mixing
queue.yaml with Cloud Tasks queue management methods.
To prevent users from mixing task management methods, one option is to
create a web app or command line tool that all users must use to create, update,
and delete queues. Whether that tool uses Cloud Tasks queue
management methods or
queue.yaml is an implementation detail of the tool
that users do not need to worry about. If users are required to use the tool,
then you can guarantee that there is no inadvertent mixing of
Cloud Tasks queue management methods and
To help enforce use of such a tool, you can grant queue admin roles to the tool
and require users to authenticate to use the tool. To learn more about access
Secure queue configuration.
You can inspect your project's Admin Activity audit logs to retrieve the history of queue configuration changes including queue creations, updates, and deletions:
gcloud logging read \ 'protoPayload.methodName= (com.google.appengine.legacy.queue_created OR com.google.appengine.legacy.queue_updated OR google.cloud.tasks.v2.CloudTasks.CreateQueue OR google.cloud.tasks.v2.CloudTasks.UpdateQueue OR google.cloud.tasks.v2.CloudTasks.DeleteQueue)'
For example, if an existing queue is disabled by a
queue.yaml upload, a
"Disabled queue '[QUEUE_NAME]'" message would appear in the audit log through
How to resume a queue disabled by a
queue.yaml with Cloud Tasks queue management methods,
queue.yaml file might accidentally disable a queue created through the Cloud Tasks API.
To resume the queue, you can either call
ResumeQueue on the queue or add it
queue.yaml and upload. Be aware that if you had previously set a custom
queue.yaml configuration for the queue,
ResumeQueue resets the queue to the default
rate. This is reflected in the
field of the response to
If you use
queue.yaml to create your queues, by default you can create a maximum of 100 queues. Queues created
using the Cloud Tasks API have a default maximum of 1,000 queues. As in
other cases, mixing
queue.yaml and Cloud Tasks API methods can produce unexpected
results. For example, suppose you create some queues using
queue.yaml, and then get a
quota increase to, for example, 2,000. If you then subsequently use the Cloud Tasks API
method of creating further queues, you will get out of quota errors. To remedy this,
file a request using Edit Quotas from the Quotas page of
Google Cloud console.
Additional information about Cloud Tasks queue management methods
Queue configuration and queue startup delay
Changes to queue configuration can require several minutes to take effect. For
example, after calling
UpdateQueue, several minutes might
pass before you can successfully call
CreateTask on that queue.
Cloud Tasks and the
default App Engine queue
The App Engine queue named "default" is given special treatment in the App Engine SDK and in the Cloud Tasks API.
default queue does not already exist, it is created in the following
- When a task is first added to the
defaultqueue using the App Engine SDK.
- When a
queue.yamlfile that specifies a
defaultqueue is uploaded.
UpdateQueueis called to create the
To preserve compatibility with App Engine, Cloud Tasks enforces the following restrictions:
- If a queue named "default" is created, it must be a queue using App Engine tasks.
- Once created, users cannot delete the
In the Cloud Tasks API, the following also applies to the
- The Cloud Tasks API does not automatically create the
defaultqueue or any other queues.
- Just like any other queue, calling
defaultqueue results in a not found error if the call is made before the queue is created.
- Similarly, the
defaultqueue does not appear in the output of
ListQueuesbefore it is created.
- The configuration of the
defaultqueue can be changed with the