Leasing Pull Tasks

Once tasks are in a pull queue, a worker can lease them. After the tasks are processed the worker must delete them.

This API is supported for first-generation runtimes and can be used when upgrading to corresponding second-generation runtimes. If you are updating to the App Engine Java 11/17 runtime, refer to the migration guide to learn about your migration options for legacy bundled services.

Before you begin

• Create a pull queue.
• Create tasks and add them to the pull queue.

Important context

• This method is only applicable to workers that are running within a service in the standard environment.
• When you use pull queues, you are responsible for scaling your workers based on your processing volume.

Leasing tasks

After the tasks are in the queue, a worker can lease one or more of them using the leaseTasks() method. There may be a short delay before tasks recently added using add() become available via leaseTasks().

When you request a lease, you specify the number of tasks to lease (up to a maximum of 1,000 tasks) and the duration of the lease in seconds (up to a maximum of one week). The lease duration needs to be long enough to ensure that the slowest task will have time to finish before the lease period expires. You can modify a task lease using modifyTaskLease().

Leasing a task makes it unavailable for processing by another worker, and it remains unavailable until the lease expires.

Note: leaseTasks() operates only on pull queues. If you attempt to lease tasks added in a push queue, App Engine throws an exception.

The following code sample leases tasks from the queue pull-queue for one hour:

List<TaskHandle> tasks = q.leaseTasks(3600, TimeUnit.SECONDS, numberOfTasksToLease);

Batching with task tags

Beta

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Not all tasks are alike; your code can "tag" tasks and then choose tasks to lease by tag. The tag acts as a filter.

q.add(
    TaskOptions.Builder.withMethod(TaskOptions.Method.PULL)
        .payload(content.toString())
        .tag("process".getBytes()));

Then lease the filtered tasks:

// Lease only tasks tagged with "process"
List<TaskHandle> tasks =
    q.leaseTasksByTag(3600, TimeUnit.SECONDS, numberOfTasksToLease, "process");
// You can also specify a tag to lease via LeaseOptions passed to leaseTasks.

Regulating polling rates

Workers that poll the queue for tasks to lease should detect whether they are attempting to lease tasks faster than the queue can supply them. If this failure occurs, the following exceptions from leaseTasks() can be generated:

• TransientFailureException
• ApiDeadlineExceededException

Your code must catch these exceptions, back off from calling leaseTasks(), and then try again later. To avoid this problem, consider setting a higher RPC deadline when calling leaseTasks(). You should also back off when a lease request returns an empty list of tasks.

If you generate more than 10 LeaseTasks requests per queue per second, only the first 10 requests will return results. If requests exceed this limit, OK is returned with zero results.

Monitoring tasks in the Google Cloud console

To view information about all the tasks and queues in your application:

1. Open the Cloud Tasks page in the Google Cloud console and look for the Pull value in column Type.

   Go to Cloud Tasks

2. Click on the name of the queue in which you are interested, opening the queue details page. It displays all of the tasks in the selected queue.

Deleting tasks

Once a worker completes a task, it needs to delete the task from the queue. If you see tasks remaining in a queue after a worker finishes processing them, it is likely that the worker failed; in this case, the tasks will be processed by another worker.

You can delete an individual task or a list of tasks using deleteTask(). You must know the name of a task in order to delete it. You can find task names in the Task object returned by leaseTasks().

The following code sample demonstrates how to delete a task from a queue:

q.deleteTask(task);