Quickstart: Manage Labeling Tasks

There are two kinds of roles in a labeler group: labelers and labeler managers. - Labelers are the actual human reviewers for your task. - Labeler managers manage the labelers. They can track the task progress and add/remove labelers to/from a task.

A typical labeler group usually has multiple labelers and at least one manager.

Both labelers and managers use the DataCompute console to handle the labeling tasks.

Before you begin

Complete the previous Quickstarts before proceeding.

Manage Human Review Queues

  1. In the Google Cloud console under Document AI > Human-in-the-Loop, you can see the counts by project for the status of documents.

    image

  2. From the Manager console (accessible from the link sent in email) the Manager can manage Tasks, Labelers and Assignments:

    • The Tasks menu item lists tasks:

      image

      • By clicking on the 3 dots to the right of entry, you can:
        • Pause
        • Perform Assignment

    • The Labelers menu item lists Labelers:

      image

      • With the Add Labeler button, you can add Labelers.
        • Delete
        • Perform Assignment

    • The Assignments menu item manages Assignments:

      image

Receive Email Notifications

Once a labeling group is created by the customer, the labeler managers will receive an email similar to the one listed below. They can log in to the "Manager console" to manage labelers for the task.

image

Add Labelers

The managers need to add labelers (Google email addresses) to the labeler group in the DataCompute console. They can achieve this by following the screenshot below.

image

Assign Tasks

Once labelers are added to the labeler group, labeler managers need to explicitly assign them to the labeling task by following the screenshot below.

image

Notify Labelers

After the assignment, labeler managers need to explicitly notify all labelers to work on the task by simply forwarding them the emails (listed below) and asking them to go to the labeler console directly.

image

The labeler managers only need to add labelers and assign tasks to them for the first time. If the labeling task is assigned to an existing labeler group, all labelers (as well as labeler managers) in that group will receive an email (similar to the one below), which will take them to the labeling task.

image

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

Use the Google Cloud console to delete your processor and project if you do not need them.

What's next