Concepts
Following are some concepts and features used in this product:
Concept | Definition |
---|---|
Review | The process of visually comparing the extracted field values against actual values in the document and correcting any incorrect extractions, or adding missed extracted fields missed by the DocAI processors. |
Labeler | The human that reviews the extracted document. The customer can use their own workforce (Bring-your-own-labeler or BYOL) or use Google labelers for HITL Review. |
Task | A queue of extracted documents that labelers review. A processor generates a single task when configured for HITL Review. |
Labeler Workbench |
The UI used by a Labeler to review documents. The UI presents documents from the queue, that the labeler can review, correct and either submit or reject.
|
Answer Time | This is the time taken by a labeler to process a document. The Labeler Workbench tracks document submission time and presents efficiency analytics (e.g. for each labeler document review). |
Labeling Manager |
One or more labeling managers are assigned to a pool of labelers, so that they can:
|
Labeling Manager Console | UI used by a Labeling Manager to manage labeler pools and task assignments. Open console. |
Enqueue, Answered, Completed, Rejected Documents in a Task |
A task is a continual workflow. A document goes through the following states:
|
Single Task per Processor | We do not support multiple tasks per processor. If customers need to process a single document type (invoices, for example) in different tasks, they can configure multiple processors with HITL Review. |
Task Assignment vs Labeler Pools | Labeling Manager adds labelers to a pool. Once added, any labelers from the pool can be assigned to a task. Note, "Labeler pool" is not to be confused with the "group" of labelers assigned to a task. A Pool is managed at a Project level and is used to determine labeler access to the analytics and the tasks. Any labeler from the pool can be assigned to one of more tasks in the Project. |
Labeler Pool | A pool of labelers is created at a project level and not to be confused with task assignments. The Labeling Manager can assign any Any labeler Any labeler assigned to a task, so that multiple labelers can review documents in parallel and complete the task quicker. A labeler pool can be assigned to any task in the project by the customer. |
Validation filters and thresholds |
Extracted fields have a confidence score (0-100) representing the confidence that the DocAI extraction is accurate. Customers can configure the validation threshold for each field, so that only pages with fields that are below this validation threshold are enqueued for review, ; fields above the threshold are not enqueued.
There are 3 types of validation filters customers can configure:
|
Labeler Manager Analytics | The Labeling Manager gets analytics for each Task and each Labeler, including Enqueued, Answered, Skipped, Completed, Average Handling Time/document and total Answer time. Analytics are accessed in the Analytics tab of the Labeling Manager Console. |