Send a processing request
After you have set up your Google Cloud account and created a processor, you can send a request to your Document AI processor.
The code used to send the request is the same for all processors. You see differences in processor functioning in the information each processor outputs.
Using with the v1
API version of Document AI or in Google Cloud console, you
can send processing requests to that specific processor version. If you don't
specify a processor version, the default version is used.
For more information, see Managing processor versions.
Online processing
Online (synchronous) requests let you send a single document for processing.
Document AI immediately processes the request and returns a document
.
Send request to a processor
The following code samples show you how to send a request to a processor.
REST
This sample shows you how to provide document content (raw document content in bytes
via a base64 encoded string) in the rawDocument
object.
Alternatively, you could also specify inlineDocument
,
which is the same Document
JSON format returned
by Document AI. This allows you to chain requests by passing the same format back and forth
(for example, if you classify a document and then extract its content).
Before using any of the request data, make the following replacements:
- LOCATION: your processor's location, for example:
us
- United Stateseu
- European Union
- PROJECT_ID: Your Google Cloud project ID.
- PROCESSOR_ID: the ID of your custom processor.
- skipHumanReview: A boolean to disable human review (Supported by Human-in-the-Loop processors only.)
true
- skips human reviewfalse
- enables human review (default)
- MIME_TYPE†: One of the valid MIME type options.
- IMAGE_CONTENT†: One of the valid
Inline document content, represented as
a stream of bytes. For JSON representations, the base64
encoding (ASCII string) of your binary image data. This string should look similar to the
following string:
/9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
- FIELD_MASK: Specifies which fields to include in the
Document
output. This is a comma-separated list of fully qualified names of fields inFieldMask
format.- Example:
text,entities,pages.pageNumber
- Example:
- INDIVIDUAL_PAGES: A list of individual pages to process.
† This content can also be specified using base64-encoded content in the
inlineDocument
object.
HTTP method and URL:
POST https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID:process
Request JSON body:
{ "skipHumanReview": skipHumanReview, "rawDocument": { "mimeType": "MIME_TYPE", "content": "IMAGE_CONTENT" }, "fieldMask": "FIELD_MASK", "processOptions": { "individualPageSelector" { "pages": [INDIVIDUAL_PAGES] } } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID:process"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID:process" | Select-Object -Expand Content
If the request is successful, the server returns a 200 OK
HTTP status code and the
response in JSON format. The response body contains an instance of
Document
.
Send request to a processor version
Before using any of the request data, make the following replacements:
- LOCATION: your processor's location, for example:
us
- United Stateseu
- European Union
- PROJECT_ID: Your Google Cloud project ID.
- PROCESSOR_ID: the ID of your custom processor.
- PROCESSOR_VERSION: the processor version identifier. Refer to Select a processor version for more information. For example:
pretrained-TYPE-vX.X-YYYY-MM-DD
stable
rc
- skipHumanReview: A boolean to disable human review (Supported by Human-in-the-Loop processors only.)
true
- skips human reviewfalse
- enables human review (default)
- MIME_TYPE†: One of the valid MIME type options.
- IMAGE_CONTENT†: One of the valid
Inline document content, represented as
a stream of bytes. For JSON representations, the base64
encoding (ASCII string) of your binary image data. This string should look similar to the
following string:
/9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
- FIELD_MASK: Specifies which fields to include in the
Document
output. This is a comma-separated list of fully qualified names of fields inFieldMask
format.- Example:
text,entities,pages.pageNumber
- Example:
† This content can also be specified using base64-encoded content in the
inlineDocument
object.
HTTP method and URL:
POST https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:process
Request JSON body:
{ "skipHumanReview": skipHumanReview, "rawDocument": { "mimeType": "MIME_TYPE", "content": "IMAGE_CONTENT" }, "fieldMask": "FIELD_MASK" }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:process"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:process" | Select-Object -Expand Content
If the request is successful, the server returns a 200 OK
HTTP status code and the
response in JSON format. The response body contains an instance of
Document
.
C#
For more information, see the Document AI C# API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
For more information, see the Document AI Java API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Node.js
For more information, see the Document AI Node.js API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Python
For more information, see the Document AI Python API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Batch processing
Batch (asynchronous) requests let you send multiple documents in a single
request. Document AI responds with an operation
that you can poll for the status of the request. When this operation finishes,
it contains a BatchProcessMetadata
that points to the Cloud Storage bucket where the processed results are
stored.
If the input files you want to access are in a bucket in another project, you must provide access to that bucket before you can access the files. See setup file access.
Send request to a processor
The following code samples show you how to send a batch process request to a processor.
REST
This sample shows how to send a POST
request to the
batchProcess
method for large document asynchronous processing.
The example uses
the access token for a service account set up for the project using the Google Cloud CLI. For
instructions on installing the Google Cloud CLI, setting up a project with a service account, and
obtaining an access token, see Before you begin.
A batchProcess
request starts a long-running operation and
stores results in a Cloud Storage bucket. This sample also shows how to
get the status of this long-running operation after it has started.
Send the process request
Before using any of the request data, make the following replacements:
- LOCATION: your processor's location, for example:
us
- United Stateseu
- European Union
- PROJECT_ID: Your Google Cloud project ID.
- PROCESSOR_ID: the ID of your custom processor.
- INPUT_BUCKET_FOLDER†: A Cloud Storage
bucket/directory to read input files from, expressed in the following form:
gs://bucket/directory/
- MIME_TYPE: One of the valid MIME type options.
- OUTPUT_BUCKET_FOLDER: A Cloud Storage
bucket/directory to save output files to, expressed in the following form:
gs://bucket/directory/
- skipHumanReview: A boolean to disable human review (Supported by Human-in-the-Loop processors only.)
true
- skips human reviewfalse
- enables human review (default)
- FIELD_MASK: Specifies which fields to include in the
Document
output. This is a comma-separated list of fully qualified names of fields inFieldMask
format.- Example:
text,entities,pages.pageNumber
- Example:
† Instead of using gcsPrefix
to include all the files in a GCS folder, you can also
use documents
to individually list each file:
"inputDocuments": { "gcsDocuments": { "documents": [ { "gcsUri": "gs://BUCKET/PATH/TO/DOCUMENT1.ext", "mimeType": "MIME_TYPE" }, { "gcsUri": "gs://BUCKET/PATH/TO/DOCUMENT2.ext", "mimeType": "MIME_TYPE" } ] } }
HTTP method and URL:
POST https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID:batchProcess
Request JSON body:
{ "inputDocuments": { "gcsPrefix": { "gcsUriPrefix": "INPUT_BUCKET_FOLDER" } }, "documentOutputConfig": { "gcsOutputConfig": { "gcsUri": "OUTPUT_BUCKET_FOLDER", "fieldMask": "FIELD_MASK" } }, "skipHumanReview": BOOLEAN }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID:batchProcess"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID:batchProcess" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID" }
Send request to a processor version
Before using any of the request data, make the following replacements:
- LOCATION: your processor's location, for example:
us
- United Stateseu
- European Union
- PROJECT_ID: Your Google Cloud project ID.
- PROCESSOR_ID: the ID of your custom processor.
- PROCESSOR_VERSION: the processor version identifier. Refer to Select a processor version for more information. For example:
pretrained-TYPE-vX.X-YYYY-MM-DD
stable
rc
- INPUT_BUCKET_FOLDER†: A Cloud Storage
bucket/directory to read input files from, expressed in the following form:
gs://bucket/directory/
- MIME_TYPE: One of the valid MIME type options.
- OUTPUT_BUCKET_FOLDER: A Cloud Storage
bucket/directory to save output files to, expressed in the following form:
gs://bucket/directory/
- skipHumanReview: A boolean to disable human review (Supported by Human-in-the-Loop processors only.)
true
- skips human reviewfalse
- enables human review (default)
- FIELD_MASK: Specifies which fields to include in the
Document
output. This is a comma-separated list of fully qualified names of fields inFieldMask
format.- Example:
text,entities,pages.pageNumber
- Example:
† Instead of using gcsPrefix
to include all the files in a GCS folder, you can also
use documents
to individually list each file:
"inputDocuments": { "gcsDocuments": { "documents": [ { "gcsUri": "gs://BUCKET/PATH/TO/DOCUMENT1.ext", "mimeType": "MIME_TYPE" }, { "gcsUri": "gs://BUCKET/PATH/TO/DOCUMENT2.ext", "mimeType": "MIME_TYPE" } ] } }
HTTP method and URL:
POST https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:batchProcess
Request JSON body:
{ "inputDocuments": { "gcsPrefix": { "gcsUriPrefix": "INPUT_BUCKET_FOLDER" } }, "documentOutputConfig": { "gcsOutputConfig": { "gcsUri": "OUTPUT_BUCKET_FOLDER", "fieldMask": "FIELD_MASK" } }, "skipHumanReview": BOOLEAN }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:batchProcess"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:batchProcess" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID" }
If the request is successful, the Document AI API returns the name for your operation.
Get the results
To get the results of your request, you must send a GET
request to
the operations
resource. The following shows how to send such a request.
You can read more information in the Long-Running Operations documentation.
Before using any of the request data, make the following replacements:
- PROJECT_ID: Your Google Cloud project ID.
- LOCATION: the location where the LRO is running, for example:
us
- United Stateseu
- European Union
- OPERATION_ID: The ID of your operation. The ID is the last element of the name
of your operation. For example:
- Operation name:
projects/PROJECT_ID/locations/LOCATION/operations/bc4e1d412863e626
- Operation id:
bc4e1d412863e626
- Operation name:
HTTP method and URL:
GET https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID
To send your request, choose one of these options:
curl
Execute the following command:
curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://LOCATION-documentai.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.documentai.v1.BatchProcessMetadata", "state": "SUCCEEDED", "stateMessage": "Processed 1 document(s) successfully", "createTime": "TIMESTAMP", "updateTime": "TIMESTAMP", "individualProcessStatuses": [ { "inputGcsSource": "INPUT_BUCKET_FOLDER/DOCUMENT1.ext", "status": {}, "outputGcsDestination": "OUTPUT_BUCKET_FOLDER/OPERATION_ID/0", "humanReviewStatus": { "state": "ERROR", "stateMessage": "Sharded document protos are not supported for human review." } } ] }, "done": true, "response": { "@type": "type.googleapis.com/google.cloud.documentai.v1.BatchProcessResponse" } }
The response body contains an instance of Operation
with information about the status of the operation.
If the operation has completed successfully,
the metadata
field will be populated with an instance of BatchProcessMetadata
which contains information about the processed documents.
C#
For more information, see the Document AI C# API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
For more information, see the Document AI Go API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
For more information, see the Document AI Java API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Node.js
For more information, see the Document AI Node.js API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Python
For more information, see the Document AI Python API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
For more information, see the Document AI Go API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Ruby
For more information, see the Document AI Ruby API reference documentation.
To authenticate to Document AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Create document batches with Python SDK
Batch processing allows a maximum of 1,000 files per request. If you have more documents to process, then you must divide the documents into multiple batches for processing.
Document AI Toolbox is an SDK for Python that provides utility functions for Document AI. One of the functions is to create batches of documents for processing from a Cloud Storage folder.
Refer to Handling the processing response for more information on how Document AI Toolbox assists with post-processing.
Code Samples
The following code samples demonstrate how to use Document AI Toolbox.