Toolbox - Create document batches
Stay organized with collections
Save and categorize content based on your preferences.
Create batches of documents for processing with batch_process_documents()
.
Explore further
For detailed documentation that includes this code sample, see the following:
Code sample
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],[],[[["\u003cp\u003eThe \u003ccode\u003ecreate_batches_sample()\u003c/code\u003e function creates batches of documents from a specified Google Cloud Storage (GCS) location for processing.\u003c/p\u003e\n"],["\u003cp\u003eBatches are created from documents located within a specified GCS bucket and prefix, with the size of each batch determined by the \u003ccode\u003ebatch_size\u003c/code\u003e parameter.\u003c/p\u003e\n"],["\u003cp\u003eThe code provides a method to generate batches that can be utilized as input for \u003ccode\u003ebatch_process_documents()\u003c/code\u003e, which is further explained in the provided link to send a batch processing request.\u003c/p\u003e\n"],["\u003cp\u003eTo use the code, you need to configure Application Default Credentials (ADC) for authentication with Document AI, as well as setting the GCS bucket name, GCS prefix, and the desired batch size.\u003c/p\u003e\n"]]],[],null,["# Toolbox - Create document batches\n\nCreate batches of documents for processing with `batch_process_documents()`.\n\nExplore further\n---------------\n\n\nFor detailed documentation that includes this code sample, see the following:\n\n- [Document AI Toolbox client libraries](/document-ai/docs/toolbox)\n- [Handle processing response](/document-ai/docs/handle-response)\n- [Send a processing request](/document-ai/docs/send-request)\n\nCode sample\n-----------\n\n### Python\n\n\nFor more information, see the\n[Document AI Python API\nreference documentation](/python/docs/reference/documentai/latest).\n\n\nTo authenticate to Document AI, set up Application Default Credentials.\nFor more information, see\n\n[Set up authentication for a local development environment](/docs/authentication/set-up-adc-local-dev-environment).\n\n\n from google.cloud import documentai\n from google.cloud.documentai_toolbox import https://cloud.google.com/python/docs/reference/documentai-toolbox/latest/google.cloud.documentai_toolbox.utilities.gcs_utilities.html\n\n # TODO(developer): Uncomment these variables before running the sample.\n # Given unprocessed documents in path gs://bucket/path/to/folder\n # gcs_bucket_name = \"bucket\"\n # gcs_prefix = \"path/to/folder\"\n # batch_size = 50\n\n\n def create_batches_sample(\n gcs_bucket_name: str,\n gcs_prefix: str,\n batch_size: int = 50,\n ) -\u003e None:\n # Creating batches of documents for processing\n batches = https://cloud.google.com/python/docs/reference/documentai-toolbox/latest/google.cloud.documentai_toolbox.utilities.gcs_utilities.html.https://cloud.google.com/python/docs/reference/documentai-toolbox/latest/google.cloud.documentai_toolbox.utilities.gcs_utilities.html(\n gcs_bucket_name=gcs_bucket_name, gcs_prefix=gcs_prefix, batch_size=batch_size\n )\n\n print(f\"{len(batches)} batch(es) created.\")\n for batch in batches:\n print(f\"{len(batch.gcs_documents.documents)} files in batch.\")\n print(batch.gcs_documents.documents)\n\n # Use as input for batch_process_documents()\n # Refer to https://cloud.google.com/document-ai/docs/send-request\n # for how to send a batch processing request\n request = documentai.https://cloud.google.com/python/docs/reference/documentai/latest/google.cloud.documentai_v1.types.BatchProcessRequest.html(\n name=\"processor_name\", input_documents=batch\n )\n print(request)\n\nWhat's next\n-----------\n\n\nTo search and filter code samples for other Google Cloud products, see the\n[Google Cloud sample browser](/docs/samples?product=documentai)."]]