Making batch predictions

After you have created (trained) a model you can make an asynchronous prediction request for a batch of images using the batchPredict method. The batchPredict method applies annotations to your image based on the objects identified by your model.

Batch prediction often offers a lower cost per inference and higher throughput than synchronous (online) prediction. However, batch prediction produces a long-running operation (LRO), meaning that results are only available once the LRO has completed.

The maximum lifespan for a custom model is 18 months. You must create and train a new model to continue annotating content after that time.

Batch prediction

You can request annotations (predictions) for images by using the batchPredict command. The batchPredict command takes, as input, a CSV file stored in your Google Cloud Storage location that contains the paths to the images to annotate. Each line specifies a separate path to an image in Google Cloud Storage. For example:

batch_prediction.csv:

gs://my-cloud-storage-bucket/prediction_files/image1.jpg
gs://my-cloud-storage-bucket/prediction_files/image2.jpg
gs://my-cloud-storage-bucket/prediction_files/image3.jpg
gs://my-cloud-storage-bucket/prediction_files/image4.jpg
gs://my-cloud-storage-bucket/prediction_files/image5.jpg
gs://my-cloud-storage-bucket/prediction_files/image6.png

If your filenames have spaces in them use quotation marks around the Cloud Storage location. For example:

• "gs://my-cloud-storage-bucket/prediction_files/image filename with spaces.jpg"

Depending on the number of images that you specified in your CSV file, the batch predict task can take some time to complete. Even on a small number of images batch prediction will take at minimum 30 minutes to complete.

REST

Before using any of the request data, make the following replacements:

• project-id: your GCP project ID.
• location-id: A valid location identifier. Currently you must use the following value:
  • us-central1
• model-id: the ID of your model, from the response when you created the model. The ID is the last element of the name of your model. For example:
  • model name: projects/project-id/locations/location-id/models/IOD4412217016962778756
  • model id: IOD4412217016962778756
• input-storage-path: the path to a CSV file stored on Google Cloud Storage. The requesting user must have at least read permission to the bucket.
• output-storage-bucket: a Google Cloud Storage bucket/directory to save output files to, expressed in the following form: gs://bucket/directory/. The requesting user must have write permission to the bucket.

Field-specific considerations:

• params.score_threshold - A value between 0.0 and 1.0. Only results with scores greater or equal to this value will be returned.

HTTP method and URL:

POST https://automl.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID:batchPredict

Request JSON body:

{
  "inputConfig": {
    "gcsSource": {
       "inputUris": [ "INPUT_STORAGE_PATH" ]
    }
  },
  "outputConfig": {
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_STORAGE_BUCKET"
    }
  },
  "params": {
    "score_threshold": "0.0"
  }
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by executing gcloud init or gcloud auth login, or by using Cloud Shell, which automatically logs you into the gcloud CLI. You can check the currently active account by executing gcloud auth list.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-id" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    "https://automl.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID:batchPredict"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by executing gcloud init or gcloud auth login, or by using Cloud Shell, which automatically logs you into the gcloud CLI. You can check the currently active account by executing gcloud auth list.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-id" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://automl.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/models/MODEL_ID:batchPredict" | Select-Object -Expand Content

Response:

You should see output similar to the following:

{
  "name": "projects/PROJECT_ID/locations/LOCATION_ID/operations/IOD926615623331479552",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.automl.v1.OperationMetadata",
    "createTime": "2019-06-19T21:28:35.302067Z",
    "updateTime": "2019-06-19T21:28:35.302067Z",
    "batchPredictDetails": {
      "inputConfig": {
        "gcsSource": {
          "inputUris": [
            "INPUT_STORAGE_PATH"
          ]
        }
      }
    }
  }
}

You can use the operation ID (IOD926615623331479552, in this case) to get the status of the task. For an example, see Working with long-running operations.

Depending on the number of images that you specified in your CSV file, the batch predict task can take some time to complete. Even on a small number of images batch prediction will take at minimum 30 minutes to complete.

Once the operation has completed, the state shows as DONE and your results are written to the Google Cloud Storage file you specified:

{
  "name": "projects/PROJECT_ID/locations/LOCATION_ID/operations/IOD926615623331479552",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.automl.v1.OperationMetadata",
    "createTime": "2019-06-19T21:28:35.302067Z",
    "updateTime": "2019-06-19T21:57:18.310033Z",
    "batchPredictDetails": {
      "inputConfig": {
        "gcsSource": {
          "inputUris": [
            "INPUT_STORAGE_PATH"
          ]
        }
      },
      "outputInfo": {
        "gcsOutputDirectory": "gs://STORAGE_BUCKET_VCM/SUBDIRECTORY/prediction-8370559933346329705-YYYY-MM-DDThh:mm:ss.sssZ"
      }
    }
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.automl.v1.BatchPredictResult"
  }
}

See the Output JSONL files section below for a sample output file.

Java

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

Open in Editor View on GitHub Feedback

import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.automl.v1.BatchPredictInputConfig;
import com.google.cloud.automl.v1.BatchPredictOutputConfig;
import com.google.cloud.automl.v1.BatchPredictRequest;
import com.google.cloud.automl.v1.BatchPredictResult;
import com.google.cloud.automl.v1.GcsDestination;
import com.google.cloud.automl.v1.GcsSource;
import com.google.cloud.automl.v1.ModelName;
import com.google.cloud.automl.v1.OperationMetadata;
import com.google.cloud.automl.v1.PredictionServiceClient;
import java.io.IOException;
import java.util.concurrent.ExecutionException;

class BatchPredict {

  static void batchPredict() throws IOException, ExecutionException, InterruptedException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    String modelId = "YOUR_MODEL_ID";
    String inputUri = "gs://YOUR_BUCKET_ID/path_to_your_input_csv_or_jsonl";
    String outputUri = "gs://YOUR_BUCKET_ID/path_to_save_results/";
    batchPredict(projectId, modelId, inputUri, outputUri);
  }

  static void batchPredict(String projectId, String modelId, String inputUri, String outputUri)
      throws IOException, ExecutionException, InterruptedException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (PredictionServiceClient client = PredictionServiceClient.create()) {
      // Get the full path of the model.
      ModelName name = ModelName.of(projectId, "us-central1", modelId);
      GcsSource gcsSource = GcsSource.newBuilder().addInputUris(inputUri).build();
      BatchPredictInputConfig inputConfig =
          BatchPredictInputConfig.newBuilder().setGcsSource(gcsSource).build();
      GcsDestination gcsDestination =
          GcsDestination.newBuilder().setOutputUriPrefix(outputUri).build();
      BatchPredictOutputConfig outputConfig =
          BatchPredictOutputConfig.newBuilder().setGcsDestination(gcsDestination).build();
      BatchPredictRequest request =
          BatchPredictRequest.newBuilder()
              .setName(name.toString())
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .build();

      OperationFuture<BatchPredictResult, OperationMetadata> future =
          client.batchPredictAsync(request);

      System.out.println("Waiting for operation to complete...");
      BatchPredictResult response = future.get();
      System.out.println("Batch Prediction results saved to specified Cloud Storage bucket.");
    }
  }
}

Node.js

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

Open in Editor View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const modelId = 'YOUR_MODEL_ID';
// const inputUri = 'gs://YOUR_BUCKET_ID/path_to_your_input_csv_or_jsonl';
// const outputUri = 'gs://YOUR_BUCKET_ID/path_to_save_results/';

// Imports the Google Cloud AutoML library
const {PredictionServiceClient} = require('@google-cloud/automl').v1;

// Instantiates a client
const client = new PredictionServiceClient();

async function batchPredict() {
  // Construct request
  const request = {
    name: client.modelPath(projectId, location, modelId),
    inputConfig: {
      gcsSource: {
        inputUris: [inputUri],
      },
    },
    outputConfig: {
      gcsDestination: {
        outputUriPrefix: outputUri,
      },
    },
  };

  const [operation] = await client.batchPredict(request);

  console.log('Waiting for operation to complete...');
  // Wait for operation to complete.
  const [response] = await operation.promise();
  console.log(
    `Batch Prediction results saved to Cloud Storage bucket. ${response}`
  );
}

batchPredict();

Python

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

View on GitHub Feedback

from google.cloud import automl

# TODO(developer): Uncomment and set the following variables
# project_id = "YOUR_PROJECT_ID"
# model_id = "YOUR_MODEL_ID"
# input_uri = "gs://YOUR_BUCKET_ID/path/to/your/input/csv_or_jsonl"
# output_uri = "gs://YOUR_BUCKET_ID/path/to/save/results/"

prediction_client = automl.PredictionServiceClient()

# Get the full path of the model.
model_full_id = f"projects/{project_id}/locations/us-central1/models/{model_id}"

gcs_source = automl.GcsSource(input_uris=[input_uri])

input_config = automl.BatchPredictInputConfig(gcs_source=gcs_source)
gcs_destination = automl.GcsDestination(output_uri_prefix=output_uri)
output_config = automl.BatchPredictOutputConfig(gcs_destination=gcs_destination)

response = prediction_client.batch_predict(
    name=model_full_id, input_config=input_config, output_config=output_config
)

print("Waiting for operation to complete...")
print(
    f"Batch Prediction results saved to Cloud Storage bucket. {response.result()}"
)

Additional languages

C#: Please follow the C# setup instructions on the client libraries page and then visit the AutoML Vision Object Detection reference documentation for .NET.

PHP: Please follow the PHP setup instructions on the client libraries page and then visit the AutoML Vision Object Detection reference documentation for PHP.

Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the AutoML Vision Object Detection reference documentation for Ruby.

Note: You can use your model to make batch predictions even if your model is not deployed.

Output JSONL files

When the batch predict task is complete, the output of the prediction is stored in the Google Cloud Storage location that you specified in your command.

In your output storage location (with your chosen object prefix) the files image_object_detection_1.jsonl, image_object_detection_2.jsonl,..., image_object_detection_N.jsonl will be created, where N may be 1, and depends on the total number of the successfully predicted images and annotations.

A single image will be listed only once with all its annotations, and its annotations will never be split across files.

You can specify a minimum score in the request to limit the results returned.

Each JSONL file will contain, per line, a JSON representation of a proto that wraps image's "ID" : "<id_value>" followed by a list of zero or more AnnotationPayload protos (called annotations), which have imageObjectDetection detail populated.

If prediction for any image failed (partially or completely), then an additional errors_1.jsonl, errors_2.jsonl,..., errors_N.jsonl files will be created (N depends on total number of failed predictions). These files will have a JSON representation of a proto that wraps the same "ID" : "<id_value>" but here followed by exactly one google.rpc.Status containing only code and message fields.

Example JSONL file (a single .jsonl file with 2 lines/file annotations):

image_object_detection_0.jsonl

Line 1 (salad4.jpg annotations JSON)

{
  "ID": "gs://my-storage-bucket/salad4.jpg",
  "annotations": [
    {
      "annotation_spec_id": "6133348538618216448",
      "display_name": "Tomato",
      "image_object_detection": {
        "bounding_box": {
          "normalized_vertices": [
            {
              "x": 0.51460195,
              "y": 0.60645401
            },
            {
              "x": 0.64264798,
              "y": 0.71467251
            }
          ],
          "vertices": []
        },
        "score": 0.87153709
      }
    },
    {
      "annotation_spec_id": "3827505529404522496",
      "display_name": "Salad",
      "image_object_detection": {
        "bounding_box": {
          "normalized_vertices": [
            {
              "y": 0.17702378
            },
            {
              "x": 1,
              "y": 0.88745028
            }
          ],
          "vertices": []
        },
        "score": 0.8579272
      }
    },
    ...
    {
      "annotation_spec_id": "2674584024797675520",
      "display_name": "Baked goods",
      "image_object_detection": {
        "bounding_box": {
          "normalized_vertices": [
            {
              "x": 0.69695503,
              "y": 0.17798239
            },
            {
              "x": 0.82379168,
              "y": 0.37175834
            }
          ],
          "vertices": []
        },
        "score": 0.0031145806
      }
    }
  ]
}

Line 2 (salad5.jpg annotations JSON)

{
  "ID": "gs://my-storage-bucket/salad5.jpg",
  "annotations": [
    {
      "annotation_spec_id": "3827505529404522496",
      "display_name": "Salad",
      "image_object_detection": {
        "bounding_box": {
          "normalized_vertices": [
            {
              "x": 0.0044940538,
              "y": 0.070641488
            },
            {
              "x": 0.99745369,
              "y": 0.87991059
            }
          ],
          "vertices": []
        },
        "score": 0.91780394
      }
    },
    {
      "annotation_spec_id": "6133348538618216448",
      "display_name": "Tomato",
      "image_object_detection": {
        "bounding_box": {
          "normalized_vertices": [
            {
              "x": 0.71612686,
              "y": 0.42916849
            },
            {
              "x": 0.90833008,
              "y": 0.55670345
            }
          ],
          "vertices": []
        },
        "score": 0.31472182
      }
    },
    ...
    {
      "annotation_spec_id": "7286270043225063424",
      "display_name": "Seafood",
      "image_object_detection": {
        "bounding_box": {
          "normalized_vertices": [
            {
              "x": 0.72394878,
              "y": 0.42934245
            },
            {
              "x": 0.91764188,
              "y": 0.56444579
            }
          ],
          "vertices": []
        },
        "score": 0.00031806546
      }
    }
  ]
}

Working with long-running operations

REST

Before using any of the request data, make the following replacements:

• project-id: your GCP project ID.
• operation-id: the ID of your operation. The ID is the last element of the name of your operation. For example:
  • operation name: projects/project-id/locations/location-id/operations/IOD5281059901324392598
  • operation id: IOD5281059901324392598

HTTP method and URL:

GET https://automl.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/operations/OPERATION_ID

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by executing gcloud init or gcloud auth login, or by using Cloud Shell, which automatically logs you into the gcloud CLI. You can check the currently active account by executing gcloud auth list.

Execute the following command:

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-id" \
    "https://automl.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/operations/OPERATION_ID"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by executing gcloud init or gcloud auth login, or by using Cloud Shell, which automatically logs you into the gcloud CLI. You can check the currently active account by executing gcloud auth list.

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-id" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://automl.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/operations/OPERATION_ID" | Select-Object -Expand Content

You should see output similar to the following for a completed import operation:

{
  "name": "projects/PROJECT_ID/locations/us-central1/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.automl.v1.OperationMetadata",
    "createTime": "2018-10-29T15:56:29.176485Z",
    "updateTime": "2018-10-29T16:10:41.326614Z",
    "importDataDetails": {}
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.protobuf.Empty"
  }
}

You should see output similar to the following for a completed create model operation:

{
  "name": "projects/PROJECT_ID/locations/us-central1/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.automl.v1.OperationMetadata",
    "createTime": "2019-07-22T18:35:06.881193Z",
    "updateTime": "2019-07-22T19:58:44.972235Z",
    "createModelDetails": {}
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.automl.v1.Model",
    "name": "projects/PROJECT_ID/locations/us-central1/models/MODEL_ID"
  }
}

Go

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

View on GitHub Feedback

import (
	"context"
	"fmt"
	"io"

	automl "cloud.google.com/go/automl/apiv1"
	"cloud.google.com/go/automl/apiv1/automlpb"
)

// getOperationStatus gets an operation's status.
func getOperationStatus(w io.Writer, projectID string, location string, datasetID string, modelName string) error {
	// projectID := "my-project-id"
	// location := "us-central1"
	// datasetID := "ICN123456789..."
	// modelName := "model_display_name"

	ctx := context.Background()
	client, err := automl.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("NewClient: %v", err)
	}
	defer client.Close()

	req := &automlpb.CreateModelRequest{
		Parent: fmt.Sprintf("projects/%s/locations/%s", projectID, location),
		Model: &automlpb.Model{
			DisplayName: modelName,
			DatasetId:   datasetID,
			ModelMetadata: &automlpb.Model_ImageClassificationModelMetadata{
				ImageClassificationModelMetadata: &automlpb.ImageClassificationModelMetadata{
					TrainBudgetMilliNodeHours: 1000, // 1000 milli-node hours are 1 hour
				},
			},
		},
	}

	op, err := client.CreateModel(ctx, req)
	if err != nil {
		return err
	}
	fmt.Fprintf(w, "Name: %v\n", op.Name())

	// Wait for the longrunning operation complete.
	resp, err := op.Wait(ctx)
	if err != nil && !op.Done() {
		fmt.Println("failed to fetch operation status", err)
		return err
	}
	if err != nil && op.Done() {
		fmt.Println("operation completed with error", err)
		return err
	}
	fmt.Fprintf(w, "Response: %v\n", resp)

	return nil
}

Java

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

View on GitHub Feedback

import com.google.cloud.automl.v1.AutoMlClient;
import com.google.longrunning.Operation;
import java.io.IOException;

class GetOperationStatus {

  static void getOperationStatus() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String operationFullId = "projects/[projectId]/locations/us-central1/operations/[operationId]";
    getOperationStatus(operationFullId);
  }

  // Get the status of an operation
  static void getOperationStatus(String operationFullId) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
      // Get the latest state of a long-running operation.
      Operation operation = client.getOperationsClient().getOperation(operationFullId);

      // Display operation details.
      System.out.println("Operation details:");
      System.out.format("\tName: %s\n", operation.getName());
      System.out.format("\tMetadata Type Url: %s\n", operation.getMetadata().getTypeUrl());
      System.out.format("\tDone: %s\n", operation.getDone());
      if (operation.hasResponse()) {
        System.out.format("\tResponse Type Url: %s\n", operation.getResponse().getTypeUrl());
      }
      if (operation.hasError()) {
        System.out.println("\tResponse:");
        System.out.format("\t\tError code: %s\n", operation.getError().getCode());
        System.out.format("\t\tError message: %s\n", operation.getError().getMessage());
      }
    }
  }
}

Node.js

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const operationId = 'YOUR_OPERATION_ID';

// Imports the Google Cloud AutoML library
const {AutoMlClient} = require('@google-cloud/automl').v1;

// Instantiates a client
const client = new AutoMlClient();

async function getOperationStatus() {
  // Construct request
  const request = {
    name: `projects/${projectId}/locations/${location}/operations/${operationId}`,
  };

  const [response] = await client.operationsClient.getOperation(request);

  console.log(`Name: ${response.name}`);
  console.log('Operation details:');
  console.log(`${response}`);
}

getOperationStatus();

Python

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

View on GitHub Feedback

from google.cloud import automl

# TODO(developer): Uncomment and set the following variables
# operation_full_id = \
#     "projects/[projectId]/locations/us-central1/operations/[operationId]"

client = automl.AutoMlClient()
# Get the latest state of a long-running operation.
response = client._transport.operations_client.get_operation(operation_full_id)

print("Name: {}".format(response.name))
print("Operation details:")
print(response)

Additional languages

C#: Please follow the C# setup instructions on the client libraries page and then visit the AutoML Vision Object Detection reference documentation for .NET.

PHP: Please follow the PHP setup instructions on the client libraries page and then visit the AutoML Vision Object Detection reference documentation for PHP.

Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the AutoML Vision Object Detection reference documentation for Ruby.