Get batch predictions

Requesting a batch prediction is an asynchronous request (as opposed to online prediction, which is a synchronous request). You request batch predictions directly from the model resource; you don't need to deploy the model to an endpoint. For data types that support both batch and online predictions, use batch predictions when you don't require an immediate response and want to process accumulated data by using a single request.

To make a batch prediction, you specify an input source and an output location where Vertex AI stores predictions results. The inputs and outputs depend on the model type that you're working with. For example, batch predictions for the AutoML image model type require an input JSONL file and the name of a Cloud Storage bucket to store the output.

Note: To minimize processing time when you use the Cloud Console to create batch predictions, we recommend that you select input and output locations that are in the same region as your model. If you use the API to create batch predictions, send requests to a service endpoint (such as https://us-central1-aiplatform.googleapis.com) that is in the same region or geographically close to your input and output locations.

Batch request input

The input for batch requests specifies the items to send to your model for prediction. The following content provides input information for each model type.

Custom-trained

To get batch predictions from a custom-trained model, prepare your input data in one of the following ways:

JSONL

Use a JSONL file to specify a list of input instances to make predictions about. Store the JSONL file in a Cloud Storage bucket. The following example shows two instances in an input JSONL file:

{"dense_input": [1, 2, 3]}
{"dense_input": [4, 5, 6]}

TFRecord

Save input instances in the TFRecord format. You can optionally compress the TFRecord files with Gzip. Store the TFRecord files in a Cloud Storage bucket.

Vertex AI reads each instance in your TFRecord files as binary, then base64-encodes it and sets it in a JSON instance to the container that serves your model's predictions.

CSV

Specify one input instance per row in a CSV file. You must include a header row. Vertex AI uses your header row to convert each input instance into JSON before sending it to the container that serves your model's predictions.

The following example shows a CSV file with two input instances:

input1,input2,input3
0.1,1.2,3.0
4.0,5.0,6.0

File list

Create a text file where each row is the Cloud Storage URI to a file. Vertex AI reads each URI as binary, then base64-encodes it and sends it in a JSON instance to the container that serves your model's predictions.

If you plan to use the Google Cloud Console to get batch predictions, paste your file list directly into the Cloud Console. Otherwise save your file list in a Cloud Storage bucket.

The following example shows a file list with two input instances:

gs://path/to/image/image1.jpg
gs://path/to/image/image2.jpg

Image

Use a JSONL file to specify a list of images to make predictions about and then store the JSONL file in a Cloud Storage bucket. The following sample shows a single line in an input JSONL file.

{"content": "gs://sourcebucket/datasets/images/source_image.jpg", "mimeType": "image/jpeg"}

Tabular

For tabular data, you can use a CSV file in a Cloud Storage bucket or a table in BigQuery. You can provide prediction data as a CSV in Cloud Storage or as a table in BigQuery. We recommended that you use the same input format for your training and prediction data. Since Vertex AI treats all CSV input fields as strings, using a different input format in training and prediction may cause an error. For example, if you trained your model using data in BigQuery, it is best to use a BigQuery table as the source for your batch predictions.

Your data source must contain tabular data that includes all of the columns, in any order, that was used to train the model. You can include columns that were not in the training data, or that were in the training data but excluded from use for training. These extra columns are included in the prediction output, but they are not used for generating the prediction.

You do not need to use the same destination for your results as your prediction data. For example, if you used BigQuery for the prediction data source, you can send results to a CSV file in Cloud Storage.

BigQuery table requirements

• BigQuery data source tables must be no larger than 100 GB.
• You must use a multi-regional BigQuery dataset in the US or EU locations.
• If the table is in a different project, you must provide the BigQuery Data Editor role to the Vertex AI service account in that project.

CSV file requirements

• The first line of the data source must contain the name of the columns.
• Each data source file must not be larger than 10 GB. You can include multiple files, up to a maximum amount of 100 GB.
• If the Cloud Storage bucket is in a different project than where you use Vertex AI, you must provide the Storage Object Creator role to the Vertex AI service account in that project.

Forecasting requirements

Preview

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

In addition to the BigQuery table or CSV file requirements, input for forecasting models must also adhere to the following requirements:

• All values in the time column must be present and valid.
• The data frequency in the batch prediction table must be the same frequency as the training data. You cannot have missing rows in the time series. Manually insert missing rows according to the proper domain knowledge.
• Time series with duplicate timestamps are removed from predictions. To include them, remove any duplicate timestamps.
• Provide historical data for each time series to predict. For the most accurate predictions, the amount of data should equal the context window, which was set in training. If you provide less data, Vertex AI pads data with empty values. For example, if the context window is 14 days, provide at least 14 days of historical data.
• The forecast starts on the first row of a time series (ordered by time) with a null value in the target column. The null value must be continuous within the time series. For example, if the target column is ordered by time, you cannot have something like `1`, `2`, `null`, `3`, `4`, `null`, `null` for a single time series. For CSV files, Vertex AI treats an empty string as null, and for BigQuery, null values are natively supported.

Text

Classification and sentiment analysis

Use a JSONL file to specify a list of documents to make predictions about and then store the JSONL file in a Cloud Storage bucket. The following sample shows a single line in an input JSONL file.

{"content": "gs://sourcebucket/datasets/texts/source_text.txt", "mimeType": "text/plain"}

Entity extraction

For entity extraction, you can include inline text or references to documents that are in a Cloud Storage bucket. For each document, you can also add a key field to your input.

Typically, batch prediction results map inputs and outputs by using the instance field, which includes the content and mimeType fields. If you use the key field in your input, the batch prediction output replaces use the instance field with the key field. This helps simplify your batch prediction output if, for example, your input includes large text snippets.

The following sample shows a JSONL file that includes references to documents and inline text snippets with and without the key field.

{"content": "gs://sourcebucket/datasets/texts/source_text.txt", "mimeType": "text/plain"}
{"content": "gs://bucket/sample.txt", "mimeType": "text/plain", "key": "sample-file"}
{"content": "Text snippet", "mimeType": "text/plain"}
{"content": "Sample text snippet", "mimeType": "text/plain", "key": "sample-snippet"}

Video

Use a JSONL file to specify a list of videos to make predictions for, and then store the JSONL file in a Cloud Storage bucket. You can specify Infinity for the timeSegmentEnd field to specify the end of the video. The following sample shows a single line in an input JSONL file.

{'content': 'gs://sourcebucket/datasets/videos/source_video.mp4', 'mimeType': 'video/mp4', 'timeSegmentStart': '0.0s', 'timeSegmentEnd': '2.366667s'}

Request a batch prediction

For batch prediction requests, you can use the Google Cloud Console or the Vertex AI API. Depending on the number of items that you've submitted, a batch prediction task can take some time to complete.

Cloud Console

Use the Cloud Console to request a batch prediction.

1. In the Cloud Console, in the Vertex AI section, go to the Batch predictions page.

   Go to the Batch predictions page

2. Click Create to open the New batch prediction window and complete the following steps:

   Custom-trained

   1. Enter a name for the batch prediction.
   2. For Model name, select the name of the model to use for this batch prediction.
   3. For Select source:
      • If you have formatted your input as JSONL, CSV, or TFRecord, select File on Cloud Storage (JSONL, CSV, TFRecord, TFRecord Gzip). Then specify your input file in the Source path field.
      • If you are using a file list as input, select Files on Cloud Storage (other) and paste your file list into the following text box.
   4. In the Destination path field, specify the Cloud Storage directory where you want Vertex AI to store batch prediction output.
   5. Optionally, you may check Enable feature attributions for this model, in order to get feature attributions as part of the batch prediction response. Then click Edit to configure explanation settings. (Editing the explanation settings is optional if you previously configured explanation settings for the model, and required otherwise.)
   6. Specify compute options for the batch prediction job: Number of compute nodes, Machine type, and (optionally) Accelerator type and Accelerator count

   Image, text, or video

   1. Enter a name for the batch prediction.
   2. For Model name, select the name of the model to use for this batch prediction.
   3. For Source path, specify the Cloud Storage location where your JSONL input file is located.
   4. For the Destination path, specify a Cloud Storage location where the batch prediction results are stored. The Output format is determined by your model's objective. For example, AutoML models for text objectives output JSONL files.

   Tabular

   1. Enter a name for the batch prediction.
   2. For Model name, select the name of the model to use for this batch prediction.
   3. For Select source, select whether your source input data is a CSV file on Cloud Storage or a table in BigQuery.
      • For CSV files, specify the Cloud Storage location where your CSV input file is located.
      • For BigQuery tables, specify the project ID where the table is located, the BigQuery dataset ID, and the BigQuery table or view ID.
   4. For the Output, select CSV or BigQuery.
      • For CSV, specify the Cloud Storage bucket where Vertex AI stores your output.
      • For BigQuery, you can specify a project ID or an existing dataset:
        • To specify the project ID, enter the project ID in the Google Cloud project ID field. Vertex AI creates a new output dataset for you.
        • To specify an existing dataset, enter its BigQuery path in the Google Cloud project ID field, such as bq://projectid.datasetid.

      • If you specified BigQuery as your output destination, and you want feature importance values returned for your predictions, check the Generate feature importance checkbox.

        Feature importance is not supported when the prediction data is returned in Cloud Storage or for forecasting models.

API

Use the Vertex AI API to send batch prediction requests.

Custom-trained

Select a tab depending on which tool you are using to get batch predictions:

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.

• PROJECT: Your project ID or project number.

• BATCH_JOB_NAME: Display name for the batch prediction job.

• MODEL_ID: The ID for the model to use for making predictions.

• INPUT_FORMAT: The format of your input data: jsonl, csv, tf-record, tf-record-gzip, or file-list.

• INPUT_URI: Cloud Storage URI of your input data. May contain wildcards.

• OUTPUT_DIRECTORY: Cloud Storage URI of a directory where you want Vertex AI to save output.

• MACHINE_TYPE: The machine resources to be used for this batch prediction job.

  You can optionally configure the machineSpec field to use accelerators, but the following example does not demonstrate this.

• BATCH_SIZE: The number of instances to send in each prediction request; the default is 64. Increasing the batch size can lead to higher throughput, but it can also cause request timeouts.

• STARTING_REPLICA_COUNT: The number of nodes for this batch prediction job.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
  "displayName": "BATCH_JOB_NAME",
  "model": "projects/PROJECT/locations/LOCATION/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "INPUT_FORMAT",
    "gcsSource": {
      "uris": ["INPUT_URI"],
    },
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_DIRECTORY",
    },
  },
  "dedicatedResources" : {
    "machineSpec" : {
      "machineType": MACHINE_TYPE
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT
  },
  "manualBatchTuningParameters": {
    "batch_size": BATCH_SIZE,
  }
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT/locations/us-central1/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "INPUT_URI"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_DIRECTORY"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
}

Java

In the following sample, replace PREDICTIONS_FORMAT with jsonl. To learn how to replace the other placeholders, see the REST & CMD LINE tab of this section.

View on GitHub Feedback

import com.google.cloud.aiplatform.util.ValueConverter;
import com.google.cloud.aiplatform.v1.AcceleratorType;
import com.google.cloud.aiplatform.v1.BatchDedicatedResources;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.MachineSpec;
import com.google.cloud.aiplatform.v1.ModelName;
import com.google.protobuf.Value;
import java.io.IOException;

public class CreateBatchPredictionJobSample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String displayName = "DISPLAY_NAME";
    String modelName = "MODEL_NAME";
    String instancesFormat = "INSTANCES_FORMAT";
    String gcsSourceUri = "GCS_SOURCE_URI";
    String predictionsFormat = "PREDICTIONS_FORMAT";
    String gcsDestinationOutputUriPrefix = "GCS_DESTINATION_OUTPUT_URI_PREFIX";
    createBatchPredictionJobSample(
        project,
        displayName,
        modelName,
        instancesFormat,
        gcsSourceUri,
        predictionsFormat,
        gcsDestinationOutputUriPrefix);
  }

  static void createBatchPredictionJobSample(
      String project,
      String displayName,
      String model,
      String instancesFormat,
      String gcsSourceUri,
      String predictionsFormat,
      String gcsDestinationOutputUriPrefix)
      throws IOException {
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();
    String location = "us-central1";

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {

      // Passing in an empty Value object for model parameters
      Value modelParameters = ValueConverter.EMPTY_VALUE;

      GcsSource gcsSource = GcsSource.newBuilder().addUris(gcsSourceUri).build();
      BatchPredictionJob.InputConfig inputConfig =
          BatchPredictionJob.InputConfig.newBuilder()
              .setInstancesFormat(instancesFormat)
              .setGcsSource(gcsSource)
              .build();
      GcsDestination gcsDestination =
          GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
      BatchPredictionJob.OutputConfig outputConfig =
          BatchPredictionJob.OutputConfig.newBuilder()
              .setPredictionsFormat(predictionsFormat)
              .setGcsDestination(gcsDestination)
              .build();
      MachineSpec machineSpec =
          MachineSpec.newBuilder()
              .setMachineType("n1-standard-2")
              .setAcceleratorType(AcceleratorType.NVIDIA_TESLA_K80)
              .setAcceleratorCount(1)
              .build();
      BatchDedicatedResources dedicatedResources =
          BatchDedicatedResources.newBuilder()
              .setMachineSpec(machineSpec)
              .setStartingReplicaCount(1)
              .setMaxReplicaCount(1)
              .build();
      String modelName = ModelName.of(project, location, model).toString();
      BatchPredictionJob batchPredictionJob =
          BatchPredictionJob.newBuilder()
              .setDisplayName(displayName)
              .setModel(modelName)
              .setModelParameters(modelParameters)
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .setDedicatedResources(dedicatedResources)
              .build();
      LocationName parent = LocationName.of(project, location);
      BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
      System.out.format("response: %s\n", response);
      System.out.format("\tName: %s\n", response.getName());
    }
  }
}

Python

View on GitHub Feedback

def create_batch_prediction_job_dedicated_resources_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    machine_type: str = "n1-standard-2",
    accelerator_count: int = 1,
    accelerator_type: Union[str, aiplatform_v1.AcceleratorType] = "NVIDIA_TESLA_K80",
    starting_replica_count: int = 1,
    max_replica_count: int = 1,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        machine_type=machine_type,
        accelerator_count=accelerator_count,
        accelerator_type=accelerator_type,
        starting_replica_count=starting_replica_count,
        max_replica_count=max_replica_count,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Feature importance

If you want feature importance values returned for your predictions, set the generateExplanation property to true. Note that forecasting models don't support feature importance, so you cannot include it in your batch prediction requests.

Feature importance, sometimes called feature attributions, is part of Vertex Explainable AI.

You can only set generateExplanation to true if you have configured your Model for explanations or if you specify the BatchPredictionJob's explanationSpec field.

Image

The following batch prediction example works for classification and object detection objectives.

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• THRESHOLD_VALUE (optional): Vertex AI returns only predictions that have confidence scores with at least this value. The default is 0.0.
• MAX_PREDICTIONS (optional): Vertex AI returns up to this many predictions starting with the predictions that have highest confidence scores. The default is 10.
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/LOCATION/models/MODEL_ID",
    "modelParameters": {
      "confidenceThreshold": THRESHOLD_VALUE,
      "maxPredictions": MAX_PREDICTIONS
    },
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT/locations/us-central1/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Python

Open in Editor View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Tabular

The code needed to request batch predictions depends on whether the prediction data is CSV files in Cloud Storage or a BigQuery table. You can use a different format (Cloud Storage or BigQuery) for your prediction data and output destination. If the prediction data resides in a different project than where Vertex AI is running, make sure you set up the required permissions.

Requesting a batch prediction using CSV files

REST & CMD LINE

You use the batchPredictionJobs.create method to request a batch prediction.

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• URI: Paths (URIs) to the Cloud Storage buckets containing the training data. There can be more than one. Each URI has the form:

  gs://bucketName/pathToFileName
  

• OUTPUT_URI_PREFIX: Path to a Cloud Storage destination where the predictions will be written. Vertex AI writes batch predictions to a timestamped subdirectory of this path. Set this value to a string with the following format:

  gs://bucketName/pathToOutputDirectory
  

• MACHINE_TYPE: The machine resources to be used for this batch prediction job. Learn more.
• STARTING_REPLICA_COUNT: The starting number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, up to the maximum number of nodes, but will never fall below this number.
• MAX_REPLICA_COUNT: The maximum number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, but will never exceed the maximum. Optional, defaults to 10.
• BATCH_SIZE: The number of rows of prediction data used in each batch. Higher values can increase prediction performance, but if the size of the batch exceeds the size of the memory for a node, the prediction job fails. The default value is 4.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
  "displayName": "BATCH_JOB_NAME,
  "model": "MODEL_ID",
  "inputConfig": {
    "instancesFormat": "csv",
    "gcsSource": {
      "uris": [
        URI1,...
      ]
    },
  },
  "outputConfig": {
    "predictionsFormat": "csv",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_URI_PREFIX"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "MACHINE_TYPE",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT,
    "maxReplicaCount": MAX_REPLICA_COUNT
  },
  "manualBatchTuningParameters": {
    "batchSize": BATCH_SIZE
  }
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/12345/locations/us-central1/batchPredictionJobs/67890",
  "displayName": "batch_job_1 202005291958",
  "model": "projects/12345/locations/us-central1/models/5678",
  "state": "JOB_STATE_PENDING",
  "inputConfig": {
    "instancesFormat": "csv",
    "gcsSource": {
      "uris": [
        "gs://bp_bucket/reg_mode_test"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "csv",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_URI_PREFIX"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "n1-standard-32",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": 2,
    "maxReplicaCount": 6
  },
  "manualBatchTuningParameters": {
    "batchSize": 4
  }
  "outputInfo": {
    "gcsOutputDataset": "OUTPUT_URI_PREFIX/prediction-batch_job_1 202005291958-2020-09-30T02:58:44.341643Z"
  }
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-09-30T02:58:44.341643Z",
  "updateTime": "2020-09-30T02:58:44.341643Z",
}

Feature importance is not supported when the prediction data is returned in Cloud Storage.

Requesting a batch prediction using BigQuery

REST & CMD LINE

You use the batchPredictionJobs.create method to request a batch prediction.

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• INPUT_URI: Reference to the BigQuery data source. In the form:

  bq://bqprojectId.bqDatasetId.bqTableId
  

• OUTPUT_URI: Reference to the BigQuery destination (where the predictions will be written). Specify the project ID and, optionally, an existing dataset ID. If you specify just the project ID, Vertex AI creates a new output dataset for you. Use the following form:

  bq://bqprojectId.bqDatasetId
  

• MACHINE_TYPE: The machine resources to be used for this batch prediction job. Learn more.
• STARTING_REPLICA_COUNT: The starting number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, up to the maximum number of nodes, but will never fall below this number.
• MAX_REPLICA_COUNT: The maximum number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, but will never exceed the maximum. Optional, defaults to 10.
• BATCH_SIZE: The number of rows of prediction data used in each batch. Higher values can increase prediction performance, but if the size of the batch exceeds the size of the memory for a node, the prediction job fails. The default value is 4.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
  "displayName": "BATCH_JOB_NAME,
  "model": "MODEL_ID",
  "inputConfig": {
    "instancesFormat": "bigquery",
    "bigquerySource": {
      "inputUri": "INPUT_URI"
    }
  },
  "outputConfig": {
    "predictionsFormat": "bigquery",
    "bigqueryDestination": {
      "outputUri": "OUTPUT_URI"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "MACHINE_TYPE",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT,
    "maxReplicaCount": MAX_REPLICA_COUNT
  },
  "manualBatchTuningParameters": {
    "batchSize": BATCH_SIZE
  }
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/12345/locations/us-central1/batchPredictionJobs/67890",
  "displayName": "batch_job_1 202005291958",
  "model": "projects/12345/locations/us-central1/models/5678",
  "state": "JOB_STATE_PENDING",
  "inputConfig": {
    "instancesFormat": "bigquery",
    "bigquerySource": {
      "inputUri": "INPUT_URI"
    }
  },
  "outputConfig": {
    "predictionsFormat": "bigquery",
    "bigqueryDestination": {
        "outputUri": bq://12345
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "n1-standard-32",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": 2,
    "maxReplicaCount": 6
  },
  "manualBatchTuningParameters": {
    "batchSize": 4
  },
  "generateExplanation": false,
  "outputInfo": {
    "bigqueryOutputDataset": "bq://12345.reg_model_2020_10_02_06_04
  }
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-09-30T02:58:44.341643Z",
  "updateTime": "2020-09-30T02:58:44.341643Z",
}

Java

In the following sample, replace INSTANCES_FORMAT and PREDICTIONS_FORMAT with bigquery. To learn how to replace the other placeholders, see the REST & CMD LINE tab of this section.

View on GitHub Feedback

import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.BigQueryDestination;
import com.google.cloud.aiplatform.v1.BigQuerySource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.ModelName;
import com.google.gson.JsonObject;
import com.google.protobuf.Value;
import com.google.protobuf.util.JsonFormat;
import java.io.IOException;

public class CreateBatchPredictionJobBigquerySample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String displayName = "DISPLAY_NAME";
    String modelName = "MODEL_NAME";
    String instancesFormat = "INSTANCES_FORMAT";
    String bigquerySourceInputUri = "BIGQUERY_SOURCE_INPUT_URI";
    String predictionsFormat = "PREDICTIONS_FORMAT";
    String bigqueryDestinationOutputUri = "BIGQUERY_DESTINATION_OUTPUT_URI";
    createBatchPredictionJobBigquerySample(
        project,
        displayName,
        modelName,
        instancesFormat,
        bigquerySourceInputUri,
        predictionsFormat,
        bigqueryDestinationOutputUri);
  }

  static void createBatchPredictionJobBigquerySample(
      String project,
      String displayName,
      String model,
      String instancesFormat,
      String bigquerySourceInputUri,
      String predictionsFormat,
      String bigqueryDestinationOutputUri)
      throws IOException {
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();
    String location = "us-central1";

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {
      JsonObject jsonModelParameters = new JsonObject();
      Value.Builder modelParametersBuilder = Value.newBuilder();
      JsonFormat.parser().merge(jsonModelParameters.toString(), modelParametersBuilder);
      Value modelParameters = modelParametersBuilder.build();
      BigQuerySource bigquerySource =
          BigQuerySource.newBuilder().setInputUri(bigquerySourceInputUri).build();
      BatchPredictionJob.InputConfig inputConfig =
          BatchPredictionJob.InputConfig.newBuilder()
              .setInstancesFormat(instancesFormat)
              .setBigquerySource(bigquerySource)
              .build();
      BigQueryDestination bigqueryDestination =
          BigQueryDestination.newBuilder().setOutputUri(bigqueryDestinationOutputUri).build();
      BatchPredictionJob.OutputConfig outputConfig =
          BatchPredictionJob.OutputConfig.newBuilder()
              .setPredictionsFormat(predictionsFormat)
              .setBigqueryDestination(bigqueryDestination)
              .build();
      String modelName = ModelName.of(project, location, model).toString();
      BatchPredictionJob batchPredictionJob =
          BatchPredictionJob.newBuilder()
              .setDisplayName(displayName)
              .setModel(modelName)
              .setModelParameters(modelParameters)
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .build();
      LocationName parent = LocationName.of(project, location);
      BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
      System.out.format("response: %s\n", response);
      System.out.format("\tName: %s\n", response.getName());
    }
  }
}

Python

In the following sample, set the instances_format and predictions_format parameters to "bigquery". To learn how to set the other parameters, see the REST & CMD LINE tab of this section.

View on GitHub Feedback

from google.cloud import aiplatform_v1beta1
from google.protobuf import json_format
from google.protobuf.struct_pb2 import Value


def create_batch_prediction_job_bigquery_sample(
    project: str,
    display_name: str,
    model_name: str,
    instances_format: str,
    bigquery_source_input_uri: str,
    predictions_format: str,
    bigquery_destination_output_uri: str,
    location: str = "us-central1",
    api_endpoint: str = "us-central1-aiplatform.googleapis.com",
):
    # The AI Platform services require regional API endpoints.
    client_options = {"api_endpoint": api_endpoint}
    # Initialize client that will be used to create and send requests.
    # This client only needs to be created once, and can be reused for multiple requests.
    client = aiplatform_v1beta1.JobServiceClient(client_options=client_options)
    model_parameters_dict = {}
    model_parameters = json_format.ParseDict(model_parameters_dict, Value())

    batch_prediction_job = {
        "display_name": display_name,
        # Format: 'projects/{project}/locations/{location}/models/{model_id}'
        "model": model_name,
        "model_parameters": model_parameters,
        "input_config": {
            "instances_format": instances_format,
            "bigquery_source": {"input_uri": bigquery_source_input_uri},
        },
        "output_config": {
            "predictions_format": predictions_format,
            "bigquery_destination": {"output_uri": bigquery_destination_output_uri},
        },
        # optional
        "generate_explanation": True,
    }
    parent = f"projects/{project}/locations/{location}"
    response = client.create_batch_prediction_job(
        parent=parent, batch_prediction_job=batch_prediction_job
    )
    print("response:", response)

Feature importance

If you want feature importance values returned for your predictions, set the generateExplanation property to true. Note that forecasting models don't support feature importance, so you cannot include it in your batch prediction requests.

Feature importance, sometimes called feature attributions, is part of Vertex Explainable AI.

Feature importance is not supported when the prediction data is returned in Cloud Storage or for forecasting models.

Text

Select a text objective to view an example of a batch prediction request.

Classification

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/LOCATION/MODEL_ID",
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Java

View on GitHub Feedback

import com.google.api.gax.rpc.ApiException;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.ModelName;
import java.io.IOException;

public class CreateBatchPredictionJobTextClassificationSample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String location = "us-central1";
    String displayName = "DISPLAY_NAME";
    String modelId = "MODEL_ID";
    String gcsSourceUri = "GCS_SOURCE_URI";
    String gcsDestinationOutputUriPrefix = "GCS_DESTINATION_OUTPUT_URI_PREFIX";
    createBatchPredictionJobTextClassificationSample(
        project, location, displayName, modelId, gcsSourceUri, gcsDestinationOutputUriPrefix);
  }

  static void createBatchPredictionJobTextClassificationSample(
      String project,
      String location,
      String displayName,
      String modelId,
      String gcsSourceUri,
      String gcsDestinationOutputUriPrefix)
      throws IOException {
    // The AI Platform services require regional API endpoints.
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {
      try {
        String modelName = ModelName.of(project, location, modelId).toString();
        GcsSource gcsSource = GcsSource.newBuilder().addUris(gcsSourceUri).build();
        BatchPredictionJob.InputConfig inputConfig =
            BatchPredictionJob.InputConfig.newBuilder()
                .setInstancesFormat("jsonl")
                .setGcsSource(gcsSource)
                .build();
        GcsDestination gcsDestination =
            GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
        BatchPredictionJob.OutputConfig outputConfig =
            BatchPredictionJob.OutputConfig.newBuilder()
                .setPredictionsFormat("jsonl")
                .setGcsDestination(gcsDestination)
                .build();
        BatchPredictionJob batchPredictionJob =
            BatchPredictionJob.newBuilder()
                .setDisplayName(displayName)
                .setModel(modelName)
                .setInputConfig(inputConfig)
                .setOutputConfig(outputConfig)
                .build();
        LocationName parent = LocationName.of(project, location);
        BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
        System.out.format("response: %s\n", response);
      } catch (ApiException ex) {
        System.out.format("Exception: %s\n", ex.getLocalizedMessage());
      }
    }
  }
}

Node.js

View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.\
 * (Not necessary if passing values as arguments)
 */

// const batchPredictionDisplayName = 'YOUR_BATCH_PREDICTION_DISPLAY_NAME';
// const modelId = 'YOUR_MODEL_ID';
// const gcsSourceUri = 'YOUR_GCS_SOURCE_URI';
// const gcsDestinationOutputUriPrefix = 'YOUR_GCS_DEST_OUTPUT_URI_PREFIX';
//    eg. "gs://<your-gcs-bucket>/destination_path"
// const project = 'YOUR_PROJECT_ID';
// const location = 'YOUR_PROJECT_LOCATION';

// Imports the Google Cloud Job Service Client library
const {JobServiceClient} = require('@google-cloud/aiplatform').v1;

// Specifies the location of the api endpoint
const clientOptions = {
  apiEndpoint: 'us-central1-aiplatform.googleapis.com',
};

// Instantiates a client
const jobServiceClient = new JobServiceClient(clientOptions);

async function createBatchPredictionJobTextClassification() {
  // Configure the parent resource
  const parent = `projects/${project}/locations/${location}`;
  const modelName = `projects/${project}/locations/${location}/models/${modelId}`;

  const inputConfig = {
    instancesFormat: 'jsonl',
    gcsSource: {uris: [gcsSourceUri]},
  };
  const outputConfig = {
    predictionsFormat: 'jsonl',
    gcsDestination: {outputUriPrefix: gcsDestinationOutputUriPrefix},
  };
  const batchPredictionJob = {
    displayName: batchPredictionDisplayName,
    model: modelName,
    inputConfig,
    outputConfig,
  };
  const request = {
    parent,
    batchPredictionJob,
  };

  // Create batch prediction job request
  const [response] = await jobServiceClient.createBatchPredictionJob(request);

  console.log('Create batch prediction job text classification response');
  console.log(`Name : ${response.name}`);
  console.log('Raw response:');
  console.log(JSON.stringify(response, null, 2));
}
createBatchPredictionJobTextClassification();

Python

View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Entity extraction

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/LOCATION/MODEL_ID",
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Java

View on GitHub Feedback

import com.google.api.gax.rpc.ApiException;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.ModelName;
import java.io.IOException;

public class CreateBatchPredictionJobTextEntityExtractionSample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String location = "us-central1";
    String displayName = "DISPLAY_NAME";
    String modelId = "MODEL_ID";
    String gcsSourceUri = "GCS_SOURCE_URI";
    String gcsDestinationOutputUriPrefix = "GCS_DESTINATION_OUTPUT_URI_PREFIX";
    createBatchPredictionJobTextEntityExtractionSample(
        project, location, displayName, modelId, gcsSourceUri, gcsDestinationOutputUriPrefix);
  }

  static void createBatchPredictionJobTextEntityExtractionSample(
      String project,
      String location,
      String displayName,
      String modelId,
      String gcsSourceUri,
      String gcsDestinationOutputUriPrefix)
      throws IOException {
    // The AI Platform services require regional API endpoints.
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {
      try {
        String modelName = ModelName.of(project, location, modelId).toString();
        GcsSource gcsSource = GcsSource.newBuilder().addUris(gcsSourceUri).build();
        BatchPredictionJob.InputConfig inputConfig =
            BatchPredictionJob.InputConfig.newBuilder()
                .setInstancesFormat("jsonl")
                .setGcsSource(gcsSource)
                .build();
        GcsDestination gcsDestination =
            GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
        BatchPredictionJob.OutputConfig outputConfig =
            BatchPredictionJob.OutputConfig.newBuilder()
                .setPredictionsFormat("jsonl")
                .setGcsDestination(gcsDestination)
                .build();
        BatchPredictionJob batchPredictionJob =
            BatchPredictionJob.newBuilder()
                .setDisplayName(displayName)
                .setModel(modelName)
                .setInputConfig(inputConfig)
                .setOutputConfig(outputConfig)
                .build();
        LocationName parent = LocationName.of(project, location);
        BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
        System.out.format("response: %s\n", response);
        System.out.format("\tname:%s\n", response.getName());
      } catch (ApiException ex) {
        System.out.format("Exception: %s\n", ex.getLocalizedMessage());
      }
    }
  }
}

Node.js

View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.\
 * (Not necessary if passing values as arguments)
 */

// const batchPredictionDisplayName = 'YOUR_BATCH_PREDICTION_DISPLAY_NAME';
// const modelId = 'YOUR_MODEL_ID';
// const gcsSourceUri = 'YOUR_GCS_SOURCE_URI';
// const gcsDestinationOutputUriPrefix = 'YOUR_GCS_DEST_OUTPUT_URI_PREFIX';
//    eg. "gs://<your-gcs-bucket>/destination_path"
// const project = 'YOUR_PROJECT_ID';
// const location = 'YOUR_PROJECT_LOCATION';

// Imports the Google Cloud Job Service Client library
const {JobServiceClient} = require('@google-cloud/aiplatform').v1;

// Specifies the location of the api endpoint
const clientOptions = {
  apiEndpoint: 'us-central1-aiplatform.googleapis.com',
};

// Instantiates a client
const jobServiceClient = new JobServiceClient(clientOptions);

async function createBatchPredictionJobTextEntityExtraction() {
  // Configure the parent resource
  const parent = `projects/${project}/locations/${location}`;
  const modelName = `projects/${project}/locations/${location}/models/${modelId}`;

  const inputConfig = {
    instancesFormat: 'jsonl',
    gcsSource: {uris: [gcsSourceUri]},
  };
  const outputConfig = {
    predictionsFormat: 'jsonl',
    gcsDestination: {outputUriPrefix: gcsDestinationOutputUriPrefix},
  };
  const batchPredictionJob = {
    displayName: batchPredictionDisplayName,
    model: modelName,
    inputConfig,
    outputConfig,
  };
  const request = {
    parent,
    batchPredictionJob,
  };

  // Create batch prediction job request
  const [response] = await jobServiceClient.createBatchPredictionJob(request);

  console.log('Create batch prediction job text entity extraction response');
  console.log(`Name : ${response.name}`);
  console.log('Raw response:');
  console.log(JSON.stringify(response, null, 2));
}
createBatchPredictionJobTextEntityExtraction();

Python

View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Sentiment analysis

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/LOCATION/MODEL_ID",
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Java

View on GitHub Feedback

import com.google.api.gax.rpc.ApiException;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.ModelName;
import java.io.IOException;

public class CreateBatchPredictionJobTextSentimentAnalysisSample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String location = "us-central1";
    String displayName = "DISPLAY_NAME";
    String modelId = "MODEL_ID";
    String gcsSourceUri = "GCS_SOURCE_URI";
    String gcsDestinationOutputUriPrefix = "GCS_DESTINATION_OUTPUT_URI_PREFIX";
    createBatchPredictionJobTextSentimentAnalysisSample(
        project, location, displayName, modelId, gcsSourceUri, gcsDestinationOutputUriPrefix);
  }

  static void createBatchPredictionJobTextSentimentAnalysisSample(
      String project,
      String location,
      String displayName,
      String modelId,
      String gcsSourceUri,
      String gcsDestinationOutputUriPrefix)
      throws IOException {
    // The AI Platform services require regional API endpoints.
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {
      try {
        String modelName = ModelName.of(project, location, modelId).toString();
        GcsSource gcsSource = GcsSource.newBuilder().addUris(gcsSourceUri).build();
        BatchPredictionJob.InputConfig inputConfig =
            BatchPredictionJob.InputConfig.newBuilder()
                .setInstancesFormat("jsonl")
                .setGcsSource(gcsSource)
                .build();
        GcsDestination gcsDestination =
            GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
        BatchPredictionJob.OutputConfig outputConfig =
            BatchPredictionJob.OutputConfig.newBuilder()
                .setPredictionsFormat("jsonl")
                .setGcsDestination(gcsDestination)
                .build();
        BatchPredictionJob batchPredictionJob =
            BatchPredictionJob.newBuilder()
                .setDisplayName(displayName)
                .setModel(modelName)
                .setInputConfig(inputConfig)
                .setOutputConfig(outputConfig)
                .build();
        LocationName parent = LocationName.of(project, location);
        BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
        System.out.format("response: %s\n", response);
      } catch (ApiException ex) {
        System.out.format("Exception: %s\n", ex.getLocalizedMessage());
      }
    }
  }
}

Node.js

View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.\
 * (Not necessary if passing values as arguments)
 */

// const batchPredictionDisplayName = 'YOUR_BATCH_PREDICTION_DISPLAY_NAME';
// const modelId = 'YOUR_MODEL_ID';
// const gcsSourceUri = 'YOUR_GCS_SOURCE_URI';
// const gcsDestinationOutputUriPrefix = 'YOUR_GCS_DEST_OUTPUT_URI_PREFIX';
//    eg. "gs://<your-gcs-bucket>/destination_path"
// const project = 'YOUR_PROJECT_ID';
// const location = 'YOUR_PROJECT_LOCATION';

// Imports the Google Cloud Job Service Client library
const {JobServiceClient} = require('@google-cloud/aiplatform').v1;

// Specifies the location of the api endpoint
const clientOptions = {
  apiEndpoint: 'us-central1-aiplatform.googleapis.com',
};

// Instantiates a client
const jobServiceClient = new JobServiceClient(clientOptions);

async function createBatchPredictionJobTextSentimentAnalysis() {
  // Configure the parent resource
  const parent = `projects/${project}/locations/${location}`;
  const modelName = `projects/${project}/locations/${location}/models/${modelId}`;

  const inputConfig = {
    instancesFormat: 'jsonl',
    gcsSource: {uris: [gcsSourceUri]},
  };
  const outputConfig = {
    predictionsFormat: 'jsonl',
    gcsDestination: {outputUriPrefix: gcsDestinationOutputUriPrefix},
  };
  const batchPredictionJob = {
    displayName: batchPredictionDisplayName,
    model: modelName,
    inputConfig,
    outputConfig,
  };
  const request = {
    parent,
    batchPredictionJob,
  };

  // Create batch prediction job request
  const [response] = await jobServiceClient.createBatchPredictionJob(request);

  console.log('Create batch prediction job text sentiment analysis response');
  console.log(`Name : ${response.name}`);
  console.log('Raw response:');
  console.log(JSON.stringify(response, null, 2));
}
createBatchPredictionJobTextSentimentAnalysis();

Python

View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Video

Action recognition

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• THRESHOLD_VALUE (optional): Model returns only predictions that have confidence scores with at least this value
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/us-central1/MODEL_ID",
    "modelParameters": {
      "confidenceThreshold": THRESHOLD_VALUE,
    },
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_NUMBER/locations/us-central1/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Java

View on GitHub Feedback

import com.google.cloud.aiplatform.util.ValueConverter;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.ModelName;
import com.google.protobuf.Value;
import java.io.IOException;

public class CreateBatchPredictionJobVideoActionRecognitionSample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String displayName = "DISPLAY_NAME";
    String model = "MODEL";
    String gcsSourceUri = "GCS_SOURCE_URI";
    String gcsDestinationOutputUriPrefix = "GCS_DESTINATION_OUTPUT_URI_PREFIX";
    createBatchPredictionJobVideoActionRecognitionSample(
        project, displayName, model, gcsSourceUri, gcsDestinationOutputUriPrefix);
  }

  static void createBatchPredictionJobVideoActionRecognitionSample(
      String project,
      String displayName,
      String model,
      String gcsSourceUri,
      String gcsDestinationOutputUriPrefix)
      throws IOException {
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();
    String location = "us-central1";

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {
      Value modelParameters = ValueConverter.EMPTY_VALUE;
      GcsSource gcsSource = GcsSource.newBuilder().addUris(gcsSourceUri).build();
      BatchPredictionJob.InputConfig inputConfig =
          BatchPredictionJob.InputConfig.newBuilder()
              .setInstancesFormat("jsonl")
              .setGcsSource(gcsSource)
              .build();
      GcsDestination gcsDestination =
          GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
      BatchPredictionJob.OutputConfig outputConfig =
          BatchPredictionJob.OutputConfig.newBuilder()
              .setPredictionsFormat("jsonl")
              .setGcsDestination(gcsDestination)
              .build();

      String modelName = ModelName.of(project, location, model).toString();

      BatchPredictionJob batchPredictionJob =
          BatchPredictionJob.newBuilder()
              .setDisplayName(displayName)
              .setModel(modelName)
              .setModelParameters(modelParameters)
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .build();
      LocationName parent = LocationName.of(project, location);
      BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
      System.out.format("response: %s\n", response);
      System.out.format("\tName: %s\n", response.getName());
    }
  }
}

Python

View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Classification

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• THRESHOLD_VALUE (optional): Model returns only predictions that have confidence scores with at least this value
• SEGMENT_CLASSIFICATION (optional): A Boolean value that determines whether to request segment-level classification. Vertex AI returns labels and their confidence scores for the entire time segment of the video that you specified in the input instance. The default is true.
• SHOT_CLASSIFICATION (optional): A Boolean value that determines whether to request shot-level classification. Vertex AI determines the boundaries for each camera shot in the entire time segment of the video that you specified in the input instance. Vertex AI then returns labels and their confidence scores for each detected shot, along with the start and end time of the shot. The default is false.
• ONE_SEC_INTERVAL_CLASSIFICATION (optional): A Boolean value that determines whether to request classification for a video at one-second intervals. Vertex AI returns labels and their confidence scores for each second of the entire time segment of the video that you specified in the input instance. The default is false.
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/us-central1/MODEL_ID",
    "modelParameters": {
      "confidenceThreshold": THRESHOLD_VALUE,
      "segmentClassification": SEGMENT_CLASSIFICATION,
      "shotClassification": SHOT_CLASSIFICATION,
      "oneSecIntervalClassification": ONE_SEC_INTERVAL_CLASSIFICATION
    },
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_NUMBER/locations/us-central1/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Java

View on GitHub Feedback

import com.google.cloud.aiplatform.util.ValueConverter;
import com.google.cloud.aiplatform.v1.BatchDedicatedResources;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.BatchPredictionJob.InputConfig;
import com.google.cloud.aiplatform.v1.BatchPredictionJob.OutputConfig;
import com.google.cloud.aiplatform.v1.BatchPredictionJob.OutputInfo;
import com.google.cloud.aiplatform.v1.BigQueryDestination;
import com.google.cloud.aiplatform.v1.BigQuerySource;
import com.google.cloud.aiplatform.v1.CompletionStats;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.MachineSpec;
import com.google.cloud.aiplatform.v1.ManualBatchTuningParameters;
import com.google.cloud.aiplatform.v1.ModelName;
import com.google.cloud.aiplatform.v1.ResourcesConsumed;
import com.google.cloud.aiplatform.v1.schema.predict.params.VideoClassificationPredictionParams;
import com.google.protobuf.Any;
import com.google.protobuf.Value;
import com.google.rpc.Status;
import java.io.IOException;
import java.util.List;

public class CreateBatchPredictionJobVideoClassificationSample {

  public static void main(String[] args) throws IOException {
    String batchPredictionDisplayName = "YOUR_VIDEO_CLASSIFICATION_DISPLAY_NAME";
    String modelId = "YOUR_MODEL_ID";
    String gcsSourceUri =
        "gs://YOUR_GCS_SOURCE_BUCKET/path_to_your_video_source/[file.csv/file.jsonl]";
    String gcsDestinationOutputUriPrefix =
        "gs://YOUR_GCS_SOURCE_BUCKET/destination_output_uri_prefix/";
    String project = "YOUR_PROJECT_ID";
    createBatchPredictionJobVideoClassification(
        batchPredictionDisplayName, modelId, gcsSourceUri, gcsDestinationOutputUriPrefix, project);
  }

  static void createBatchPredictionJobVideoClassification(
      String batchPredictionDisplayName,
      String modelId,
      String gcsSourceUri,
      String gcsDestinationOutputUriPrefix,
      String project)
      throws IOException {
    JobServiceSettings jobServiceSettings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient jobServiceClient = JobServiceClient.create(jobServiceSettings)) {
      String location = "us-central1";
      LocationName locationName = LocationName.of(project, location);

      VideoClassificationPredictionParams modelParamsObj =
          VideoClassificationPredictionParams.newBuilder()
              .setConfidenceThreshold(((float) 0.5))
              .setMaxPredictions(10000)
              .setSegmentClassification(true)
              .setShotClassification(true)
              .setOneSecIntervalClassification(true)
              .build();

      Value modelParameters = ValueConverter.toValue(modelParamsObj);

      ModelName modelName = ModelName.of(project, location, modelId);
      GcsSource.Builder gcsSource = GcsSource.newBuilder();
      gcsSource.addUris(gcsSourceUri);
      InputConfig inputConfig =
          InputConfig.newBuilder().setInstancesFormat("jsonl").setGcsSource(gcsSource).build();

      GcsDestination gcsDestination =
          GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
      OutputConfig outputConfig =
          OutputConfig.newBuilder()
              .setPredictionsFormat("jsonl")
              .setGcsDestination(gcsDestination)
              .build();

      BatchPredictionJob batchPredictionJob =
          BatchPredictionJob.newBuilder()
              .setDisplayName(batchPredictionDisplayName)
              .setModel(modelName.toString())
              .setModelParameters(modelParameters)
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .build();
      BatchPredictionJob batchPredictionJobResponse =
          jobServiceClient.createBatchPredictionJob(locationName, batchPredictionJob);

      System.out.println("Create Batch Prediction Job Video Classification Response");
      System.out.format("\tName: %s\n", batchPredictionJobResponse.getName());
      System.out.format("\tDisplay Name: %s\n", batchPredictionJobResponse.getDisplayName());
      System.out.format("\tModel %s\n", batchPredictionJobResponse.getModel());
      System.out.format(
          "\tModel Parameters: %s\n", batchPredictionJobResponse.getModelParameters());

      System.out.format("\tState: %s\n", batchPredictionJobResponse.getState());
      System.out.format("\tCreate Time: %s\n", batchPredictionJobResponse.getCreateTime());
      System.out.format("\tStart Time: %s\n", batchPredictionJobResponse.getStartTime());
      System.out.format("\tEnd Time: %s\n", batchPredictionJobResponse.getEndTime());
      System.out.format("\tUpdate Time: %s\n", batchPredictionJobResponse.getUpdateTime());
      System.out.format("\tLabels: %s\n", batchPredictionJobResponse.getLabelsMap());

      InputConfig inputConfigResponse = batchPredictionJobResponse.getInputConfig();
      System.out.println("\tInput Config");
      System.out.format("\t\tInstances Format: %s\n", inputConfigResponse.getInstancesFormat());

      GcsSource gcsSourceResponse = inputConfigResponse.getGcsSource();
      System.out.println("\t\tGcs Source");
      System.out.format("\t\t\tUris %s\n", gcsSourceResponse.getUrisList());

      BigQuerySource bigQuerySource = inputConfigResponse.getBigquerySource();
      System.out.println("\t\tBigquery Source");
      System.out.format("\t\t\tInput_uri: %s\n", bigQuerySource.getInputUri());

      OutputConfig outputConfigResponse = batchPredictionJobResponse.getOutputConfig();
      System.out.println("\tOutput Config");
      System.out.format(
          "\t\tPredictions Format: %s\n", outputConfigResponse.getPredictionsFormat());

      GcsDestination gcsDestinationResponse = outputConfigResponse.getGcsDestination();
      System.out.println("\t\tGcs Destination");
      System.out.format(
          "\t\t\tOutput Uri Prefix: %s\n", gcsDestinationResponse.getOutputUriPrefix());

      BigQueryDestination bigQueryDestination = outputConfigResponse.getBigqueryDestination();
      System.out.println("\t\tBig Query Destination");
      System.out.format("\t\t\tOutput Uri: %s\n", bigQueryDestination.getOutputUri());

      BatchDedicatedResources batchDedicatedResources =
          batchPredictionJobResponse.getDedicatedResources();
      System.out.println("\tBatch Dedicated Resources");
      System.out.format(
          "\t\tStarting Replica Count: %s\n", batchDedicatedResources.getStartingReplicaCount());
      System.out.format(
          "\t\tMax Replica Count: %s\n", batchDedicatedResources.getMaxReplicaCount());

      MachineSpec machineSpec = batchDedicatedResources.getMachineSpec();
      System.out.println("\t\tMachine Spec");
      System.out.format("\t\t\tMachine Type: %s\n", machineSpec.getMachineType());
      System.out.format("\t\t\tAccelerator Type: %s\n", machineSpec.getAcceleratorType());
      System.out.format("\t\t\tAccelerator Count: %s\n", machineSpec.getAcceleratorCount());

      ManualBatchTuningParameters manualBatchTuningParameters =
          batchPredictionJobResponse.getManualBatchTuningParameters();
      System.out.println("\tManual Batch Tuning Parameters");
      System.out.format("\t\tBatch Size: %s\n", manualBatchTuningParameters.getBatchSize());

      OutputInfo outputInfo = batchPredictionJobResponse.getOutputInfo();
      System.out.println("\tOutput Info");
      System.out.format("\t\tGcs Output Directory: %s\n", outputInfo.getGcsOutputDirectory());
      System.out.format("\t\tBigquery Output Dataset: %s\n", outputInfo.getBigqueryOutputDataset());

      Status status = batchPredictionJobResponse.getError();
      System.out.println("\tError");
      System.out.format("\t\tCode: %s\n", status.getCode());
      System.out.format("\t\tMessage: %s\n", status.getMessage());
      List<Any> details = status.getDetailsList();

      for (Status partialFailure : batchPredictionJobResponse.getPartialFailuresList()) {
        System.out.println("\tPartial Failure");
        System.out.format("\t\tCode: %s\n", partialFailure.getCode());
        System.out.format("\t\tMessage: %s\n", partialFailure.getMessage());
        List<Any> partialFailureDetailsList = partialFailure.getDetailsList();
      }

      ResourcesConsumed resourcesConsumed = batchPredictionJobResponse.getResourcesConsumed();
      System.out.println("\tResources Consumed");
      System.out.format("\t\tReplica Hours: %s\n", resourcesConsumed.getReplicaHours());

      CompletionStats completionStats = batchPredictionJobResponse.getCompletionStats();
      System.out.println("\tCompletion Stats");
      System.out.format("\t\tSuccessful Count: %s\n", completionStats.getSuccessfulCount());
      System.out.format("\t\tFailed Count: %s\n", completionStats.getFailedCount());
      System.out.format("\t\tIncomplete Count: %s\n", completionStats.getIncompleteCount());
    }
  }
}

Node.js

View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.\
 * (Not necessary if passing values as arguments)
 */

// const batchPredictionDisplayName = 'YOUR_BATCH_PREDICTION_DISPLAY_NAME';
// const modelId = 'YOUR_MODEL_ID';
// const gcsSourceUri = 'YOUR_GCS_SOURCE_URI';
// const gcsDestinationOutputUriPrefix = 'YOUR_GCS_DEST_OUTPUT_URI_PREFIX';
//    eg. "gs://<your-gcs-bucket>/destination_path"
// const project = 'YOUR_PROJECT_ID';
// const location = 'YOUR_PROJECT_LOCATION';
const aiplatform = require('@google-cloud/aiplatform');
const {params} = aiplatform.protos.google.cloud.aiplatform.v1.schema.predict;

// Imports the Google Cloud Job Service Client library
const {JobServiceClient} = require('@google-cloud/aiplatform').v1;

// Specifies the location of the api endpoint
const clientOptions = {
  apiEndpoint: 'us-central1-aiplatform.googleapis.com',
};

// Instantiates a client
const jobServiceClient = new JobServiceClient(clientOptions);

async function createBatchPredictionJobVideoClassification() {
  // Configure the parent resource
  const parent = `projects/${project}/locations/${location}`;
  const modelName = `projects/${project}/locations/${location}/models/${modelId}`;

  // For more information on how to configure the model parameters object, see
  // https://cloud.google.com/ai-platform-unified/docs/predictions/batch-predictions
  const modelParamsObj = new params.VideoClassificationPredictionParams({
    confidenceThreshold: 0.5,
    maxPredictions: 1000,
    segmentClassification: true,
    shotClassification: true,
    oneSecIntervalClassification: true,
  });

  const modelParameters = modelParamsObj.toValue();

  const inputConfig = {
    instancesFormat: 'jsonl',
    gcsSource: {uris: [gcsSourceUri]},
  };
  const outputConfig = {
    predictionsFormat: 'jsonl',
    gcsDestination: {outputUriPrefix: gcsDestinationOutputUriPrefix},
  };
  const batchPredictionJob = {
    displayName: batchPredictionDisplayName,
    model: modelName,
    modelParameters,
    inputConfig,
    outputConfig,
  };
  const request = {
    parent,
    batchPredictionJob,
  };

  // Create batch prediction job request
  const [response] = await jobServiceClient.createBatchPredictionJob(request);

  console.log('Create batch prediction job video classification response');
  console.log(`Name : ${response.name}`);
  console.log('Raw response:');
  console.log(JSON.stringify(response, null, 2));
}
createBatchPredictionJobVideoClassification();

Python

View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Object tracking

REST & CMD LINE

Before using any of the request data, make the following replacements:

• LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
• PROJECT: Your project ID or project number.
• BATCH_JOB_NAME: Display name for the batch job
• MODEL_ID: The ID for the model to use for making predictions
• THRESHOLD_VALUE (optional): Vertex AI returns only predictions that have confidence scores with at least this value. The default is 0.0.
• URI: Cloud Storage URI where your input JSONL file is located.
• BUCKET: Your Cloud Storage bucket
• PROJECT_NUMBER: Project number for your project (appears in the response)

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs

Request JSON body:

{
    "displayName": "BATCH_JOB_NAME",
    "model": "projects/PROJECT/locations/us-central1/MODEL_ID",
    "modelParameters": {
      "confidenceThreshold": THRESHOLD_VALUE,
    },
    "inputConfig": {
        "instancesFormat": "jsonl",
        "gcsSource": {
            "uris": ["URI"],
        },
    },
    "outputConfig": {
        "predictionsFormat": "jsonl",
        "gcsDestination": {
            "outputUriPrefix": "OUTPUT_BUCKET",
        },
    },
}

To send your request, choose one of these options:

curl

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs"

PowerShell

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path for your service account private key file.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
  "displayName": "BATCH_JOB_NAME 202005291958",
  "model": "projects/PROJECT_NUMBER/locations/us-central1/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "jsonl",
    "gcsSource": {
      "uris": [
        "CONTENT"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET"
    }
  },
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-05-30T02:58:44.341643Z",
  "updateTime": "2020-05-30T02:58:44.341643Z",
  "modelDisplayName": "MODEL_NAME",
  "modelObjective": "MODEL_OBJECTIVE"
}

You can poll for the status of the batch job using the BATCH_JOB_ID until the job state is JOB_STATE_SUCCEEDED.

Java

View on GitHub Feedback

import com.google.cloud.aiplatform.util.ValueConverter;
import com.google.cloud.aiplatform.v1.BatchDedicatedResources;
import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.BatchPredictionJob.InputConfig;
import com.google.cloud.aiplatform.v1.BatchPredictionJob.OutputConfig;
import com.google.cloud.aiplatform.v1.BatchPredictionJob.OutputInfo;
import com.google.cloud.aiplatform.v1.BigQueryDestination;
import com.google.cloud.aiplatform.v1.BigQuerySource;
import com.google.cloud.aiplatform.v1.CompletionStats;
import com.google.cloud.aiplatform.v1.GcsDestination;
import com.google.cloud.aiplatform.v1.GcsSource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.MachineSpec;
import com.google.cloud.aiplatform.v1.ManualBatchTuningParameters;
import com.google.cloud.aiplatform.v1.ModelName;
import com.google.cloud.aiplatform.v1.ResourcesConsumed;
import com.google.cloud.aiplatform.v1.schema.predict.params.VideoObjectTrackingPredictionParams;
import com.google.protobuf.Any;
import com.google.protobuf.Value;
import com.google.rpc.Status;
import java.io.IOException;
import java.util.List;

public class CreateBatchPredictionJobVideoObjectTrackingSample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String batchPredictionDisplayName = "YOUR_VIDEO_OBJECT_TRACKING_DISPLAY_NAME";
    String modelId = "YOUR_MODEL_ID";
    String gcsSourceUri =
        "gs://YOUR_GCS_SOURCE_BUCKET/path_to_your_video_source/[file.csv/file.jsonl]";
    String gcsDestinationOutputUriPrefix =
        "gs://YOUR_GCS_SOURCE_BUCKET/destination_output_uri_prefix/";
    String project = "YOUR_PROJECT_ID";
    batchPredictionJobVideoObjectTracking(
        batchPredictionDisplayName, modelId, gcsSourceUri, gcsDestinationOutputUriPrefix, project);
  }

  static void batchPredictionJobVideoObjectTracking(
      String batchPredictionDisplayName,
      String modelId,
      String gcsSourceUri,
      String gcsDestinationOutputUriPrefix,
      String project)
      throws IOException {
    JobServiceSettings jobServiceSettings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient jobServiceClient = JobServiceClient.create(jobServiceSettings)) {
      String location = "us-central1";
      LocationName locationName = LocationName.of(project, location);
      ModelName modelName = ModelName.of(project, location, modelId);

      VideoObjectTrackingPredictionParams modelParamsObj =
          VideoObjectTrackingPredictionParams.newBuilder()
              .setConfidenceThreshold(((float) 0.5))
              .build();

      Value modelParameters = ValueConverter.toValue(modelParamsObj);

      GcsSource.Builder gcsSource = GcsSource.newBuilder();
      gcsSource.addUris(gcsSourceUri);
      InputConfig inputConfig =
          InputConfig.newBuilder().setInstancesFormat("jsonl").setGcsSource(gcsSource).build();

      GcsDestination gcsDestination =
          GcsDestination.newBuilder().setOutputUriPrefix(gcsDestinationOutputUriPrefix).build();
      OutputConfig outputConfig =
          OutputConfig.newBuilder()
              .setPredictionsFormat("jsonl")
              .setGcsDestination(gcsDestination)
              .build();

      BatchPredictionJob batchPredictionJob =
          BatchPredictionJob.newBuilder()
              .setDisplayName(batchPredictionDisplayName)
              .setModel(modelName.toString())
              .setModelParameters(modelParameters)
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .build();
      BatchPredictionJob batchPredictionJobResponse =
          jobServiceClient.createBatchPredictionJob(locationName, batchPredictionJob);

      System.out.println("Create Batch Prediction Job Video Object Tracking Response");
      System.out.format("\tName: %s\n", batchPredictionJobResponse.getName());
      System.out.format("\tDisplay Name: %s\n", batchPredictionJobResponse.getDisplayName());
      System.out.format("\tModel %s\n", batchPredictionJobResponse.getModel());
      System.out.format(
          "\tModel Parameters: %s\n", batchPredictionJobResponse.getModelParameters());

      System.out.format("\tState: %s\n", batchPredictionJobResponse.getState());
      System.out.format("\tCreate Time: %s\n", batchPredictionJobResponse.getCreateTime());
      System.out.format("\tStart Time: %s\n", batchPredictionJobResponse.getStartTime());
      System.out.format("\tEnd Time: %s\n", batchPredictionJobResponse.getEndTime());
      System.out.format("\tUpdate Time: %s\n", batchPredictionJobResponse.getUpdateTime());
      System.out.format("\tLabels: %s\n", batchPredictionJobResponse.getLabelsMap());

      InputConfig inputConfigResponse = batchPredictionJobResponse.getInputConfig();
      System.out.println("\tInput Config");
      System.out.format("\t\tInstances Format: %s\n", inputConfigResponse.getInstancesFormat());

      GcsSource gcsSourceResponse = inputConfigResponse.getGcsSource();
      System.out.println("\t\tGcs Source");
      System.out.format("\t\t\tUris %s\n", gcsSourceResponse.getUrisList());

      BigQuerySource bigQuerySource = inputConfigResponse.getBigquerySource();
      System.out.println("\t\tBigquery Source");
      System.out.format("\t\t\tInput_uri: %s\n", bigQuerySource.getInputUri());

      OutputConfig outputConfigResponse = batchPredictionJobResponse.getOutputConfig();
      System.out.println("\tOutput Config");
      System.out.format(
          "\t\tPredictions Format: %s\n", outputConfigResponse.getPredictionsFormat());

      GcsDestination gcsDestinationResponse = outputConfigResponse.getGcsDestination();
      System.out.println("\t\tGcs Destination");
      System.out.format(
          "\t\t\tOutput Uri Prefix: %s\n", gcsDestinationResponse.getOutputUriPrefix());

      BigQueryDestination bigQueryDestination = outputConfigResponse.getBigqueryDestination();
      System.out.println("\t\tBig Query Destination");
      System.out.format("\t\t\tOutput Uri: %s\n", bigQueryDestination.getOutputUri());

      BatchDedicatedResources batchDedicatedResources =
          batchPredictionJobResponse.getDedicatedResources();
      System.out.println("\tBatch Dedicated Resources");
      System.out.format(
          "\t\tStarting Replica Count: %s\n", batchDedicatedResources.getStartingReplicaCount());
      System.out.format(
          "\t\tMax Replica Count: %s\n", batchDedicatedResources.getMaxReplicaCount());

      MachineSpec machineSpec = batchDedicatedResources.getMachineSpec();
      System.out.println("\t\tMachine Spec");
      System.out.format("\t\t\tMachine Type: %s\n", machineSpec.getMachineType());
      System.out.format("\t\t\tAccelerator Type: %s\n", machineSpec.getAcceleratorType());
      System.out.format("\t\t\tAccelerator Count: %s\n", machineSpec.getAcceleratorCount());

      ManualBatchTuningParameters manualBatchTuningParameters =
          batchPredictionJobResponse.getManualBatchTuningParameters();
      System.out.println("\tManual Batch Tuning Parameters");
      System.out.format("\t\tBatch Size: %s\n", manualBatchTuningParameters.getBatchSize());

      OutputInfo outputInfo = batchPredictionJobResponse.getOutputInfo();
      System.out.println("\tOutput Info");
      System.out.format("\t\tGcs Output Directory: %s\n", outputInfo.getGcsOutputDirectory());
      System.out.format("\t\tBigquery Output Dataset: %s\n", outputInfo.getBigqueryOutputDataset());

      Status status = batchPredictionJobResponse.getError();
      System.out.println("\tError");
      System.out.format("\t\tCode: %s\n", status.getCode());
      System.out.format("\t\tMessage: %s\n", status.getMessage());
      List<Any> details = status.getDetailsList();

      for (Status partialFailure : batchPredictionJobResponse.getPartialFailuresList()) {
        System.out.println("\tPartial Failure");
        System.out.format("\t\tCode: %s\n", partialFailure.getCode());
        System.out.format("\t\tMessage: %s\n", partialFailure.getMessage());
        List<Any> partialFailureDetailsList = partialFailure.getDetailsList();
      }

      ResourcesConsumed resourcesConsumed = batchPredictionJobResponse.getResourcesConsumed();
      System.out.println("\tResources Consumed");
      System.out.format("\t\tReplica Hours: %s\n", resourcesConsumed.getReplicaHours());

      CompletionStats completionStats = batchPredictionJobResponse.getCompletionStats();
      System.out.println("\tCompletion Stats");
      System.out.format("\t\tSuccessful Count: %s\n", completionStats.getSuccessfulCount());
      System.out.format("\t\tFailed Count: %s\n", completionStats.getFailedCount());
      System.out.format("\t\tIncomplete Count: %s\n", completionStats.getIncompleteCount());
    }
  }
}

Node.js

View on GitHub Feedback

/**
 * TODO(developer): Uncomment these variables before running the sample.\
 * (Not necessary if passing values as arguments)
 */

// const batchPredictionDisplayName = 'YOUR_BATCH_PREDICTION_DISPLAY_NAME';
// const modelId = 'YOUR_MODEL_ID';
// const gcsSourceUri = 'YOUR_GCS_SOURCE_URI';
// const gcsDestinationOutputUriPrefix = 'YOUR_GCS_DEST_OUTPUT_URI_PREFIX';
//    eg. "gs://<your-gcs-bucket>/destination_path"
// const project = 'YOUR_PROJECT_ID';
// const location = 'YOUR_PROJECT_LOCATION';
const aiplatform = require('@google-cloud/aiplatform');
const {params} = aiplatform.protos.google.cloud.aiplatform.v1.schema.predict;

// Imports the Google Cloud Job Service Client library
const {JobServiceClient} = require('@google-cloud/aiplatform').v1;

// Specifies the location of the api endpoint
const clientOptions = {
  apiEndpoint: 'us-central1-aiplatform.googleapis.com',
};

// Instantiates a client
const jobServiceClient = new JobServiceClient(clientOptions);

async function createBatchPredictionJobVideoObjectTracking() {
  // Configure the parent resource
  const parent = `projects/${project}/locations/${location}`;
  const modelName = `projects/${project}/locations/${location}/models/${modelId}`;

  // For more information on how to configure the model parameters object, see
  // https://cloud.google.com/ai-platform-unified/docs/predictions/batch-predictions
  const modelParamsObj = new params.VideoObjectTrackingPredictionParams({
    confidenceThreshold: 0.5,
  });

  const modelParameters = modelParamsObj.toValue();

  const inputConfig = {
    instancesFormat: 'jsonl',
    gcsSource: {uris: [gcsSourceUri]},
  };
  const outputConfig = {
    predictionsFormat: 'jsonl',
    gcsDestination: {outputUriPrefix: gcsDestinationOutputUriPrefix},
  };
  const batchPredictionJob = {
    displayName: batchPredictionDisplayName,
    model: modelName,
    modelParameters,
    inputConfig,
    outputConfig,
  };
  const request = {
    parent,
    batchPredictionJob,
  };

  // Create batch prediction job request
  const [response] = await jobServiceClient.createBatchPredictionJob(request);

  console.log('Create batch prediction job video object tracking response');
  console.log(`Name : ${response.name}`);
  console.log('Raw response:');
  console.log(JSON.stringify(response, null, 2));
}
createBatchPredictionJobVideoObjectTracking();

Python

View on GitHub Feedback

def create_batch_prediction_job_sample(
    project: str,
    location: str,
    model_resource_name: str,
    job_display_name: str,
    gcs_source: Union[str, Sequence[str]],
    gcs_destination: str,
    sync: bool = True,
):
    aiplatform.init(project=project, location=location)

    my_model = aiplatform.Model(model_resource_name)

    batch_prediction_job = my_model.batch_predict(
        job_display_name=job_display_name,
        gcs_source=gcs_source,
        gcs_destination_prefix=gcs_destination,
        sync=sync,
    )

    batch_prediction_job.wait()

    print(batch_prediction_job.display_name)
    print(batch_prediction_job.resource_name)
    print(batch_prediction_job.state)
    return batch_prediction_job

Retrieve batch prediction results

Vertex AI sends batch prediction output to your specified destination.

Custom-trained

When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.

Image

When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.

Tabular

Retrieving your results depends on the destination that you specified for the batch prediction, either BigQuery or Cloud Storage.

• Retrieving results in BigQuery
• Retrieving results in Cloud Storage

Retrieving results in BigQuery

If you specified BigQuery as your output destination, the results of your batch prediction request are returned in a dataset in the BigQuery project you specified. If you didn't specify a BigQuery dataset, Vertex AI creates a new dataset. The name of the dataset is the name of your model prepended with "prediction_" and appended with the timestamp of when the prediction job started. You can find the BigQuery dataset name in the Batch predictions tab for your model.

The BigQuery dataset contains two tables, which are also appended with the timestamp of when the prediction job started: predictions_timestamp and errors_timestampe. The errors table has a row for every row in your prediction request for which Vertex AI could not return a prediction (for example, if a non-nullable feature was null). The predictions table contains a row for every prediction returned.

In the predictions table, Vertex AI returns your prediction data, and creates a new column for the prediction results by prepending "predicted_" onto your target column name. The prediction results column contains a nested BigQuery structure that contains the prediction results.

To retrieve the prediction results, you can use a query in the BigQuery console. The format of the query depends on your model type. The following sample queries show how to get results for each objective.

Classification:

SELECT predicted_<target-column-name>.classes AS classes,
predicted_<target-column-name>.scores AS scores,

"classes" is the list of potential classes, and "scores" are the corresponding confidence scores.

Forecasting:

Preview

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

For models optimized for quantile loss, use the following query:

SELECT predicted_target-column-name.quantile_predictions,
predicted_target-column-name.quantile_values,
FROM bq-dataset-name.predictions_timestamp

For all other optimization, use the following query:

SELECT predicted_target-column-name.value,
FROM bq-dataset-name.predictions_timestamp

Regression:

SELECT predicted_<target-column-name>.value,
predicted_<target-column-name>.lower_bound,
predicted_<target-column-name>.upper_bound
FROM <bq-dataset-name>.predictions_timestamp

Retrieving results in Cloud Storage

If you specified Cloud Storage as your output destination, the results of your batch prediction request are returned as CSV files in a new folder in the bucket you specified. The name of the folder is the name of your model, prepended with "prediction-" and appended with the timestamp of when the prediction job started. You can find the Cloud Storage folder name in the Batch predictions tab for your model.

The Cloud Storage folder contains two types of files: error files and prediction files. If the results are large, additional files are created.

The error files are named errors_1.csv, errors_2.csv, and so on. They contain a header row, and a row for every row in your prediction request for which Vertex AI could not return a prediction.

The prediction files are named predictions_1.csv, predictions_2.csv, and so on. They contain a header row with the column names, and a row for every prediction returned.

In the prediction files, Vertex AI returns your prediction data, and creates one or more new columns for the prediction results, depending on your model type:

Classification:

For each potential value of your target column, a column named <target-column-name>_<value>_score is added to the results. This column contains the score, or confidence estimate, for that value.

Forecasting:

Preview

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

The predicted values are returned in a column named predicted_<target-column-name>. For quantile forecasts, the output column contains the quantile predictions and quantile values in the JSON format.

Regression:

The predicted value for that row is returned in a column named predicted_<target-column-name>. The prediction interval is not returned for CSV output.

Text

When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.

Video

When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.

Example batch prediction results

The following examples are batch prediction results for several AutoML model types and objectives. For information about the results, see Interpreting results from AutoML models.

Image

Classification

{
  "instance": {"content": "gs://bucket/image.jpg", "mimeType": "image/jpeg"},
  "prediction": {
    "ids": [1, 2],
    "displayNames": ["cat", "dog"],
    "confidences": [0.7, 0.5]
  }
}

Object detection

{
  "instance": {"content": "gs://bucket/image.jpg", "mimeType": "image/jpeg"},
  "prediction": {
    "ids": [1, 2],
    "displayNames": ["cat", "dog"],
    "bboxes":  [
      [0.1, 0.2, 0.3, 0.4],
      [0.2, 0.3, 0.4, 0.5]
    ],
    "confidences": [0.7, 0.5]
  }
}

Text

Classification

{
  "instance": {"content": "gs://bucket/text.txt", "mimeType": "text/plain"},
  "predictions": [
    {
      "ids": [
        "1234567890123456789",
        "2234567890123456789",
        "3234567890123456789"
      ],
      "displayNames": [
        "GreatService",
        "Suggestion",
        "InfoRequest"
      ],
      "confidences": [
        0.8986392080783844,
        0.81984345316886902,
        0.7722353458404541
      ]
    }
  ]
}

Entity extraction

{
  "key": 1,
  "predictions": {
    "ids": [
      "1234567890123456789",
      "2234567890123456789",
      "3234567890123456789"
    ],
    "displayNames": [
      "SpecificDisease",
      "DiseaseClass",
      "SpecificDisease"
    ],
    "textSegmentStartOffsets":  [13, 40, 57],
    "textSegmentEndOffsets": [29, 51, 75],
    "confidences": [
      0.99959725141525269,
      0.99912621492484128,
      0.99935531616210938
    ]
  }
}

Sentiment analysis

{
  "instance": {"content": "gs://bucket/text.txt", "mimeType": "text/plain"},
  "prediction": {"sentiment": 8}
}

Video

Action recognition

{
  "instance": {
   "content": "gs://bucket/video.mp4",
    "mimeType": "video/mp4",
    "timeSegmentStart": "1s",
    "timeSegmentEnd": "5s"
  }
  "prediction": [{
    "id": "1",
    "displayName": "swing",
    "timeSegmentStart": "1.2s",
    "timeSegmentEnd": "1.2s",
    "confidence": 0.7
  }, {
    "id": "2",
    "displayName": "jump",
    "timeSegmentStart": "3.4s",
    "timeSegmentEnd": "3.4s",
    "confidence": 0.5
  }]
}

Classification

{
  "instance": {
   "content": "gs://bucket/video.mp4",
    "mimeType": "video/mp4",
    "timeSegmentStart": "1s",
    "timeSegmentEnd": "5s"
  }
  "prediction": [{
    "id": "1",
    "displayName": "cat",
    "type": "segment-classification",
    "timeSegmentStart": "1s",
    "timeSegmentEnd": "5s",
    "confidence": 0.7
  }, {
    "id": "1",
    "displayName": "cat",
    "type": "shot-classification",
    "timeSegmentStart": "1s",
    "timeSegmentEnd": "4s",
    "confidence": 0.9
  }, {
    "id": "2",
    "displayName": "dog",
    "type": "shot-classification",
    "timeSegmentStart": "4s",
    "timeSegmentEnd": "5s",
    "confidence": 0.6
  }, {
    "id": "1",
    "displayName": "cat",
    "type": "one-sec-interval-classification",
    "timeSegmentStart": "1s",
    "timeSegmentEnd": "1s",
    "confidence": 0.95
  }, {
    "id": "1",
    "displayName": "cat",
    "type": "one-sec-interval-classification",
    "timeSegmentStart": "2s",
    "timeSegmentEnd": "2s",
    "confidence": 0.9
  }, {
    "id": "1",
    "displayName": "cat",
    "type": "one-sec-interval-classification",
    "timeSegmentStart": "3s",
    "timeSegmentEnd": "3s",
    "confidence": 0.85
  }, {
    "id": "2",
    "displayName": "dog",
    "type": "one-sec-interval-classification",
    "timeSegmentStart": "4s",
    "timeSegmentEnd": "4s",
    "confidence": 0.6
  }]
}

Object tracking

{
  "instance": {
   "content": "gs://bucket/video.mp4",
    "mimeType": "video/mp4",
    "timeSegmentStart": "1s",
    "timeSegmentEnd": "5s"
  }
  "prediction": [{
    "id": "1",
    "displayName": "cat",
    "timeSegmentStart": "1.2s",
    "timeSegmentEnd": "3.4s",
    "frames": [{
      "timeOffset": "1.2s",
      "xMin": 0.1,
      "xMax": 0.2,
      "yMin": 0.3,
      "yMax": 0.4
    }, {
      "timeOffset": "3.4s",
      "xMin": 0.2,
      "xMax": 0.3,
      "yMin": 0.4,
      "yMax": 0.5,
    }],
    "confidence": 0.7
  }, {
    "id": "1",
    "displayName": "cat",
    "timeSegmentStart": "4.8s",
    "timeSegmentEnd": "4.8s",
    "frames": [{
      "timeOffset": "4.8s",
      "xMin": 0.2,
      "xMax": 0.3,
      "yMin": 0.4,
      "yMax": 0.5,
    }],
    "confidence": 0.6
  }, {
    "id": "2",
    "displayName": "dog",
    "timeSegmentStart": "1.2s",
    "timeSegmentEnd": "3.4s",
    "frames": [{
      "timeOffset": "1.2s",
      "xMin": 0.1,
      "xMax": 0.2,
      "yMin": 0.3,
      "yMax": 0.4
    }, {
      "timeOffset": "3.4s",
      "xMin": 0.2,
      "xMax": 0.3,
      "yMin": 0.4,
      "yMax": 0.5,
    }],
    "confidence": 0.5
  }]
}

What's next

• Interpret prediction results from AutoML models.