Requesting a batch prediction is an asynchronous request (as opposed to online prediction, which is a synchronous request). You request batch predictions directly from the model resource; you don't need to deploy the model to an endpoint. For data types that support both batch and online predictions, use batch predictions when you don't require an immediate response and want to process accumulated data by using a single request.
To make a batch prediction, you specify an input source and an output location where Vertex AI stores predictions results. The inputs and outputs depend on the model type that you're working with. For example, batch predictions for the AutoML image model type require an input JSONL file and the name of a Cloud Storage bucket to store the output.
Batch request input
The input for batch requests specifies the items to send to your model for prediction. The following content provides input information for each model type.
Custom-trained
To get batch predictions from a custom-trained model, prepare your input data in one of the following ways:
JSONL
Use a JSONL file to specify a list of input instances to make predictions about. Store the JSONl file in a Cloud Storage bucket. The following example shows two instances in an input JSONL file:
{"dense_input": [1, 2, 3]}
{"dense_input": [4, 5, 6]}
TFRecord
Save input instances in the TFRecord format. You can optionally compress the TFRecord files with Gzip. Store the TFRecord files in a Cloud Storage bucket.
Vertex AI reads each instance in your TFRecord files as binary, then base64-encodes it and sets it in a JSON instance to the container that serves your model's predictions.
CSV
Specify one input instance per row in a CSV file. You must include a header row. Vertex AI uses your header row to convert each input instance into JSON before sending it to the container that serves your model's predictions.
The following example shows a CSV file with two input instances:
input1,input2,input3 0.1,1.2,3.0 4.0,5.0,6.0
File list
Create a text file where each row is the Cloud Storage URI to a file. Vertex AI reads each URI as binary, then base64-encodes it and sends it in a JSON instance to the container that serves your model's predictions.
If you plan to use the Google Cloud Console to get batch predictions, paste your file list directly into the Cloud Console. Otherwise save your file list in a Cloud Storage bucket.
The following example shows a file list with two input instances:
gs://path/to/image/image1.jpg gs://path/to/image/image2.jpg
Image
Use a JSONL file to specify a list of images to make predictions about and then store the JSONL file in a Cloud Storage bucket. The following sample shows a single line in an input JSONL file.
{"content": "gs://sourcebucket/datasets/images/source_image.jpg", "mimeType": "image/jpeg"}
Tabular
For tabular data, you can use a CSV file in a Cloud Storage bucket or a table in BigQuery. You can provide prediction data as a CSV in Cloud Storage or as a table in BigQuery.
Your data source must contain tabular data that includes all of the columns, in any order, that was used to train the model. You can include columns that were not in the training data, or that were in the training data but excluded from use for training. These extra columns are included in the prediction output, but they are not used for generating the prediction.
You do not need to use the same destination for your results as your prediction data. For example, if you used BigQuery for the prediction data source, you can send results to a CSV file in Cloud Storage.
BigQuery table requirements
- BigQuery data source tables must be no larger than 100 GB.
- You must use a multi-regional BigQuery dataset in the
USorEUlocations. - If the table is in a different project, you must provide the
BigQuery Data Editorrole to the Vertex AI service account in that project.
CSV file requirements
- The first line of the data source must contain the name of the columns.
- Each data source file must not be larger than 10 GB. You can include multiple files, up to a maximum amount of 100 GB.
- If the Cloud Storage bucket is in a different project than where you
use Vertex AI, you must provide the
Storage Object Creatorrole to the Vertex AI service account in that project.
Forecasting requirements
In addition to the BigQuery table or CSV file requirements, input for forecasting models must also adhere to the following requirements:
- All values in the time column must be present and valid.
- The data frequency in the batch prediction table must be the same frequency as the training data. You cannot have missing rows in the time series. Manually insert missing rows according to the proper domain knowledge.
- Time series with duplicate timestamps are removed from predictions. To include them, remove any duplicate timestamps.
- Provide historical data for each time series to predict. For the most accurate predictions, the amount of data should equal the context window, which was set in training. If you provide less data, Vertex AI pads data with empty values. For example, if the context window is 14 days, provide at least 14 days of historical data.
- The forecast starts on the first row of a time series (ordered by time) with a null value in the target column. The null value must be continuous within the time series. For example, if the target column is ordered by time, you cannot have something like `1`, `2`, `null`, `3`, `4`, `null`, `null` for a single time series. For CSV files, Vertex AI treats an empty string as null, and for BigQuery, null values are natively supported.
Text
Classification and sentiment analysis
Use a JSONL file to specify a list of documents to make predictions about and then store the JSONL file in a Cloud Storage bucket. The following sample shows a single line in an input JSONL file.
{"content": "gs://sourcebucket/datasets/texts/source_text.txt", "mimeType": "text/plain"}
Entity extraction
For entity extraction, you can include inline text or references to
documents that are in a Cloud Storage bucket. For each document, you
can also add a key field to your input.
Typically, batch prediction results map inputs and outputs by using the
instance field, which includes the content and
mimeType fields. If you use the key field in your
input, the batch prediction output replaces use the instance
field with the key field. This helps simplify your batch
prediction output if, for example, your input includes large text
snippets.
The following sample shows a JSONL file that includes references to
documents and inline text snippets with and without the key
field.
{"content": "gs://sourcebucket/datasets/texts/source_text.txt", "mimeType": "text/plain"}
{"content": "gs://bucket/sample.txt", "mimeType": "text/plain", "key": "sample-file"}
{"content": "Text snippet", "mimeType": "text/plain"}
{"content": "Sample text snippet", "mimeType": "text/plain", "key": "sample-snippet"}
Video
Use a JSONL file to specify a list of videos to make predictions for, and
then store the JSONL file in a Cloud Storage bucket. You can specify
Infinity for the timeSegmentEnd field to specify the
end of the video. The following sample shows a single line in an input JSONL
file.
{'content': 'gs://sourcebucket/datasets/videos/source_video.mp4', 'mimeType': 'video/mp4', 'timeSegmentStart': '0.0s', 'timeSegmentEnd': '2.366667s'}
Requesting a batch prediction
For batch prediction requests, you can use the Google Cloud Console or the Vertex AI API. Depending on the number of items that you've submitted, a batch prediction task can take some time to complete.
Cloud Console
Use the Cloud Console to request a batch prediction.
In the Cloud Console, in the Vertex AI section, go to the Batch predictions page.
Click Create to open the New batch prediction window and complete the following steps:
Custom-trained
- Enter a name for the batch prediction.
- For Model name, select the name of the model to use for this batch prediction.
- For Select source:
- If you have formatted your input as JSONL, CSV, or TFRecord, select File on Cloud Storage (JSONL, CSV, TFRecord, TFRecord Gzip). Then specify your input file in the Source path field.
- If you are using a file list as input, select Files on Cloud Storage (other) and paste your file list into the following text box.
- In the Destination path field, specify the Cloud Storage directory where you want Vertex AI to store batch prediction output.
- Specify compute options for the batch prediction job: Number of compute nodes, Machine type, and (optionally) Accelerator type and Accelerator count
Image, text, or video
- Enter a name for the batch prediction.
- For Model name, select the name of the model to use for this batch prediction.
- For Source path, specify the Cloud Storage location where your JSONL input file is located.
- For the Destination path, specify a Cloud Storage location where the batch prediction results are stored. The Output format is determined by your model's objective. For example, AutoML models for text objectives output JSONL files.
Tabular
- Enter a name for the batch prediction.
- For Model name, select the name of the model to use for this batch prediction.
- For Select source, select whether your source input data is a CSV
file on Cloud Storage or a table in BigQuery.
- For CSV files, specify the Cloud Storage location where your CSV input file is located.
- For BigQuery tables, specify the project ID where the table is located, the BigQuery dataset ID, and the BigQuery table or view ID.
- For the Output, select CSV or BigQuery.
- For CSV, specify the Cloud Storage bucket where Vertex AI stores your output.
- For BigQuery, you can specify a project ID or an existing
dataset:
- To specify the project ID, enter the project ID in the Google Cloud project ID field. Vertex AI creates a new output dataset for you.
- To specify an existing dataset, enter its BigQuery path
in the Google Cloud project ID field, such as
bq://projectid.datasetid.
-
If you specified BigQuery as your output destination, and you want feature importance values returned for your predictions, check the Generate feature importance checkbox (Preview) .
Feature importance is not supported when the prediction data is returned in Cloud Storage or for forecasting models.
API
Use the Vertex AI API to send batch prediction requests.
Custom-trained
Select a tab depending on which tool you are using to get batch predictions:
REST & CMD LINE
Before using any of the request data below, make the following replacements:
LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
PROJECT: Your project ID or project number.
BATCH_JOB_NAME: Display name for the batch prediction job.
MODEL_ID: The ID for the model to use for making predictions.
INPUT_FORMAT: The format of your input data:
jsonl,csv,tf-record,tf-record-gzip, orfile-list.INPUT_URI: Cloud Storage URI of your input data. May contain wildcards.
OUTPUT_DIRECTORY: Cloud Storage URI of a directory where you want Vertex AI to save output.
MACHINE_TYPE: The machine resources to be used for this batch prediction job.
You can optionally configure the
machineSpecfield to use accelerators, but the following example does not demonstrate this.BATCH_SIZE: The number of instances to send in each prediction request; the default is 64. Increasing the batch size can lead to higher throughput, but it can also cause request timeouts.
STARTING_REPLICA_COUNT: The number of nodes for this batch prediction job.
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/LOCATION/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "INPUT_FORMAT",
"gcsSource": {
"uris": ["INPUT_URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_DIRECTORY",
},
},
"dedicatedResources" : {
"machineSpec" : {
"machineType": MACHINE_TYPE
},
"startingReplicaCount": STARTING_REPLICA_COUNT
},
"manualBatchTuningParameters": {
"batch_size": BATCH_SIZE,
}
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT/locations/us-central1/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"INPUT_URI"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_DIRECTORY"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
}
Java
In the following sample, replace PREDICTIONS_FORMAT with jsonl. To
learn how to replace the other placeholders, see the REST & CMD LINE tab of
this section.
Python
Feature importance
If you want feature importance values returned for your predictions, you must
use the v1beta1 version of the batchPredictionJobs.create
method, and
set the generateExplanation property to true. Note that forecasting models
don't support feature importance, so you cannot include it in your batch
prediction requests.
Feature importance, sometimes called feature attributions, is part of Vertex Explainable AI.
You can only set generateExplanation to true if you have configured your
Model for explanations.
Image
The following batch prediction example works for classification and object detection objectives.
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
- PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- THRESHOLD_VALUE (optional): Vertex AI returns only
predictions that have confidence scores with at least this value. The
default is
0.0. - MAX_PREDICTIONS (optional): Vertex AI returns up to this
many predictions starting with the predictions that have highest confidence
scores. The default is
10. - URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/LOCATION/models/MODEL_ID",
"modelParameters": {
"confidenceThreshold": THRESHOLD_VALUE,
"maxPredictions": MAX_PREDICTIONS
},
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT/locations/us-central1/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
You can poll for the status of the batch job using
the BATCH_JOB_ID until the job state is
JOB_STATE_SUCCEEDED.
Python
Tabular
The code needed to request batch predictions depends on whether the prediction data is CSV files in Cloud Storage or a BigQuery table. You can use a different format (Cloud Storage or BigQuery) for your prediction data and output destination. If the prediction data resides in a different project than where Vertex AI is running, make sure you set up the required permissions.
Requesting a batch prediction using CSV files
REST & CMD LINE
You use the batchPredictionJobs.create method to request a batch prediction.
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
- PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
-
URI: Paths (URIs) to the Cloud Storage buckets containing the training data.
There can be more than one. Each URI has the form:
gs://bucketName/pathToFileName
-
OUTPUT_URI_PREFIX: Path to a Cloud Storage destination where the
predictions will be written. Vertex AI writes batch predictions to a timestamped
subdirectory of this path. Set this value to a string with the following format:
gs://bucketName/pathToOutputDirectory
- MACHINE_TYPE: The machine resources to be used for this batch prediction job. Learn more.
- STARTING_REPLICA_COUNT: The starting number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, up to the maximum number of nodes, but will never fall below this number.
- MAX_REPLICA_COUNT: The maximum number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, but will never exceed the maximum. Optional, defaults to 10.
- BATCH_SIZE: The number of rows of prediction data used in each batch. Higher values can increase prediction performance, but if the size of the batch exceeds the size of the memory for a node, the prediction job fails. The default value is 4.
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME,
"model": "MODEL_ID",
"inputConfig": {
"instancesFormat": "csv",
"gcsSource": {
"uris": [
URI1,...
]
},
},
"outputConfig": {
"predictionsFormat": "csv",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_URI_PREFIX"
}
},
"dedicatedResources": {
"machineSpec": {
"machineType": "MACHINE_TYPE",
"acceleratorCount": "0"
},
"startingReplicaCount": STARTING_REPLICA_COUNT,
"maxReplicaCount": MAX_REPLICA_COUNT
},
"manualBatchTuningParameters": {
"batchSize": BATCH_SIZE
}
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/12345/locations/us-central1/batchPredictionJobs/67890",
"displayName": "batch_job_1 202005291958",
"model": "projects/12345/locations/us-central1/models/5678",
"state": "JOB_STATE_PENDING",
"inputConfig": {
"instancesFormat": "csv",
"gcsSource": {
"uris": [
"gs://bp_bucket/reg_mode_test"
]
}
},
"outputConfig": {
"predictionsFormat": "csv",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_URI_PREFIX"
}
},
"dedicatedResources": {
"machineSpec": {
"machineType": "n1-standard-32",
"acceleratorCount": "0"
},
"startingReplicaCount": 2,
"maxReplicaCount": 6
},
"manualBatchTuningParameters": {
"batchSize": 4
}
"outputInfo": {
"gcsOutputDataset": "OUTPUT_URI_PREFIX/prediction-batch_job_1 202005291958-2020-09-30T02:58:44.341643Z"
}
"state": "JOB_STATE_PENDING",
"createTime": "2020-09-30T02:58:44.341643Z",
"updateTime": "2020-09-30T02:58:44.341643Z",
}
Feature importance is not supported when the prediction data is returned in Cloud Storage.
Requesting a batch prediction using BigQuery
REST & CMD LINE
You use the batchPredictionJobs.create method to request a batch prediction.
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
- PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
-
INPUT_URI: Reference to the BigQuery data source. In the form:
bq://bqprojectId.bqDatasetId.bqTableId
-
OUTPUT_URI: Reference to the BigQuery destination (where the
predictions will be written). Specify the project ID and, optionally,
an existing dataset ID. If you specify just the project ID,
Vertex AI creates a new output dataset for you. Use the
following form:
bq://bqprojectId.bqDatasetId
- MACHINE_TYPE: The machine resources to be used for this batch prediction job. Learn more.
- STARTING_REPLICA_COUNT: The starting number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, up to the maximum number of nodes, but will never fall below this number.
- MAX_REPLICA_COUNT: The maximum number of nodes for this batch prediction job. The node count can be increased or decreased as required by load, but will never exceed the maximum. Optional, defaults to 10.
- BATCH_SIZE: The number of rows of prediction data used in each batch. Higher values can increase prediction performance, but if the size of the batch exceeds the size of the memory for a node, the prediction job fails. The default value is 4.
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME,
"model": "MODEL_ID",
"inputConfig": {
"instancesFormat": "bigquery",
"bigquerySource": {
"inputUri": "INPUT_URI"
}
},
"outputConfig": {
"predictionsFormat": "bigquery",
"bigqueryDestination": {
"outputUri": "OUTPUT_URI"
}
},
"dedicatedResources": {
"machineSpec": {
"machineType": "MACHINE_TYPE",
"acceleratorCount": "0"
},
"startingReplicaCount": STARTING_REPLICA_COUNT,
"maxReplicaCount": MAX_REPLICA_COUNT
},
"manualBatchTuningParameters": {
"batchSize": BATCH_SIZE
}
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/12345/locations/us-central1/batchPredictionJobs/67890",
"displayName": "batch_job_1 202005291958",
"model": "projects/12345/locations/us-central1/models/5678",
"state": "JOB_STATE_PENDING",
"inputConfig": {
"instancesFormat": "bigquery",
"bigquerySource": {
"inputUri": "INPUT_URI"
}
},
"outputConfig": {
"predictionsFormat": "bigquery",
"bigqueryDestination": {
"outputUri": bq://12345
}
},
"dedicatedResources": {
"machineSpec": {
"machineType": "n1-standard-32",
"acceleratorCount": "0"
},
"startingReplicaCount": 2,
"maxReplicaCount": 6
},
"manualBatchTuningParameters": {
"batchSize": 4
},
"generateExplanation": false,
"outputInfo": {
"bigqueryOutputDataset": "bq://12345.reg_model_2020_10_02_06_04
}
"state": "JOB_STATE_PENDING",
"createTime": "2020-09-30T02:58:44.341643Z",
"updateTime": "2020-09-30T02:58:44.341643Z",
}
Java
In the following sample, replace INSTANCES_FORMAT and
PREDICTIONS_FORMAT with bigquery. To learn how to replace the other
placeholders, see the REST & CMD LINE tab of this section.
Python
In the following sample, set the instances_format and predictions_format
parameters to "bigquery". To learn how to set the other parameters, see the
REST & CMD LINE tab of this section.
Feature importance
If you want feature importance values returned for your predictions, you must
use the v1beta1 version of the batchPredictionJobs.create
method, and
set the generateExplanation property to true. Note that forecasting models
don't support feature importance, so you cannot include it in your batch
prediction requests.
Feature importance, sometimes called feature attributions, is part of Vertex Explainable AI.
Feature importance is not supported when the prediction data is returned in Cloud Storage or for forecasting models.
Text
Select a text objective to view an example of a batch prediction request.
Classification
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
- PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/LOCATION/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/LOCATION/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
state is
JOB_STATE_SUCCEEDED.
Java
Node.js
Python
Entity extraction
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
- PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/LOCATION/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/LOCATION/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
state is
JOB_STATE_SUCCEEDED.
Java
Node.js
Python
Sentiment analysis
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For example, us-central1.
- PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/LOCATION/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/LOCATION/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
state is
JOB_STATE_SUCCEEDED.
Java
Node.js
Python
Video
Action recognition
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For
example,
us-central1. - PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- THRESHOLD_VALUE (optional): Model returns only predictions that have confidence scores with at least this value
- MAX_PREDICTIONS (optional): Model returns up to this many predictions with the highest confidence scores
- URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/us-central1/MODEL_ID",
"modelParameters": {
"confidenceThreshold": THRESHOLD_VALUE,
"maxPredictions": MAX_PREDICTIONS,
},
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT_NUMBER/locations/us-central1/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
state is
JOB_STATE_SUCCEEDED.
Java
Python
Classification
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For
example,
us-central1. - PROJECT: Your project ID or project number
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- THRESHOLD_VALUE (optional): Model returns only predictions that have confidence scores with at least this value
- MAX_PREDICTIONS (optional): Model returns up to this many predictions with the highest confidence scores
- SEGMENT_CLASSIFICATION (optional): A Boolean value that
determines whether to request segment-level classification. Vertex AI
returns labels and their confidence scores for the entire time segment of
the video that you specified in the input instance. The default is
true. - SHOT_CLASSIFICATION (optional): A Boolean value that determines
whether to request shot-level classification. Vertex AI determines
the boundaries for each camera shot in the entire time segment of the video
that you specified in the input instance. Vertex AI then returns
labels and their confidence scores for each detected shot, along with the
start and end time of the shot. The default is
false. - ONE_SEC_INTERVAL_CLASSIFICATION (optional): A Boolean value
that determines whether to request classification for a video at one-second
intervals. Vertex AI returns labels and their confidence scores for
each second of the entire time segment of the video that you
specified in the input instance. The default is
false. - URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/us-central1/MODEL_ID",
"modelParameters": {
"confidenceThreshold": THRESHOLD_VALUE,
"maxPredictions": MAX_PREDICTIONS,
"segmentClassification": SEGMENT_CLASSIFICATION,
"shotClassification": SHOT_CLASSIFICATION,
"oneSecIntervalClassification": ONE_SEC_INTERVAL_CLASSIFICATION
},
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT_NUMBER/locations/us-central1/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
state is
JOB_STATE_SUCCEEDED.
Java
Node.js
Python
Object tracking
REST & CMD LINE
Before using any of the request data below, make the following replacements:
- LOCATION: Region where Model is stored and batch prediction job is executed. For
example,
us-central1. - PROJECT: Your project ID or project number.
- BATCH_JOB_NAME: Display name for the batch job
- MODEL_ID: The ID for the model to use for making predictions
- THRESHOLD_VALUE (optional): Vertex AI returns only
predictions that have confidence scores with at least this value. The
default is
0.0. - MAX_PREDICTIONS (optional): Vertex AI returns up to this
many predictions starting with the predictions that have highest confidence
scores. The default is
10. - BOX_SIZE (optional): Vertex AI returns only bounding
boxes that are at least this long on the shortest edge. This is a normalized
value (between
0and1). The default is0.0. - URI: Cloud Storage URI where your input JSONL file is located.
- BUCKET: Your Cloud Storage bucket
- PROJECT_NUMBER: Project number for your project (appears in the response)
HTTP method and URL:
POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
Request JSON body:
{
"displayName": "BATCH_JOB_NAME",
"model": "projects/PROJECT/locations/us-central1/MODEL_ID",
"modelParameters": {
"confidenceThreshold": THRESHOLD_VALUE,
"maxPredictions": MAX_PREDICTIONS,
"minBoundingBoxSize": BOX_SIZE,
},
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": ["URI"],
},
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "OUTPUT_BUCKET",
},
},
}
To send your request, choose one of these options:
curl
Save the request body in a file called request.json,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs
PowerShell
Save the request body in a file called request.json,
and execute the following command:
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/batchPredictionJobs" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{
"name": "projects/PROJECT_NUMBER/locations/us-central1/batchPredictionJobs/BATCH_JOB_ID",
"displayName": "BATCH_JOB_NAME 202005291958",
"model": "projects/PROJECT_NUMBER/locations/us-central1/models/MODEL_ID",
"inputConfig": {
"instancesFormat": "jsonl",
"gcsSource": {
"uris": [
"CONTENT"
]
}
},
"outputConfig": {
"predictionsFormat": "jsonl",
"gcsDestination": {
"outputUriPrefix": "BUCKET"
}
},
"state": "JOB_STATE_PENDING",
"createTime": "2020-05-30T02:58:44.341643Z",
"updateTime": "2020-05-30T02:58:44.341643Z",
"modelDisplayName": "MODEL_NAME",
"modelObjective": "MODEL_OBJECTIVE"
}
state is
JOB_STATE_SUCCEEDED.
Java
Node.js
Python
Retrieving batch prediction results
Vertex AI sends batch prediction output to your specified destination.
Custom-trained
When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.
Image
When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.
Tabular
Retrieving your results depends on the destination that you specified for the batch prediction, either BigQuery or Cloud Storage.
Retrieving results in BigQuery
If you specified BigQuery as your output destination, the results of your batch prediction request are returned in a dataset in the BigQuery project you specified. If you didn't specify a BigQuery dataset, Vertex AI creates a new dataset. The name of the dataset is the name of your model prepended with "prediction_" and appended with the timestamp of when the prediction job started. You can find the BigQuery dataset name in the Batch predictions tab for your model.
The BigQuery dataset contains two tables, which are also
appended with the timestamp of when the prediction job started:
predictions_timestamp and
errors_timestampe. The errors table has a
row for every row in your prediction request for which Vertex AI
could not return a prediction (for example, if a non-nullable feature was
null). The predictions table contains a row for every prediction returned.
In the predictions table, Vertex AI returns your prediction data, and creates a new column for the prediction results by prepending "predicted_" onto your target column name. The prediction results column contains a nested BigQuery structure that contains the prediction results.
To retrieve the prediction results, you can use a query in the BigQuery console. The format of the query depends on your model type. The following sample queries show how to get results for each objective.
Classification:
SELECT predicted_<target-column-name>.classes AS classes, predicted_<target-column-name>.scores AS scores,
"classes" is the list of potential classes, and "scores" are the corresponding confidence scores.
Forecasting:
For models optimized for quantile loss, use the following query:
SELECT predicted_target-column-name.quantile_predictions, predicted_target-column-name.quantile_values, FROM bq-dataset-name.predictions_timestamp
For all other optimization, use the following query:
SELECT predicted_target-column-name.value, FROM bq-dataset-name.predictions_timestamp
Regression:
SELECT predicted_<target-column-name>.value, predicted_<target-column-name>.lower_bound, predicted_<target-column-name>.upper_bound FROM <bq-dataset-name>.predictions_timestamp
Retrieving results in Cloud Storage
If you specified Cloud Storage as your output destination, the results of your batch prediction request are returned as CSV files in a new folder in the bucket you specified. The name of the folder is the name of your model, prepended with "prediction-" and appended with the timestamp of when the prediction job started. You can find the Cloud Storage folder name in the Batch predictions tab for your model.
The Cloud Storage folder contains two types of files: error files and prediction files. If the results are large, additional files are created.
The error files are named errors_1.csv,
errors_2.csv, and so on. They contain a header row, and a row for
every row in your prediction request for which Vertex AI could
not return a prediction.
The prediction files are named predictions_1.csv,
predictions_2.csv, and so on. They contain a header row with the
column names, and a row for every prediction returned.
In the prediction files, Vertex AI returns your prediction data, and creates one or more new columns for the prediction results, depending on your model type:
Classification:
For each potential value of your target column, a column named
<target-column-name>_<value>_score is added to the
results. This column contains the score, or confidence estimate, for that
value.
Forecasting:
The predicted values are returned in a column named
predicted_<target-column-name>. For quantile forecasts, the
output column contains the quantile predictions and quantile values in the
JSON format.
Regression:
The predicted value for that row is returned in a column named
predicted_<target-column-name>. The prediction interval
is not returned for CSV output.
Text
When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.
Video
When a batch prediction task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your request.
Example batch prediction results
The following examples are batch prediction results for several AutoML model types and objectives. For information about the results, see Interpreting results from AutoML models.
Image
Classification
{
"instance": {"content": "gs://bucket/image.jpg", "mimeType": "image/jpeg"},
"prediction": {
"ids": [1, 2],
"displayNames": ["cat", "dog"],
"confidences": [0.7, 0.5]
}
}
Object detection
{
"instance": {"content": "gs://bucket/image.jpg", "mimeType": "image/jpeg"},
"prediction": {
"ids": [1, 2],
"displayNames": ["cat", "dog"],
"bboxes": [
[0.1, 0.2, 0.3, 0.4],
[0.2, 0.3, 0.4, 0.5]
],
"confidences": [0.7, 0.5]
}
}
Text
Classification
{
"instance": {"content": "gs://bucket/text.txt", "mimeType": "text/plain"},
"predictions": [
{
"ids": [
"1234567890123456789",
"2234567890123456789",
"3234567890123456789"
],
"displayNames": [
"GreatService",
"Suggestion",
"InfoRequest"
],
"confidences": [
0.8986392080783844,
0.81984345316886902,
0.7722353458404541
]
}
]
}
Entity extraction
{
"key": 1,
"predictions": {
"ids": [
"1234567890123456789",
"2234567890123456789",
"3234567890123456789"
],
"displayNames": [
"SpecificDisease",
"DiseaseClass",
"SpecificDisease"
],
"textSegmentStartOffsets": [13, 40, 57],
"textSegmentEndOffsets": [29, 51, 75],
"confidences": [
0.99959725141525269,
0.99912621492484128,
0.99935531616210938
]
}
}
Sentiment analysis
{
"instance": {"content": "gs://bucket/text.txt", "mimeType": "text/plain"},
"prediction": {"sentiment": 8}
}
Video
Action recognition
{
"instance": {
"content": "gs://bucket/video.mp4",
"mimeType": "video/mp4",
"timeSegmentStart": "1s",
"timeSegmentEnd": "5s"
}
"prediction": [{
"id": "1",
"displayName": "swing",
"timeSegmentStart": "1.2s",
"timeSegmentEnd": "1.2s",
"confidence": 0.7
}, {
"id": "2",
"displayName": "jump",
"timeSegmentStart": "3.4s",
"timeSegmentEnd": "3.4s",
"confidence": 0.5
}]
}
Classification
{
"instance": {
"content": "gs://bucket/video.mp4",
"mimeType": "video/mp4",
"timeSegmentStart": "1s",
"timeSegmentEnd": "5s"
}
"prediction": [{
"id": "1",
"displayName": "cat",
"type": "segment-classification",
"timeSegmentStart": "1s",
"timeSegmentEnd": "5s",
"confidence": 0.7
}, {
"id": "1",
"displayName": "cat",
"type": "shot-classification",
"timeSegmentStart": "1s",
"timeSegmentEnd": "4s",
"confidence": 0.9
}, {
"id": "2",
"displayName": "dog",
"type": "shot-classification",
"timeSegmentStart": "4s",
"timeSegmentEnd": "5s",
"confidence": 0.6
}, {
"id": "1",
"displayName": "cat",
"type": "one-sec-interval-classification",
"timeSegmentStart": "1s",
"timeSegmentEnd": "1s",
"confidence": 0.95
}, {
"id": "1",
"displayName": "cat",
"type": "one-sec-interval-classification",
"timeSegmentStart": "2s",
"timeSegmentEnd": "2s",
"confidence": 0.9
}, {
"id": "1",
"displayName": "cat",
"type": "one-sec-interval-classification",
"timeSegmentStart": "3s",
"timeSegmentEnd": "3s",
"confidence": 0.85
}, {
"id": "2",
"displayName": "dog",
"type": "one-sec-interval-classification",
"timeSegmentStart": "4s",
"timeSegmentEnd": "4s",
"confidence": 0.6
}]
}
Object tracking
{
"instance": {
"content": "gs://bucket/video.mp4",
"mimeType": "video/mp4",
"timeSegmentStart": "1s",
"timeSegmentEnd": "5s"
}
"prediction": [{
"id": "1",
"displayName": "cat",
"timeSegmentStart": "1.2s",
"timeSegmentEnd": "3.4s",
"frames": [{
"timeOffset": "1.2s",
"xMin": 0.1,
"xMax": 0.2,
"yMin": 0.3,
"yMax": 0.4
}, {
"timeOffset": "3.4s",
"xMin": 0.2,
"xMax": 0.3,
"yMin": 0.4,
"yMax": 0.5,
}],
"confidence": 0.7
}, {
"id": "1",
"displayName": "cat",
"timeSegmentStart": "4.8s",
"timeSegmentEnd": "4.8s",
"frames": [{
"timeOffset": "4.8s",
"xMin": 0.2,
"xMax": 0.3,
"yMin": 0.4,
"yMax": 0.5,
}],
"confidence": 0.6
}, {
"id": "2",
"displayName": "dog",
"timeSegmentStart": "1.2s",
"timeSegmentEnd": "3.4s",
"frames": [{
"timeOffset": "1.2s",
"xMin": 0.1,
"xMax": 0.2,
"yMin": 0.3,
"yMax": 0.4
}, {
"timeOffset": "3.4s",
"xMin": 0.2,
"xMax": 0.3,
"yMin": 0.4,
"yMax": 0.5,
}],
"confidence": 0.5
}]
}