Method: projects.locations.models.predict

Perform an online prediction. The prediction result is directly returned in the response. Available for following ML scenarios, and their expected request payloads:

AutoML Vision Classification An image in .JPEG, .GIF or .PNG format, imageBytes up to 30MB.
AutoML Vision Object Detection An image in .JPEG, .GIF or .PNG format, imageBytes up to 30MB.
AutoML Natural Language Classification A TextSnippet up to 60,000 characters, UTF-8 encoded or a document in .PDF, .TIF or .TIFF format with size upto 2MB.
AutoML Natural Language Entity Extraction A TextSnippet up to 10,000 characters, UTF-8 NFC encoded or a document in .PDF, .TIF or .TIFF format with size upto 20MB.
AutoML Natural Language Sentiment Analysis A TextSnippet up to 60,000 characters, UTF-8 encoded or a document in .PDF, .TIF or .TIFF format with size upto 2MB.
AutoML Translation A TextSnippet up to 25,000 characters, UTF-8 encoded.
AutoML Tables A row with column values matching the columns of the model, up to 5MB. Not available for FORECASTING `predictionType`.

HTTP request

POST https://automl.googleapis.com/v1/{name}:predict

Path parameters

Parameters
name

string

Name of the model requested to serve the prediction.

Authorization requires the following Google IAM permission on the specified resource name:

  • automl.models.predict

Request body

The request body contains data with the following structure:

JSON representation 
{
  "payload": {
    object (ExamplePayload)
  },
  "params": {
    string: string,
    ...
  }
}
Fields
payload

object (ExamplePayload)

Required. Payload to perform a prediction on. The payload must match the problem type that the model was trained to solve.
params

map (key: string, value: string)

Additional domain-specific parameters, any string must be up to 25000 characters long.

AutoML Vision Classification

score_threshold
(float) A value from 0.0 to 1.0. When the model makes predictions for an image, it will only produce results that have at least this confidence score. The default is 0.5.

AutoML Vision Object Detection

score_threshold
(float) When Model detects objects on the image, it will only produce bounding boxes which have at least this confidence score. Value in 0 to 1 range, default is 0.5.
max_bounding_box_count
(int64) The maximum number of bounding boxes returned. The default is 100. The number of returned bounding boxes might be limited by the server.

AutoML Tables

featureImportance
(boolean) Whether

[featureImportance][google.cloud.automl.v1.TablesModelColumnInfo.feature_importance] is populated in the returned list of [TablesAnnotation][google.cloud.automl.v1.TablesAnnotation] objects. The default is false.

Response body

If successful, the response body contains data with the following structure:

Response message for PredictionService.Predict.

JSON representation 
{
  "payload": [
    {
      object (AnnotationPayload)
    }
  ],
  "preprocessedInput": {
    object (ExamplePayload)
  },
  "metadata": {
    string: string,
    ...
  }
}
Fields
payload[]

object (AnnotationPayload)

Prediction result. AutoML Translation and AutoML Natural Language Sentiment Analysis return precisely one payload.
preprocessedInput

object (ExamplePayload)

The preprocessed example that AutoML actually makes prediction on. Empty if AutoML does not preprocess the input example.

For AutoML Natural Language (Classification, Entity Extraction, and Sentiment Analysis), if the input is a document, the recognized text is returned in the documentText property.
metadata

map (key: string, value: string)

Additional domain-specific prediction response metadata.

AutoML Vision Object Detection

max_bounding_box_count
(int64) The maximum number of bounding boxes to return per image.

AutoML Natural Language Sentiment Analysis

sentiment_score
(float, deprecated) A value between -1 and 1, -1 maps to least positive sentiment, while 1 maps to the most positive one and the higher the score, the more positive the sentiment in the document is. Yet these values are relative to the training data, so e.g. if all data was positive then -1 is also positive (though the least). sentiment_score is not the same as "score" and "magnitude" from Sentiment Analysis in the Natural Language API.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

ExamplePayload

Example data used for training or prediction.

JSON representation 
{

  // Union field payload can be only one of the following:
  "image": {
    object (Image)
  },
  "textSnippet": {
    object (TextSnippet)
  },
  "document": {
    object (Document)
  }
  // End of list of possible types for union field payload.
}
Fields
Union field payload. Required. The example data. payload can be only one of the following:
image

object (Image)

Example image.
textSnippet

object (TextSnippet)

Example text.
document

object (Document)

Example document.

Image

A representation of an image. Only images up to 30MB in size are supported.

JSON representation 
{
  "thumbnailUri": string,
  "imageBytes": string
}
Fields
thumbnailUri

string

Output only. HTTP URI to the thumbnail image.
imageBytes

string (bytes format)

Image content represented as a stream of bytes. Note: As with all bytes fields, protobuffers use a pure binary representation, whereas JSON representations use base64.

A base64-encoded string.

TextSnippet

A representation of a text snippet.

JSON representation 
{
  "content": string,
  "mimeType": string,
  "contentUri": string
}
Fields
content

string

Required. The content of the text snippet as a string. Up to 250000 characters long.
mimeType

string

Optional. The format of content. Currently the only two allowed values are "text/html" and "text/plain". If left blank, the format is automatically determined from the type of the uploaded content.
contentUri

string

Output only. HTTP URI where you can download the content.

Document

A structured text document e.g. a PDF.

JSON representation 
{
  "inputConfig": {
    object (DocumentInputConfig)
  },
  "documentText": {
    object (TextSnippet)
  },
  "layout": [
    {
      object (Layout)
    }
  ],
  "documentDimensions": {
    object (DocumentDimensions)
  },
  "pageCount": integer
}
Fields
inputConfig

object (DocumentInputConfig)

An input config specifying the content of the document.
documentText

object (TextSnippet)

The plain text version of this document.
layout[]

object (Layout)

Describes the layout of the document. Sorted by [pageNumber][].
documentDimensions

object (DocumentDimensions)

The dimensions of the page in the document.
pageCount

integer

Number of pages in the document.

DocumentInputConfig

Input configuration of a Document.

JSON representation 
{
  "gcsSource": {
    object (GcsSource)
  }
}
Fields
gcsSource

object (GcsSource)

The Google Cloud Storage location of the document file. Only a single path should be given.

Max supported size: 512MB.

Supported extensions: .PDF.

Layout

Describes the layout information of a textSegment in the document.

JSON representation 
{
  "textSegment": {
    object (TextSegment)
  },
  "pageNumber": integer,
  "boundingPoly": {
    object (BoundingPoly)
  },
  "textSegmentType": enum (TextSegmentType)
}
Fields
textSegment

object (TextSegment)

Text Segment that represents a segment in documentText.
pageNumber

integer

Page number of the textSegment in the original document, starts from 1.
boundingPoly

object (BoundingPoly)

The position of the textSegment in the page. Contains exactly 4

normalizedVertices and they are connected by edges in the order provided, which will represent a rectangle parallel to the frame. The NormalizedVertex-s are relative to the page. Coordinates are based on top-left as point (0,0).
textSegmentType

enum (TextSegmentType)

The type of the textSegment in document.

TextSegment

A contiguous part of a text (string), assuming it has an UTF-8 NFC encoding.

JSON representation 
{
  "content": string,
  "startOffset": string,
  "endOffset": string
}
Fields
content

string

Output only. The content of the TextSegment.
startOffset

string (int64 format)

Required. Zero-based character index of the first character of the text segment (counting characters from the beginning of the text).
endOffset

string (int64 format)

Required. Zero-based character index of the first character past the end of the text segment (counting character from the beginning of the text). The character at the endOffset is NOT included in the text segment.

BoundingPoly

A bounding polygon of a detected object on a plane. On output both vertices and normalizedVertices are provided. The polygon is formed by connecting vertices in the order they are listed.

JSON representation 
{
  "normalizedVertices": [
    {
      object (NormalizedVertex)
    }
  ]
}
Fields
normalizedVertices[]

object (NormalizedVertex)

Output only . The bounding polygon normalized vertices.

NormalizedVertex

A vertex represents a 2D point in the image. The normalized vertex coordinates are between 0 to 1 fractions relative to the original plane (image, video). E.g. if the plane (e.g. whole image) would have size 10 x 20 then a point with normalized coordinates (0.1, 0.3) would be at the position (1, 6) on that plane.

JSON representation 
{
  "x": number,
  "y": number
}
Fields
x

number

Required. Horizontal coordinate.
y

number

Required. Vertical coordinate.

TextSegmentType

The type of TextSegment in the context of the original document.

Enums
TEXT_SEGMENT_TYPE_UNSPECIFIED Should not be used.
TOKEN The text segment is a token. e.g. word.
PARAGRAPH The text segment is a paragraph.
FORM_FIELD The text segment is a form field.
FORM_FIELD_NAME The text segment is the name part of a form field. It will be treated as child of another FORM_FIELD TextSegment if its span is subspan of another TextSegment with type FORM_FIELD.
FORM_FIELD_CONTENTS The text segment is the text content part of a form field. It will be treated as child of another FORM_FIELD TextSegment if its span is subspan of another TextSegment with type FORM_FIELD.
TABLE The text segment is a whole table, including headers, and all rows.
TABLE_HEADER The text segment is a table's headers. It will be treated as child of another TABLE TextSegment if its span is subspan of another TextSegment with type TABLE.
TABLE_ROW The text segment is a row in table. It will be treated as child of another TABLE TextSegment if its span is subspan of another TextSegment with type TABLE.
TABLE_CELL The text segment is a cell in table. It will be treated as child of another TABLE_ROW TextSegment if its span is subspan of another TextSegment with type TABLE_ROW.

DocumentDimensions

Message that describes dimension of a document.

JSON representation 
{
  "unit": enum (DocumentDimensionUnit),
  "width": number,
  "height": number
}
Fields
unit

enum (DocumentDimensionUnit)

Unit of the dimension.
width

number

Width value of the document, works together with the unit.
height

number

Height value of the document, works together with the unit.

DocumentDimensionUnit

Unit of the document dimension.

Enums
DOCUMENT_DIMENSION_UNIT_UNSPECIFIED Should not be used.
INCH Document dimension is measured in inches.
CENTIMETER Document dimension is measured in centimeters.
POINT Document dimension is measured in points. 72 points = 1 inch.

AnnotationPayload

Contains annotation information that is relevant to AutoML.

JSON representation 
{
  "annotationSpecId": string,
  "displayName": string,

  // Union field detail can be only one of the following:
  "translation": {
    object (TranslationAnnotation)
  },
  "classification": {
    object (ClassificationAnnotation)
  },
  "imageObjectDetection": {
    object (ImageObjectDetectionAnnotation)
  },
  "textExtraction": {
    object (TextExtractionAnnotation)
  },
  "textSentiment": {
    object (TextSentimentAnnotation)
  }
  // End of list of possible types for union field detail.
}
Fields
annotationSpecId

string

Output only . The resource ID of the annotation spec that this annotation pertains to. The annotation spec comes from either an ancestor dataset, or the dataset that was used to train the model in use.
displayName

string

Output only. The value of displayName when the model was trained. Because this field returns a value at model training time, for different models trained using the same dataset, the returned value could be different as model owner could update the displayName between any two model training.
Union field detail. Output only . Additional information about the annotation specific to the AutoML domain. detail can be only one of the following:
translation

object (TranslationAnnotation)

Annotation details for translation.
classification

object (ClassificationAnnotation)

Annotation details for content or image classification.
imageObjectDetection

object (ImageObjectDetectionAnnotation)

Annotation details for image object detection.
textExtraction

object (TextExtractionAnnotation)

Annotation details for text extraction.
textSentiment

object (TextSentimentAnnotation)

Annotation details for text sentiment.

TranslationAnnotation

Annotation details specific to translation.

JSON representation 
{
  "translatedContent": {
    object (TextSnippet)
  }
}
Fields
translatedContent

object (TextSnippet)

Output only . The translated content.

ClassificationAnnotation

Contains annotation details specific to classification.

JSON representation 
{
  "score": number
}
Fields
score

number

Output only. A confidence estimate between 0.0 and 1.0. A higher value means greater confidence that the annotation is positive. If a user approves an annotation as negative or positive, the score value remains unchanged. If a user creates an annotation, the score is 0 for negative or 1 for positive.

ImageObjectDetectionAnnotation

Annotation details for image object detection.

JSON representation 
{
  "boundingBox": {
    object (BoundingPoly)
  },
  "score": number
}
Fields
boundingBox

object (BoundingPoly)

Output only. The rectangle representing the object location.
score

number

Output only. The confidence that this annotation is positive for the parent example, value in [0, 1], higher means higher positivity confidence.

TextExtractionAnnotation

Annotation for identifying spans of text.

JSON representation 
{
  "score": number,
  "textSegment": {
    object (TextSegment)
  }
}
Fields
score

number

Output only. A confidence estimate between 0.0 and 1.0. A higher value means greater confidence in correctness of the annotation.
textSegment

object (TextSegment)

An entity annotation will set this, which is the part of the original text to which the annotation pertains.

TextSentimentAnnotation

Contains annotation details specific to text sentiment.

JSON representation 
{
  "sentiment": integer
}
Fields
sentiment

integer

Output only. The sentiment with the semantic, as given to the AutoMl.ImportData when populating the dataset from which the model used for the prediction had been trained. The sentiment values are between 0 and Dataset.text_sentiment_dataset_metadata.sentiment_max (inclusive), with higher value meaning more positive sentiment. They are completely relative, i.e. 0 means least positive sentiment and sentimentMax means the most positive from the sentiments present in the train data. Therefore e.g. if train data had only negative sentiment, then sentimentMax, would be still negative (although least negative). The sentiment shouldn't be confused with "score" or "magnitude" from the previous Natural Language Sentiment Analysis API.