Enumerations

ClassificationType

constant   static

number

Type of the classification problem.

Value

CLASSIFICATION_TYPE_UNSPECIFIED

Should not be used, an un-set enum has this value by default.

MULTICLASS

At most one label is allowed per example.

MULTILABEL

Multiple labels are allowed for one example.

DeploymentState

static

number

Deployment state of the model.

Value

DEPLOYMENT_STATE_UNSPECIFIED

Should not be used, an un-set enum has this value by default.

DEPLOYED

Model is deployed.

UNDEPLOYED

Model is not deployed.

TypeCode

constant   static

number

TypeCode is used as a part of DataType.

Each legal value of a DataType can be encoded to or decoded from a JSON value, using the encodings listed below, and definitions of which can be found at

https: //developers.google.com/protocol-buffers // /docs/reference/google.protobuf#value.

Value

TYPE_CODE_UNSPECIFIED

Not specified. Should not be used.

FLOAT64

Encoded as number, or the strings "NaN", "Infinity", or "-Infinity".

TIMESTAMP

Must be between 0AD and 9999AD. Encoded as string according to time_format, or, if that format is not set, then in RFC 3339 date-time format, where time-offset = "Z" (e.g. 1985-04-12T23:20:50.52Z).

STRING

Encoded as string.

ARRAY

Encoded as list, where the list elements are represented according to

list_element_type.

STRUCT

Encoded as struct, where field values are represented according to struct_type.

CATEGORY

Values of this type are not further understood by AutoML, e.g. AutoML is unable to tell the order of values (as it could with FLOAT64), or is unable to say if one value contains another (as it could with STRING). Encoded as string (bytes should be base64-encoded, as described in RFC 4648, section 4).

Properties

ClassificationType

constant   static

number

Type of the classification problem.

Value

CLASSIFICATION_TYPE_UNSPECIFIED

Should not be used, an un-set enum has this value by default.

MULTICLASS

At most one label is allowed per example.

MULTILABEL

Multiple labels are allowed for one example.

DeploymentState

static

number

Deployment state of the model.

Value

DEPLOYMENT_STATE_UNSPECIFIED

Should not be used, an un-set enum has this value by default.

DEPLOYED

Model is deployed.

UNDEPLOYED

Model is not deployed.

TypeCode

constant   static

number

TypeCode is used as a part of DataType.

Each legal value of a DataType can be encoded to or decoded from a JSON value, using the encodings listed below, and definitions of which can be found at

https: //developers.google.com/protocol-buffers // /docs/reference/google.protobuf#value.

Value

TYPE_CODE_UNSPECIFIED

Not specified. Should not be used.

FLOAT64

Encoded as number, or the strings "NaN", "Infinity", or "-Infinity".

TIMESTAMP

Must be between 0AD and 9999AD. Encoded as string according to time_format, or, if that format is not set, then in RFC 3339 date-time format, where time-offset = "Z" (e.g. 1985-04-12T23:20:50.52Z).

STRING

Encoded as string.

ARRAY

Encoded as list, where the list elements are represented according to

list_element_type.

STRUCT

Encoded as struct, where field values are represented according to struct_type.

CATEGORY

Values of this type are not further understood by AutoML, e.g. AutoML is unable to tell the order of values (as it could with FLOAT64), or is unable to say if one value contains another (as it could with STRING). Encoded as string (bytes should be base64-encoded, as described in RFC 4648, section 4).

Abstract types

AnnotationPayload

static

Contains annotation information that is relevant to AutoML.

Properties

Parameter

translation

Object

Annotation details for translation.

This object should have the same structure as TranslationAnnotation

classification

Object

Annotation details for content or image classification.

This object should have the same structure as ClassificationAnnotation

imageObjectDetection

Object

Annotation details for image object detection.

This object should have the same structure as ImageObjectDetectionAnnotation

videoClassification

Object

Annotation details for video classification. Returned for Video Classification predictions.

This object should have the same structure as VideoClassificationAnnotation

textExtraction

Object

Annotation details for text extraction.

This object should have the same structure as TextExtractionAnnotation

textSentiment

Object

Annotation details for text sentiment.

This object should have the same structure as TextSentimentAnnotation

tables

Object

Annotation details for Tables.

This object should have the same structure as TablesAnnotation

annotationSpecId

string

Output only . The resource ID of the annotation spec that this annotation pertains to. The annotation spec comes from either an ancestor dataset, or the dataset that was used to train the model in use.

displayName

string

Output only. The value of AnnotationSpec.display_name when the model was trained. Because this field returns a value at model training time, for different models trained using the same dataset, the returned value could be different as model owner could update the display_name between any two model training.

See also

google.cloud.automl.v1beta1.AnnotationPayload definition in proto format

AnnotationSpec

static

A definition of an annotation.

Properties

Parameter

name

string

Output only. Resource name of the annotation spec. Form:

'projects/{project_id}/locations/{location_id}/datasets/{dataset_id}/annotationSpecs/{annotation_spec_id}'

displayName

string

Required. The name of the annotation spec to show in the interface. The name can be up to 32 characters long and can consist only of ASCII Latin letters A-Z and a-z, underscores (_), and ASCII digits 0-9.

exampleCount

number

Output only. The number of examples in the parent dataset labeled by the annotation spec.

See also

google.cloud.automl.v1beta1.AnnotationSpec definition in proto format

ArrayStats

static

The data statistics of a series of ARRAY values.

Property

Parameter

memberStats

Object

Stats of all the values of all arrays, as if they were a single long series of data. The type depends on the element type of the array.

This object should have the same structure as DataStats

See also

google.cloud.automl.v1beta1.ArrayStats definition in proto format

BatchPredictInputConfig

static

Input configuration for BatchPredict Action.

The format of input depends on the ML problem of the model used for prediction. As input source the gcs_source is expected, unless specified otherwise.

The formats are represented in EBNF with commas being literal and with non-terminal symbols defined near the end of this comment. The formats are:

  • For Video Classification:

    CSV file(s) with each line in format:
      GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END
      GCS_FILE_PATH leads to video of up to 50GB in size and up to 3h
      duration. Supported extensions: .MOV, .MPEG4, .MP4, .AVI.
      TIME_SEGMENT_START and TIME_SEGMENT_END must be within the
      length of the video, and end has to be after the start.
    Three sample rows:
      gs://folder/video1.mp4,10,40
      gs://folder/video1.mp4,20,60
      gs://folder/vid2.mov,0,inf
  • For Text Extraction

     .JSONL (i.e. JSON Lines) file(s) which either provide text in-line or
     as documents (for a single BatchPredict call only one of the these
     formats may be used).
     The in-line .JSONL file(s) contain per line a proto that
       wraps a temporary user-assigned TextSnippet ID (string up to 2000
       characters long) called "id" followed by a TextSnippet proto (in
       json representation). Any given text snippet content must have
       30,000 characters or less, and also be UTF-8 NFC encoded (ASCII
       already is). The IDs provided should be unique.
     The document .JSONL file(s) contain, per line, a proto that wraps a
       Document proto with input_config set. Only PDF documents are
       supported now, and each document must be up to 2MB large.
     Any given .JSONL file must be 100MB or smaller, and no more than 20
     files may be given.
     Sample in-line JSON Lines file (presented here with artificial line
     breaks, but the only actual line break is denoted by \n):
       {
         "id": "my_first_id",
         "text_snippet": { "content": "dog car cat"}
       }\n
       {
         "id": "2",
         "text_snippet": {
           "content": "An elaborate content",
           "mime_type": "text/plain"
         }
       }
     Sample document JSON Lines file (presented here with artificial line
     breaks, but the only actual line break is denoted by \n).:
       {
         "document": {
           "input_config": {
             "gcs_source": { "input_uris": [ "gs://folder/document1.pdf" ]
             }
           }
         }
       }\n
       {
         "document": {
           "input_config": {
             "gcs_source": { "input_uris": [ "gs://folder/document2.pdf" ]
             }
           }
         }
       }
  • For Tables:

    Either
    gcs_source or

bigquery_source. For gcs_source: CSV file(s), where first file must have a header containing column names, other files may have such header line too, and all other lines contain values for the header columns. The column names must be exactly same (order may differ) as the model's

input_feature_column_specs' display_names, with values compatible with these column specs data types. Prediction on all the rows, i.e. the CSV lines, will be attempted. Each line must have 1,000,000 or fewer characters. First three sample rows of a CSV file: "First Name","Last Name","Dob","Addresses"

"John","Doe","1968-01-22","[{"status":"current","address":"123_First_Avenue","city":"Seattle","state":"WA","zip":"11111","numberOfYears":"1"},{"status":"previous","address":"456_Main_Street","city":"Portland","state":"OR","zip":"22222","numberOfYears":"5"}]"

"Jane","Doe","1980-10-16","[{"status":"current","address":"789_Any_Avenue","city":"Albany","state":"NY","zip":"33333","numberOfYears":"2"},{"status":"previous","address":"321_Main_Street","city":"Hoboken","state":"NJ","zip":"44444","numberOfYears":"3"}]} For bigquery_source: An URI of a BigQuery table. The table's columns must be exactly same (order may differ) as all model's

input_feature_column_specs' display_names, with data compatible with these colum specs data types. Prediction on all the rows of the table will be attempted.

Definitions: GCS_FILE_PATH = A path to file on GCS, e.g. "gs://folder/video.avi". TIME_SEGMENT_START = TIME_OFFSET Expresses a beginning, inclusive, of a time segment within an example that has a time dimension (e.g. video). TIME_SEGMENT_END = TIME_OFFSET Expresses an end, exclusive, of a time segment within an example that has a time dimension (e.g. video). TIME_OFFSET = A number of seconds as measured from the start of an example (e.g. video). Fractions are allowed, up to a microsecond precision. "inf" is allowed and it means the end of the example.

Errors: If any of the provided CSV files can't be parsed or if more than certain percent of CSV rows cannot be processed then the operation fails and prediction does not happen. Regardless of overall success or failure the per-row failures, up to a certain count cap, will be listed in Operation.metadata.partial_failures.

Properties

Parameter

gcsSource

Object

The Google Cloud Storage location for the input content.

This object should have the same structure as GcsSource

bigquerySource

Object

The BigQuery location for the input content.

This object should have the same structure as BigQuerySource

See also

google.cloud.automl.v1beta1.BatchPredictInputConfig definition in proto format

BatchPredictOutputConfig

static

Output configuration for BatchPredict Action.

As destination the

gcs_destination must be set unless specified otherwise for a domain. If gcs_destination is set then in the given directory a new directory will be created. Its name will be "prediction--", where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. The contents of it depend on the ML problem the predictions are made for.

  • For Video Classification:

    In the created directory a video_classification.csv file, and a .JSON
    file per each video classification requested in the input (i.e. each
    line in given CSV(s)), will be created.
    
    The format of video_classification.csv is:

GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END,JSON_FILE_NAME,STATUS where: GCS_FILE_PATH,TIME_SEGMENT_START,TIME_SEGMENT_END = matches 1 to 1 the prediction input lines (i.e. video_classification.csv has precisely the same number of lines as the prediction input had.) JSON_FILE_NAME = Name of .JSON file in the output directory, which contains prediction responses for the video time segment. STATUS = "OK" if prediction completed successfully, or an error code and,or message otherwise. If STATUS is not "OK" then the .JSON file for that line may not exist or be empty.

    Each .JSON file, assuming STATUS is "OK", will contain a list of
    AnnotationPayload protos in JSON format, which are the predictions
    for the video time segment the file is assigned to in the
    video_classification.csv. All AnnotationPayload protos will have
    video_classification field set, and will be sorted by
    video_classification.type field (note that the returned types are
    governed by classifaction_types parameter in
    PredictService.BatchPredictRequest.params).
  • For Text Extraction: In the created directory files text_extraction_1.jsonl, text_extraction_2.jsonl,...,text_extraction_N.jsonl will be created, where N may be 1, and depends on the total number of inputs and annotations found. The contents of these .JSONL file(s) depend on whether the input used inline text, or documents. If input was inline, then each .JSONL file will contain, per line,
     a JSON representation of a proto that wraps given in request text
     snippet's "id" : "<id_value>" followed by a list of zero or more
     AnnotationPayload protos (called annotations), which have
     text_extraction detail populated. A single text snippet will be
     listed only once with all its annotations, and its annotations will
     never be split across files.
    If input used documents, then each .JSONL file will contain, per
     line, a JSON representation of a proto that wraps given in request
     document proto, followed by its OCR-ed representation in the form
     of a text snippet, finally followed by a list of zero or more
     AnnotationPayload protos (called annotations), which have
     text_extraction detail populated and refer, via their indices, to
     the OCR-ed text snippet. A single document (and its text snippet)
     will be listed only once with all its annotations, and its
     annotations will never be split across files.
    If prediction for any text snippet failed (partially or completely), then additional errors_1.jsonl, errors_2.jsonl,..., errors_N.jsonl files will be created (N depends on total number of failed predictions). These files will have a JSON representation of a proto that wraps either the "id" : "<id_value>" (in case of inline) or the document proto (in case of document) but here followed by exactly one

google.rpc.Status containing only code and message.

  • For Tables:
    Output depends on whether

gcs_destination or

bigquery_destination is set (either is allowed). GCS case: In the created directory files tables_1.csv, tables_2.csv,..., tables_N.csv will be created, where N may be 1, and depends on the total number of the successfully predicted rows. For the classification models: Each .csv file will contain a header, listing all model's

input_feature_column_specs'

display_names followed by M target column names in the format of

"<target_column_specs

display_name>__score" where M is the number of distinct target values, i.e. number of distinct values in the target column of the table used to train the model. Subsequent lines will contain the respective values of successfully predicted rows, with the last, i.e. the target, columns having the corresponding prediction scores. For the regression models: Each .csv file will contain a header, listing all model's

input_feature_column_specs display_names followed by the target column with name equal to

target_column_specs'

display_name. Subsequent lines will contain the respective values of successfully predicted rows, with the last, i.e. the target, column having the predicted target value. If prediction for any rows failed, then an additional errors_1.csv, errors_2.csv,..., errors_N.csv will be created (N depends on total number of failed rows). These files will have analogous format as tables_*.csv, but always with a single target column having

google.rpc.Status represented as a JSON string, and containing only code and message. BigQuery case:

bigquery_destination pointing to a BigQuery project must be set. In the given project a new dataset will be created with name prediction_<model-display-name>_<timestamp-of-prediction-call> where will be made BigQuery-dataset-name compatible (e.g. most special characters will become underscores), and timestamp will be in YYYY_MM_DDThh_mm_ss_sssZ "based on ISO-8601" format. In the dataset two tables will be created, predictions, and errors. The predictions table's column names will be the

input_feature_column_specs'

display_names followed by model's

target_column_specs'

display_name. The input feature columns will contain the respective values of successfully predicted rows, with the target column having an ARRAY of

AnnotationPayloads, represented as STRUCT-s, containing TablesAnnotation. The errors table contains rows for which the prediction has failed, it has analogous input feature and target columns, but here the target column as a value has

google.rpc.Status represented as a STRUCT, and containing only code and message.

Properties

Parameter

gcsDestination

Object

The Google Cloud Storage location of the directory where the output is to be written to.

This object should have the same structure as GcsDestination

bigqueryDestination

Object

The BigQuery location where the output is to be written to.

This object should have the same structure as BigQueryDestination

See also

google.cloud.automl.v1beta1.BatchPredictOutputConfig definition in proto format

BatchPredictRequest

static

Request message for PredictionService.BatchPredict.

Properties

Parameter

name

string

Name of the model requested to serve the batch prediction.

inputConfig

Object

Required. The input configuration for batch prediction.

This object should have the same structure as BatchPredictInputConfig

outputConfig

Object

Required. The Configuration specifying where output predictions should be written.

This object should have the same structure as BatchPredictOutputConfig

params

Object with string properties

Additional domain-specific parameters for the predictions, any string must be up to 25000 characters long.

  • For Video Classification : score_threshold - (float) A value from 0.0 to 1.0. When the model
    makes predictions for a video, it will only produce results that
    have at least this confidence score. The default is 0.5.
    segment_classification - (boolean) Set to true to request
    segment-level classification. AutoML Video Intelligence returns
    labels and their confidence scores for the entire segment of the
    video that user specified in the request configuration.
    The default is "true".
    shot_classification - (boolean) Set to true to request shot-level
    classification. AutoML Video Intelligence determines the boundaries
    for each camera shot in the entire segment of the video that user
    specified in the request configuration. AutoML Video Intelligence
    then returns labels and their confidence scores for each detected
    shot, along with the start and end time of the shot.
    WARNING: Model evaluation is not done for this classification type,
    the quality of it depends on training data, but there are no metrics
    provided to describe that quality. The default is "false".
    1s_interval_classification - (boolean) Set to true to request
    classification for a video at one-second intervals. AutoML Video
    Intelligence returns labels and their confidence scores for each
    second of the entire segment of the video that user specified in the
    request configuration.
    WARNING: Model evaluation is not done for this classification
    type, the quality of it depends on training data, but there are no
    metrics provided to describe that quality. The default is
    "false".
See also

google.cloud.automl.v1beta1.BatchPredictRequest definition in proto format

BatchPredictResult

static

Batch predict result.

See also

google.cloud.automl.v1beta1.BatchPredictResult definition in proto format

BigQueryDestination

static

The BigQuery location for the output content.

Property

Parameter

outputUri

string

Required. BigQuery URI to a project, up to 2000 characters long. Accepted forms:

  • BigQuery path e.g. bq://projectId
See also

google.cloud.automl.v1beta1.BigQueryDestination definition in proto format

BigQuerySource

static

The BigQuery location for the input content.

Property

Parameter

inputUri

string

Required. BigQuery URI to a table, up to 2000 characters long. Accepted forms:

  • BigQuery path e.g. bq://projectId.bqDatasetId.bqTableId
See also

google.cloud.automl.v1beta1.BigQuerySource definition in proto format

BoundingBoxMetricsEntry

static

Bounding box matching model metrics for a single intersection-over-union threshold and multiple label match confidence thresholds.

Properties

Parameter

iouThreshold

number

Output only. The intersection-over-union threshold value used to compute this metrics entry.

meanAveragePrecision

number

Output only. The mean average precision, most often close to au_prc.

confidenceMetricsEntries

Array of Object

Output only. Metrics for each label-match confidence_threshold from 0.05,0.10,...,0.95,0.96,0.97,0.98,0.99. Precision-recall curve is derived from them.

This object should have the same structure as ConfidenceMetricsEntry

See also

google.cloud.automl.v1beta1.BoundingBoxMetricsEntry definition in proto format

BoundingPoly

static

A bounding polygon of a detected object on a plane. On output both vertices and normalized_vertices are provided. The polygon is formed by connecting vertices in the order they are listed.

Property

Parameter

normalizedVertices

Array of Object

Output only . The bounding polygon normalized vertices.

This object should have the same structure as NormalizedVertex

See also

google.cloud.automl.v1beta1.BoundingPoly definition in proto format

CategoryStats

static

The data statistics of a series of CATEGORY values.

Property

Parameter

topCategoryStats

Array of Object

The statistics of the top 20 CATEGORY values, ordered by

count.

This object should have the same structure as SingleCategoryStats

See also

google.cloud.automl.v1beta1.CategoryStats definition in proto format

ClassificationAnnotation

static

Contains annotation details specific to classification.

Property

Parameter

score

number

Output only. A confidence estimate between 0.0 and 1.0. A higher value means greater confidence that the annotation is positive. If a user approves an annotation as negative or positive, the score value remains unchanged. If a user creates an annotation, the score is 0 for negative or 1 for positive.

See also

google.cloud.automl.v1beta1.ClassificationAnnotation definition in proto format

ClassificationEvaluationMetrics

static

Model evaluation metrics for classification problems. Note: For Video Classification this metrics only describe quality of the Video Classification predictions of "segment_classification" type.

Properties

Parameter

auPrc

number

Output only. The Area Under Precision-Recall Curve metric. Micro-averaged for the overall evaluation.

baseAuPrc

number

Output only. The Area Under Precision-Recall Curve metric based on priors. Micro-averaged for the overall evaluation. Deprecated.

auRoc

number

Output only. The Area Under Receiver Operating Characteristic curve metric. Micro-averaged for the overall evaluation.

logLoss

number

Output only. The Log Loss metric.

confidenceMetricsEntry

Array of Object

Output only. Metrics for each confidence_threshold in 0.00,0.05,0.10,...,0.95,0.96,0.97,0.98,0.99 and position_threshold = INT32_MAX_VALUE. Precision-recall curve is derived from them. The above metrics may also be supplied for additional values of position_threshold.

This object should have the same structure as ConfidenceMetricsEntry

confusionMatrix

Object

Output only. Confusion matrix of the evaluation. Only set for MULTICLASS classification problems where number of labels is no more than 10. Only set for model level evaluation, not for evaluation per label.

This object should have the same structure as ConfusionMatrix

annotationSpecId

Array of string

Output only. The annotation spec ids used for this evaluation.

See also

google.cloud.automl.v1beta1.ClassificationEvaluationMetrics definition in proto format

ColumnSpec

static

A representation of a column in a relational table. When listing them, column specs are returned in the same order in which they were given on import . Used by:

  • Tables

Properties

Parameter

name

string

Output only. The resource name of the column specs. Form:

projects/{project_id}/locations/{location_id}/datasets/{dataset_id}/tableSpecs/{table_spec_id}/columnSpecs/{column_spec_id}

dataType

Object

The data type of elements stored in the column.

This object should have the same structure as DataType

displayName

string

Output only. The name of the column to show in the interface. The name can be up to 100 characters long and can consist only of ASCII Latin letters A-Z and a-z, ASCII digits 0-9, underscores(_), and forward slashes(/), and must start with a letter or a digit.

dataStats

Object

Output only. Stats of the series of values in the column. This field may be stale, see the ancestor's Dataset.tables_dataset_metadata.stats_update_time field for the timestamp at which these stats were last updated.

This object should have the same structure as DataStats

topCorrelatedColumns

Array of Object

Output only. Top 10 most correlated with this column columns of the table, ordered by cramers_v metric. This field may be stale, see the ancestor's Dataset.tables_dataset_metadata.stats_update_time field for the timestamp at which these stats were last updated.

This object should have the same structure as CorrelatedColumn

etag

string

Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.

See also

google.cloud.automl.v1beta1.ColumnSpec definition in proto format

ConfidenceMetricsEntry

static

Metrics for a single confidence threshold.

Properties

Parameter

confidenceThreshold

number

Output only. The confidence threshold value used to compute the metrics. Only annotations with score of at least this threshold are considered to be ones the model would return.

recall

number

Output only. Recall under the given confidence threshold.

precision

number

Output only. Precision under the given confidence threshold.

f1Score

number

Output only. The harmonic mean of recall and precision.

See also

google.cloud.automl.v1beta1.TextExtractionEvaluationMetrics.ConfidenceMetricsEntry definition in proto format

ConfidenceMetricsEntry

static

Metrics for a single confidence threshold.

Properties

Parameter

confidenceThreshold

number

Output only. Metrics are computed with an assumption that the model never returns predictions with score lower than this value.

positionThreshold

number

Output only. Metrics are computed with an assumption that the model always returns at most this many predictions (ordered by their score, descendingly), but they all still need to meet the confidence_threshold.

recall

number

Output only. Recall (True Positive Rate) for the given confidence threshold.

precision

number

Output only. Precision for the given confidence threshold.

falsePositiveRate

number

Output only. False Positive Rate for the given confidence threshold.

f1Score

number

Output only. The harmonic mean of recall and precision.

recallAt1

number

Output only. The Recall (True Positive Rate) when only considering the label that has the highest prediction score and not below the confidence threshold for each example.

precisionAt1

number

Output only. The precision when only considering the label that has the highest prediction score and not below the confidence threshold for each example.

falsePositiveRateAt1

number

Output only. The False Positive Rate when only considering the label that has the highest prediction score and not below the confidence threshold for each example.

f1ScoreAt1

number

Output only. The harmonic mean of recall_at1 and precision_at1.

truePositiveCount

number

Output only. The number of model created labels that match a ground truth label.

falsePositiveCount

number

Output only. The number of model created labels that do not match a ground truth label.

falseNegativeCount

number

Output only. The number of ground truth labels that are not matched by a model created label.

trueNegativeCount

number

Output only. The number of labels that were not created by the model, but if they would, they would not match a ground truth label.

See also

google.cloud.automl.v1beta1.ClassificationEvaluationMetrics.ConfidenceMetricsEntry definition in proto format

ConfidenceMetricsEntry

static

Metrics for a single confidence threshold.

Properties

Parameter

confidenceThreshold

number

Output only. The confidence threshold value used to compute the metrics.

recall

number

Output only. Recall under the given confidence threshold.

precision

number

Output only. Precision under the given confidence threshold.

f1Score

number

Output only. The harmonic mean of recall and precision.

See also

google.cloud.automl.v1beta1.BoundingBoxMetricsEntry.ConfidenceMetricsEntry definition in proto format

ConfusionMatrix

static

Confusion matrix of the model running the classification.

Properties

Parameter

annotationSpecId

Array of string

Output only. IDs of the annotation specs used in the confusion matrix.

row

Array of Object

Output only. Rows in the confusion matrix. The number of rows is equal to the size of annotation_spec_id. row[i].value[j] is the number of examples that have ground truth of the annotation_spec_id[i] and are predicted as annotation_spec_id[j] by the model being evaluated.

This object should have the same structure as Row

See also

google.cloud.automl.v1beta1.ClassificationEvaluationMetrics.ConfusionMatrix definition in proto format

CorrelatedColumn

static

Identifies the table's column, and its correlation with the column this ColumnSpec describes.

Properties

Parameter

columnSpecId

string

The column_spec_id of the correlated column, which belongs to the same table as the in-context column.

correlationStats

Object

Correlation between this and the in-context column.

This object should have the same structure as CorrelationStats

See also

google.cloud.automl.v1beta1.ColumnSpec.CorrelatedColumn definition in proto format

CorrelationStats

static

A correlation statistics between two series of DataType values. The series may have differing DataType-s, but within a single series the DataType must be the same.

Property

Parameter

cramersV

number

The correlation value using the Cramer's V measure.

See also

google.cloud.automl.v1beta1.CorrelationStats definition in proto format

CreateDatasetRequest

static

Request message for AutoMl.CreateDataset.

Properties

Parameter

parent

string

The resource name of the project to create the dataset for.

dataset

Object

The dataset to create.

This object should have the same structure as Dataset

See also

google.cloud.automl.v1beta1.CreateDatasetRequest definition in proto format

CreateModelRequest

static

Request message for AutoMl.CreateModel.

Properties

Parameter

parent

string

Resource name of the parent project where the model is being created.

model

Object

The model to create.

This object should have the same structure as Model

See also

google.cloud.automl.v1beta1.CreateModelRequest definition in proto format

Dataset

static

A workspace for solving a single, particular machine learning (ML) problem. A workspace contains examples that may be annotated.

Properties

Parameter

translationDatasetMetadata

Object

Metadata for a dataset used for translation.

This object should have the same structure as TranslationDatasetMetadata

imageClassificationDatasetMetadata

Object

Metadata for a dataset used for image classification.

This object should have the same structure as ImageClassificationDatasetMetadata

textClassificationDatasetMetadata

Object

Metadata for a dataset used for text classification.

This object should have the same structure as TextClassificationDatasetMetadata

imageObjectDetectionDatasetMetadata

Object

Metadata for a dataset used for image object detection.

This object should have the same structure as ImageObjectDetectionDatasetMetadata

videoClassificationDatasetMetadata

Object

Metadata for a dataset used for video classification.

This object should have the same structure as VideoClassificationDatasetMetadata

textExtractionDatasetMetadata

Object

Metadata for a dataset used for text extraction.

This object should have the same structure as TextExtractionDatasetMetadata

textSentimentDatasetMetadata

Object

Metadata for a dataset used for text sentiment.

This object should have the same structure as TextSentimentDatasetMetadata

tablesDatasetMetadata

Object

Metadata for a dataset used for Tables.

This object should have the same structure as TablesDatasetMetadata

name

string

Output only. The resource name of the dataset. Form: projects/{project_id}/locations/{location_id}/datasets/{dataset_id}

displayName

string

Required. The name of the dataset to show in the interface. The name can be up to 32 characters long and can consist only of ASCII Latin letters A-Z and a-z, underscores (_), and ASCII digits 0-9.

description

string

User-provided description of the dataset. The description can be up to 25000 characters long.

exampleCount

number

Output only. The number of examples in the dataset.

createTime

Object

Output only. Timestamp when this dataset was created.

This object should have the same structure as Timestamp

etag

string

Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.

See also

google.cloud.automl.v1beta1.Dataset definition in proto format

DataStats

static

The data statistics of a series of values that share the same DataType.

Properties

Parameter

float64Stats

Object

The statistics for FLOAT64 DataType.

This object should have the same structure as Float64Stats

stringStats

Object

The statistics for STRING DataType.

This object should have the same structure as StringStats

timestampStats

Object

The statistics for TIMESTAMP DataType.

This object should have the same structure as TimestampStats

arrayStats

Object

The statistics for ARRAY DataType.

This object should have the same structure as ArrayStats

structStats

Object

The statistics for STRUCT DataType.

This object should have the same structure as StructStats

categoryStats

Object

The statistics for CATEGORY DataType.

This object should have the same structure as CategoryStats

distinctValueCount

number

The number of distinct values.

nullValueCount

number

The number of values that are null.

See also

google.cloud.automl.v1beta1.DataStats definition in proto format

DataType

static

Indicated the type of data that can be stored in a structured data entity (e.g. a table).

Properties

Parameter

listElementType

Object

If type_code == ARRAY, then list_element_type is the type of the elements.

This object should have the same structure as DataType

structType

Object

If type_code == STRUCT, then struct_type provides type information for the struct's fields.

This object should have the same structure as StructType

timeFormat

string

If type_code == TIMESTAMP then time_format provides the format in which that time field is expressed. The time_format must either be one of:

  • UNIX_SECONDS
  • UNIX_MILLISECONDS
  • UNIX_MICROSECONDS
  • UNIX_NANOSECONDS (for respectively number of seconds, milliseconds, microseconds and nanoseconds since start of the Unix epoch); or be written in strftime syntax. If time_format is not set, then the default format as described on the type_code is used.

typeCode

number

Required. The TypeCode for this type.

The number should be among the values of TypeCode

nullable

boolean

If true, this DataType can also be null.

See also

google.cloud.automl.v1beta1.DataType definition in proto format

DeleteDatasetRequest

static

Request message for AutoMl.DeleteDataset.

Property

Parameter

name

string

The resource name of the dataset to delete.

See also

google.cloud.automl.v1beta1.DeleteDatasetRequest definition in proto format

DeleteModelRequest

static

Request message for AutoMl.DeleteModel.

Property

Parameter

name

string

Resource name of the model being deleted.

See also

google.cloud.automl.v1beta1.DeleteModelRequest definition in proto format

DeployModelRequest

static

Request message for AutoMl.DeployModel.

Properties

Parameter

imageObjectDetectionModelDeploymentMetadata

Object

Model deployment metadata specific to Image Object Detection.

This object should have the same structure as ImageObjectDetectionModelDeploymentMetadata

name

string

Resource name of the model to deploy.

See also

google.cloud.automl.v1beta1.DeployModelRequest definition in proto format

Document

static

A structured text document e.g. a PDF.

Property

Parameter

inputConfig

Object

An input config specifying the content of the document.

This object should have the same structure as DocumentInputConfig

See also

google.cloud.automl.v1beta1.Document definition in proto format

DocumentInputConfig

static

Input configuration of a Document.

Property

Parameter

gcsSource

Object

The Google Cloud Storage location of the document file. Only a single path should be given. Max supported size: 512MB. Supported extensions: .PDF.

This object should have the same structure as GcsSource

See also

google.cloud.automl.v1beta1.DocumentInputConfig definition in proto format

DoubleRange

static

A range between two double numbers.

Properties

Parameter

start

number

Start of the range, inclusive.

end

number

End of the range, exclusive.

See also

google.cloud.automl.v1beta1.DoubleRange definition in proto format

ExamplePayload

static

Example data used for training or prediction.

Properties

Parameter

image

Object

Example image.

This object should have the same structure as Image

textSnippet

Object

Example text.

This object should have the same structure as TextSnippet

document

Object

Example document.

This object should have the same structure as Document

row

Object

Example relational table row.

This object should have the same structure as Row

See also

google.cloud.automl.v1beta1.ExamplePayload definition in proto format

ExportDataRequest

static

Request message for AutoMl.ExportData.

Properties

Parameter

name

string

Required. The resource name of the dataset.

outputConfig

Object

Required. The desired output location.

This object should have the same structure as OutputConfig

See also

google.cloud.automl.v1beta1.ExportDataRequest definition in proto format

ExportEvaluatedExamplesOutputConfig

static

Output configuration for ExportEvaluatedExamples Action. Note that this call is available only for 30 days since the moment the model was evaluated. The output depends on the domain, as follows (note that only examples from the TEST set are exported):

  • For Tables:

bigquery_destination pointing to a BigQuery project must be set. In the given project a new dataset will be created with name

export_evaluated_examples_<model-display-name>_<timestamp-of-export-call> where will be made BigQuery-dataset-name compatible (e.g. most special characters will become underscores), and timestamp will be in YYYY_MM_DDThh_mm_ss_sssZ "based on ISO-8601" format. In the dataset an evaluated_examples table will be created. It will have all the same columns as the primary

table of the dataset from which the model was created, as they were at the moment of model's evaluation (this includes the target column with its ground truth), followed by a column called "predicted_<target_column>". That last column will contain the model's prediction result for each respective row, given as ARRAY of AnnotationPayloads, represented as STRUCT-s, containing TablesAnnotation.

Property

Parameter

bigqueryDestination

Object

The BigQuery location where the output is to be written to.

This object should have the same structure as BigQueryDestination

See also

google.cloud.automl.v1beta1.ExportEvaluatedExamplesOutputConfig definition in proto format

ExportEvaluatedExamplesRequest

static

Request message for AutoMl.ExportEvaluatedExamples.

Properties

Parameter

name

string

Required. The resource name of the model whose evaluated examples are to be exported.

outputConfig

Object

Required. The desired output location and configuration.

This object should have the same structure as ExportEvaluatedExamplesOutputConfig

See also

google.cloud.automl.v1beta1.ExportEvaluatedExamplesRequest definition in proto format

ExportModelRequest

static

Request message for AutoMl.ExportModel. Models need to be enabled for exporting, otherwise an error code will be returned.

Properties

Parameter

name

string

Required. The resource name of the model to export.

outputConfig

Object

Required. The desired output location and configuration.

This object should have the same structure as ModelExportOutputConfig

See also

google.cloud.automl.v1beta1.ExportModelRequest definition in proto format

Float64Stats

static

The data statistics of a series of FLOAT64 values.

Properties

Parameter

mean

number

The mean of the series.

standardDeviation

number

The standard deviation of the series.

quantiles

Array of number

Ordered from 0 to k k-quantile values of the data series of n values. The value at index i is, approximately, the i*n/k-th smallest value in the series; for i = 0 and i = k these are, respectively, the min and max values.

histogramBuckets

Array of Object

Histogram buckets of the data series. Sorted by the min value of the bucket, ascendingly, and the number of the buckets is dynamically generated. The buckets are non-overlapping and completely cover whole FLOAT64 range with min of first bucket being "-Infinity", and max of the last one being "Infinity".

This object should have the same structure as HistogramBucket

See also

google.cloud.automl.v1beta1.Float64Stats definition in proto format

GcrDestination

static

The GCR location where the image must be pushed to.

Property

Parameter

outputUri

string

Required. Google Contained Registry URI of the new image, up to 2000 characters long. See

https: //cloud.google.com/container-registry/do // cs/pushing-and-pulling#pushing_an_image_to_a_registry Accepted forms:

  • [HOSTNAME]/[PROJECT-ID]/[IMAGE]
  • [HOSTNAME]/[PROJECT-ID]/[IMAGE]:[TAG]

    The requesting user must have permission to push images the project.

See also

google.cloud.automl.v1beta1.GcrDestination definition in proto format

GcsDestination

static

The Google Cloud Storage location where the output is to be written to.

Property

Parameter

outputUriPrefix

string

Required. Google Cloud Storage URI to output directory, up to 2000 characters long. Accepted forms:

  • Prefix path: gs://bucket/directory The requesting user must have write permission to the bucket. The directory is created if it doesn't exist.
See also

google.cloud.automl.v1beta1.GcsDestination definition in proto format

GcsSource

static

The Google Cloud Storage location for the input content.

Property

Parameter

inputUris

Array of string

Required. Google Cloud Storage URIs to input files, up to 2000 characters long. Accepted forms:

  • Full object path, e.g. gs://bucket/directory/object.csv
See also

google.cloud.automl.v1beta1.GcsSource definition in proto format

GetAnnotationSpecRequest

static

Request message for AutoMl.GetAnnotationSpec.

Property

Parameter

name

string

The resource name of the annotation spec to retrieve.

See also

google.cloud.automl.v1beta1.GetAnnotationSpecRequest definition in proto format

GetColumnSpecRequest

static

Request message for AutoMl.GetColumnSpec.

Properties

Parameter

name

string

The resource name of the column spec to retrieve.

fieldMask

Object

Mask specifying which fields to read.

This object should have the same structure as FieldMask

See also

google.cloud.automl.v1beta1.GetColumnSpecRequest definition in proto format

GetDatasetRequest

static

Request message for AutoMl.GetDataset.

Property

Parameter

name

string

The resource name of the dataset to retrieve.

See also

google.cloud.automl.v1beta1.GetDatasetRequest definition in proto format

GetModelEvaluationRequest

static

Request message for AutoMl.GetModelEvaluation.

Property

Parameter

name

string

Resource name for the model evaluation.

See also

google.cloud.automl.v1beta1.GetModelEvaluationRequest definition in proto format

GetModelRequest

static

Request message for AutoMl.GetModel.

Property

Parameter

name

string

Resource name of the model.

See also

google.cloud.automl.v1beta1.GetModelRequest definition in proto format

GetTableSpecRequest

static

Request message for AutoMl.GetTableSpec.

Properties

Parameter

name

string

The resource name of the table spec to retrieve.

fieldMask

Object

Mask specifying which fields to read.

This object should have the same structure as FieldMask

See also

google.cloud.automl.v1beta1.GetTableSpecRequest definition in proto format

GranularStats

static

Stats split by a defined in context granularity.

Property

Parameter

buckets

Object with number keys and number properties

A map from granularity key to example count for that key. E.g. for hour_of_day 13 means 1pm, or for month_of_year 5 means May).

See also

google.cloud.automl.v1beta1.TimestampStats.GranularStats definition in proto format

HistogramBucket

static

A bucket of a histogram.

Properties

Parameter

min

number

The minimum value of the bucket, inclusive.

max

number

The maximum value of the bucket, exclusive unless max = "Infinity", in which case it's inclusive.

count

number

The number of data values that are in the bucket, i.e. are between min and max values.

See also

google.cloud.automl.v1beta1.Float64Stats.HistogramBucket definition in proto format

Image

static

A representation of an image. Only images up to 30MB in size are supported.

Properties

Parameter

imageBytes

string

Image content represented as a stream of bytes. Note: As with all bytes fields, protobuffers use a pure binary representation, whereas JSON representations use base64.

inputConfig

Object

An input config specifying the content of the image.

This object should have the same structure as InputConfig

thumbnailUri

string

Output only. HTTP URI to the thumbnail image.

See also

google.cloud.automl.v1beta1.Image definition in proto format

ImageClassificationDatasetMetadata

static

Dataset metadata that is specific to image classification.

Property

Parameter

classificationType

number

Required. Type of the classification problem.

The number should be among the values of ClassificationType

See also

google.cloud.automl.v1beta1.ImageClassificationDatasetMetadata definition in proto format

ImageClassificationModelMetadata

static

Model metadata for image classification.

Properties

Parameter

baseModelId

string

Optional. The ID of the base model. If it is specified, the new model will be created based on the base model. Otherwise, the new model will be created from scratch. The base model must be in the same project and location as the new model to create, and have the same model_type.

trainBudget

number

Required. The train budget of creating this model, expressed in hours. The actual train_cost will be equal or less than this value.

trainCost

number

Output only. The actual train cost of creating this model, expressed in hours. If this model is created from a base model, the train cost used to create the base model are not included.

stopReason

string

Output only. The reason that this create model operation stopped, e.g. BUDGET_REACHED, MODEL_CONVERGED.

modelType

string

Optional. Type of the model. The available values are:

  • cloud - Model to be used via prediction calls to AutoML API.
          This is the default value.
  • mobile-low-latency-1 - A model that, in addition to providing
          prediction via AutoML API, can also be exported (see
          AutoMl.ExportModel) and used on a mobile or edge device
          with TensorFlow afterwards. Expected to have low latency, but
          may have lower prediction quality than other models.
  • mobile-versatile-1 - A model that, in addition to providing
          prediction via AutoML API, can also be exported (see
          AutoMl.ExportModel) and used on a mobile or edge device
          with TensorFlow afterwards.
  • mobile-high-accuracy-1 - A model that, in addition to providing
          prediction via AutoML API, can also be exported (see
          AutoMl.ExportModel) and used on a mobile or edge device
          with TensorFlow afterwards.  Expected to have a higher
          latency, but should also have a higher prediction quality
          than other models.
  • mobile-core-ml-low-latency-1 - A model that, in addition to providing
          prediction via AutoML API, can also be exported (see
          AutoMl.ExportModel) and used on a mobile device with Core
          ML afterwards. Expected to have low latency, but may have
          lower prediction quality than other models.
  • mobile-core-ml-versatile-1 - A model that, in addition to providing
          prediction via AutoML API, can also be exported (see
          AutoMl.ExportModel) and used on a mobile device with Core
          ML afterwards.
  • mobile-core-ml-high-accuracy-1 - A model that, in addition to
          providing prediction via AutoML API, can also be exported
          (see AutoMl.ExportModel) and used on a mobile device with
          Core ML afterwards.  Expected to have a higher latency, but
          should also have a higher prediction quality than other
          models.
See also

google.cloud.automl.v1beta1.ImageClassificationModelMetadata definition in proto format

ImageObjectDetectionAnnotation

static

Annotation details for image object detection.

Properties

Parameter

boundingBox

Object

Output only. The rectangle representing the object location.

This object should have the same structure as BoundingPoly

score

number

Output only. The confidence that this annotation is positive for the parent example, value in [0, 1], higher means higher positivity confidence.

See also

google.cloud.automl.v1beta1.ImageObjectDetectionAnnotation definition in proto format

ImageObjectDetectionDatasetMetadata

static

Dataset metadata specific to image object detection.

See also

google.cloud.automl.v1beta1.ImageObjectDetectionDatasetMetadata definition in proto format

ImageObjectDetectionEvaluationMetrics

static

Model evaluation metrics for image object detection problems. Evaluates prediction quality of labeled bounding boxes.

Properties

Parameter

evaluatedBoundingBoxCount

number

Output only. The total number of bounding boxes (i.e. summed over all images) the ground truth used to create this evaluation had.

boundingBoxMetricsEntries

Array of Object

Output only. The bounding boxes match metrics for each Intersection-over-union threshold 0.05,0.10,...,0.95,0.96,0.97,0.98,0.99 and each label confidence threshold 0.05,0.10,...,0.95,0.96,0.97,0.98,0.99 pair.

This object should have the same structure as BoundingBoxMetricsEntry

boundingBoxMeanAveragePrecision

number

Output only. The single metric for bounding boxes evaluation: the mean_average_precision averaged over all bounding_box_metrics_entries.

See also

google.cloud.automl.v1beta1.ImageObjectDetectionEvaluationMetrics definition in proto format

ImageObjectDetectionModelDeploymentMetadata

static

Model deployment metadata specific to Image Object Detection.

Property

Parameter

nodeCount

number

Input only. The number of nodes to deploy the model on. A node is an abstraction of a machine resource, which can handle online prediction QPS as given in the model's

qps_per_node. Must be between 1 and 100, inclusive on both ends.

See also

google.cloud.automl.v1beta1.ImageObjectDetectionModelDeploymentMetadata definition in proto format

ImageObjectDetectionModelMetadata

static

Model metadata specific to image object detection.

Properties

Parameter

modelType

string

Optional. Type of the model. The available values are:

  • cloud-high-accuracy-1 - (default) A model to be used via prediction
          calls to AutoML API. Expected to have a higher latency, but
          should also have a higher prediction quality than other
          models.
  • cloud-low-latency-1 - A model to be used via prediction
          calls to AutoML API. Expected to have low latency, but may
          have lower prediction quality than other models.

nodeCount

number

Output only. The number of nodes this model is deployed on. A node is an abstraction of a machine resource, which can handle online prediction QPS as given in the qps_per_node field.

nodeQps

number

Output only. An approximate number of online prediction QPS that can be supported by this model per each node on which it is deployed.

See also

google.cloud.automl.v1beta1.ImageObjectDetectionModelMetadata definition in proto format

ImportDataRequest

static

Request message for AutoMl.ImportData.

Properties

Parameter

name

string

Required. Dataset name. Dataset must already exist. All imported annotations and examples will be added.

inputConfig

Object

Required. The desired input location and its domain specific semantics, if any.

This object should have the same structure as InputConfig

See also

google.cloud.automl.v1beta1.ImportDataRequest definition in proto format

InputConfig

static

Input configuration for ImportData Action.

The format of input depends on dataset_metadata the Dataset into which the import is happening has. As input source the gcs_source is expected, unless specified otherwise. If a file with identical content (even if it had different GCS_FILE_PATH) is mentioned multiple times , then its label, bounding boxes etc. are appended. The same file should be always provided with the same ML_USE and GCS_FILE_PATH, if it is not then these values are nondeterministically selected from the given ones.

The formats are represented in EBNF with commas being literal and with non-terminal symbols defined near the end of this comment. The formats are:

  • For Image Object Detection:
    CSV file(s) with each line in format:
      ML_USE,GCS_FILE_PATH,LABEL,BOUNDING_BOX
      GCS_FILE_PATH leads to image of up to 30MB in size. Supported
      extensions: .JPEG, .GIF, .PNG.
      Each image is assumed to be exhaustively labeled. The
      minimum allowed BOUNDING_BOX edge length is 0.01, and no more than
      500 BOUNDING_BOX-es per image are allowed.
    Three sample rows:
      TRAIN,gs://folder/image1.png,car,0.1,0.1,,,0.3,0.3,,
      TRAIN,gs://folder/image1.png,bike,.7,.6,,,.8,.9,,
      TEST,gs://folder/im2.png,car,0.1,0.1,0.2,0.1,0.2,0.3,0.1,0.3
  • For Video Classification:
    CSV file(s) with each line in format:
      ML_USE,GCS_FILE_PATH
      where ML_USE VALIDATE value should not be used. The GCS_FILE_PATH
      should lead to another .csv file which describes examples that have
      given ML_USE, using the following row format:
      GCS_FILE_PATH,LABEL,TIME_SEGMENT_START,TIME_SEGMENT_END
      Here GCS_FILE_PATH leads to a video of up to 50GB in size and up
      to 3h duration. Supported extensions: .MOV, .MPEG4, .MP4, .AVI.
      TIME_SEGMENT_START and TIME_SEGMENT_END must be within the
      length of the video, and end has to be after the start. Any segment
      of a video which has one or more labels on it, is considered a
      hard negative for all other labels. Any segment with no labels on
      it is considered to be unknown.
    Sample top level CSV file:
      TRAIN,gs://folder/train_videos.csv
      TEST,gs://folder/test_videos.csv
      UNASSIGNED,gs://folder/other_videos.csv
    Three sample rows of a CSV file for a particular ML_USE:
      gs://folder/video1.avi,car,120,180.000021
      gs://folder/video1.avi,bike,150,180.000021
      gs://folder/vid2.avi,car,0,60.5
  • For Text Extraction:
    CSV file(s) with each line in format:
      ML_USE,GCS_FILE_PATH
      GCS_FILE_PATH leads to a .JSONL (i.e. JSON Lines) file which either
      imports text in-line or as documents.
      The in-line .JSONL file contains, per line, a proto that wraps a
        TextSnippet proto (in json representation) followed by one or
        more AnnotationPayload protos (called annotations), which have
        display_name and text_extraction detail populated.
        Given text is expected to be annotated exhaustively, e.g. if you
        look for animals and text contains "dolphin" that is not labeled,
        then "dolphin" will be assumed to not be an animal.
        Any given text snippet content must have 30,000 characters or
        less, and also be UTF-8 NFC encoded (ASCII already is).
      The document .JSONL file contains, per line, a proto that wraps a
        Document proto with input_config set. Only PDF documents are
        supported now, and each document may be up to 2MB large.
        Currently annotations on documents cannot be specified at import.
      Any given .JSONL file must be 100MB or smaller.
    Three sample CSV rows:
      TRAIN,gs://folder/file1.jsonl
      VALIDATE,gs://folder/file2.jsonl
      TEST,gs://folder/file3.jsonl
    Sample in-line JSON Lines file (presented here with artificial line
    breaks, but the only actual line break is denoted by \n).:
      {
        "text_snippet": {
          "content": "dog car cat"
        },
        "annotations": [
           {
             "display_name": "animal",
             "text_extraction": {
               "text_segment": {"start_offset": 0, "end_offset": 2}
             }
           },
           {
             "display_name": "vehicle",
             "text_extraction": {
               "text_segment": {"start_offset": 4, "end_offset": 6}
             }
           },
           {
             "display_name": "animal",
             "text_extraction": {
               "text_segment": {"start_offset": 8, "end_offset": 10}
             }
           }
        ]
      }\n
      {
         "text_snippet": {
           "content": "This dog is good."
         },
         "annotations": [
            {
              "display_name": "animal",
              "text_extraction": {
                "text_segment": {"start_offset": 5, "end_offset": 7}
              }
            }
         ]
      }
    Sample document JSON Lines file (presented here with artificial line
    breaks, but the only actual line break is denoted by \n).:
      {
        "document": {
          "input_config": {
            "gcs_source": { "input_uris": [ "gs://folder/document1.pdf" ]
            }
          }
        }
      }\n
      {
        "document": {
          "input_config": {
            "gcs_source": { "input_uris": [ "gs://folder/document2.pdf" ]
            }
          }
        }
      }
    • For Tables: Either gcs_source or

bigquery_source can be used. All inputs will be concatenated into a single

primary_table For gcs_source: CSV file(s), where first file must have a header containing unique column names, other files may have such header line too, and all other lines contain values for the header columns. Each line must have 1,000,000 or fewer characters. First three sample rows of a CSV file: "Id","First Name","Last Name","Dob","Addresses"

"1","John","Doe","1968-01-22","[{"status":"current","address":"123_First_Avenue","city":"Seattle","state":"WA","zip":"11111","numberOfYears":"1"},{"status":"previous","address":"456_Main_Street","city":"Portland","state":"OR","zip":"22222","numberOfYears":"5"}]"

"2","Jane","Doe","1980-10-16","[{"status":"current","address":"789_Any_Avenue","city":"Albany","state":"NY","zip":"33333","numberOfYears":"2"},{"status":"previous","address":"321_Main_Street","city":"Hoboken","state":"NJ","zip":"44444","numberOfYears":"3"}]} For bigquery_source: An URI of a BigQuery table. An imported table must have between 2 and 1,000 columns, inclusive, and between 1,000 and 10,000,000 rows, inclusive.

  • For Text Sentiment:

    CSV file(s) with each line in format:
      ML_USE,TEXT_SNIPPET,SENTIMENT
      TEXT_SNIPPET must have up to 500 characters.
    Three sample rows:
    TRAIN,"@freewrytin God is way too good for Claritin",2
    TRAIN,"I need Claritin so bad",3
    TEST,"Thank god for Claritin.",4

    Definitions: ML_USE = "TRAIN" | "VALIDATE" | "TEST" | "UNASSIGNED"

      Describes how the given example (file) should be used for model
      training. "UNASSIGNED" can be used when user has no preference.

    GCS_FILE_PATH = A path to file on GCS, e.g. "gs://folder/image1.png". LABEL = A display name of an object on an image, video etc., e.g. "dog".

     Must be up to 32 characters long and can consist only of ASCII
     Latin letters A-Z and a-z, underscores(_), and ASCII digits 0-9.
     For each label an AnnotationSpec is created which display_name
     becomes the label; AnnotationSpecs are given back in predictions.

    INSTANCE_ID = A positive integer that identifies a specific instance of a

           labeled entity on an example. Used e.g. to track two cars on
           a video while being able to tell apart which one is which.

    BOUNDING_BOX = VERTEX,VERTEX,VERTEX,VERTEX | VERTEX,,,VERTEX,,

            A rectangle parallel to the frame of the example (image,
            video). If 4 vertices are given they are connected by edges
            in the order provided, if 2 are given they are recognized
            as diagonally opposite vertices of the rectangle.

    VERTEX = COORDINATE,COORDINATE

      First coordinate is horizontal (x), the second is vertical (y).

    COORDINATE = A float in 0 to 1 range, relative to total length of

          image or video in given dimension. For fractions the
          leading non-decimal 0 can be omitted (i.e. 0.3 = .3).
          Point 0,0 is in top left.

    TIME_SEGMENT_START = TIME_OFFSET

                  Expresses a beginning, inclusive, of a time segment
                  within an example that has a time dimension
                  (e.g. video).

    TIME_SEGMENT_END = TIME_OFFSET

                Expresses an end, exclusive, of a time segment within
                an example that has a time dimension (e.g. video).

    TIME_OFFSET = A number of seconds as measured from the start of an

           example (e.g. video). Fractions are allowed, up to a
           microsecond precision. "inf" is allowed, and it means the end
           of the example.

    TEXT_SNIPPET = A content of a text snippet, UTF-8 encoded. SENTIMENT = An integer between 0 and

         Dataset.text_sentiment_dataset_metadata.sentiment_max
         (inclusive). Describes the ordinal of the sentiment - higher
         value means a more positive sentiment. All the values are
         completely relative, i.e. neither 0 needs to mean a negative or
         neutral sentiment nor sentiment_max needs to mean a positive one
         - it is just required that 0 is the least positive sentiment
         in the data, and sentiment_max is the  most positive one.
         The SENTIMENT shouldn't be confused with "score" or "magnitude"
         from the previous Natural Language Sentiment Analysis API.
         All SENTIMENT values between 0 and sentiment_max must be
         represented in the imported data. On prediction the same 0 to
         sentiment_max range will be used. The difference between
         neighboring sentiment values needs not to be uniform, e.g. 1 and
         2 may be similar whereas the difference between 2 and 3 may be
         huge.

    Errors: If any of the provided CSV files can't be parsed or if more than certain percent of CSV rows cannot be processed then the operation fails and nothing is imported. Regardless of overall success or failure the per-row failures, up to a certain count cap, will be listed in Operation.metadata.partial_failures.

Properties

Parameter

gcsSource

Object

The Google Cloud Storage location for the input content.

This object should have the same structure as GcsSource

bigquerySource

Object

The BigQuery location for the input content.

This object should have the same structure as BigQuerySource

params

Object with string properties

Additional domain-specific parameters describing the semantic of the imported data, any string must be up to 25000 characters long.

  • For Tables: schema_inference_version - (integer) Required. The version of the
    algorithm that should be used for the initial inference of the
    schema (columns' DataTypes) of the table the data is being imported
    into. Allowed values: "1".
See also

google.cloud.automl.v1beta1.InputConfig definition in proto format

ListColumnSpecsRequest

static

Request message for AutoMl.ListColumnSpecs.

Properties

Parameter

parent

string

The resource name of the table spec to list column specs from.

fieldMask

Object

Mask specifying which fields to read.

This object should have the same structure as FieldMask

filter

string

Filter expression, see go/filtering.

pageSize

number

Requested page size. The server can return fewer results than requested. If unspecified, the server will pick a default size.

pageToken

string

A token identifying a page of results for the server to return. Typically obtained from the ListColumnSpecsResponse.next_page_token field of the previous AutoMl.ListColumnSpecs call.

See also

google.cloud.automl.v1beta1.ListColumnSpecsRequest definition in proto format

ListColumnSpecsResponse

static

Response message for AutoMl.ListColumnSpecs.

Properties

Parameter

columnSpecs

Array of Object

The column specs read.

This object should have the same structure as ColumnSpec

nextPageToken

string

A token to retrieve next page of results. Pass to ListColumnSpecsRequest.page_token to obtain that page.

See also

google.cloud.automl.v1beta1.ListColumnSpecsResponse definition in proto format

ListDatasetsRequest

static

Request message for AutoMl.ListDatasets.

Properties

Parameter

parent

string

The resource name of the project from which to list datasets.

filter

string

An expression for filtering the results of the request.

* `dataset_metadata` - for existence of the case (e.g.
          image_classification_dataset_metadata:*). Some examples of using the filter are:

* `translation_dataset_metadata:*` --> The dataset has
                                       translation_dataset_metadata.

pageSize

number

Requested page size. Server may return fewer results than requested. If unspecified, server will pick a default size.

pageToken

string

A token identifying a page of results for the server to return Typically obtained via ListDatasetsResponse.next_page_token of the previous AutoMl.ListDatasets call.

See also

google.cloud.automl.v1beta1.ListDatasetsRequest definition in proto format

ListDatasetsResponse

static

Response message for AutoMl.ListDatasets.

Properties

Parameter

datasets

Array of Object

The datasets read.

This object should have the same structure as Dataset

nextPageToken

string

A token to retrieve next page of results. Pass to ListDatasetsRequest.page_token to obtain that page.

See also

google.cloud.automl.v1beta1.ListDatasetsResponse definition in proto format

ListModelEvaluationsRequest

static

Request message for AutoMl.ListModelEvaluations.

Properties

Parameter

parent

string

Resource name of the model to list the model evaluations for. If modelId is set as "-", this will list model evaluations from across all models of the parent location.

filter

string

An expression for filtering the results of the request.

* `annotation_spec_id` - for =, !=  or existence. See example below for
                       the last.

Some examples of using the filter are:

* `annotation_spec_id!=4` --> The model evaluation was done for
                          annotation spec with ID different than 4.
* `NOT annotation_spec_id:*` --> The model evaluation was done for
                             aggregate of all annotation specs.

pageSize

number

Requested page size.

pageToken

string

A token identifying a page of results for the server to return. Typically obtained via ListModelEvaluationsResponse.next_page_token of the previous AutoMl.ListModelEvaluations call.

See also

google.cloud.automl.v1beta1.ListModelEvaluationsRequest definition in proto format

ListModelEvaluationsResponse

static

Response message for AutoMl.ListModelEvaluations.

Properties

Parameter

modelEvaluation

Array of Object

List of model evaluations in the requested page.

This object should have the same structure as ModelEvaluation

nextPageToken

string

A token to retrieve next page of results. Pass to the ListModelEvaluationsRequest.page_token field of a new AutoMl.ListModelEvaluations request to obtain that page.

See also

google.cloud.automl.v1beta1.ListModelEvaluationsResponse definition in proto format

ListModelsRequest

static

Request message for AutoMl.ListModels.

Properties

Parameter

parent

string

Resource name of the project, from which to list the models.

filter

string

An expression for filtering the results of the request.

* `model_metadata` - for existence of the case (e.g.
          video_classification_model_metadata:*).
* `dataset_id` - for = or !=. Some examples of using the filter are:

* `image_classification_model_metadata:*` --> The model has
                                     image_classification_model_metadata.
* `dataset_id=5` --> The model was created from a dataset with ID 5.

pageSize

number

Requested page size.

pageToken

string

A token identifying a page of results for the server to return Typically obtained via ListModelsResponse.next_page_token of the previous AutoMl.ListModels call.

See also

google.cloud.automl.v1beta1.ListModelsRequest definition in proto format

ListModelsResponse

static

Response message for AutoMl.ListModels.

Properties

Parameter

model

Array of Object

List of models in the requested page.

This object should have the same structure as Model

nextPageToken

string

A token to retrieve next page of results. Pass to ListModelsRequest.page_token to obtain that page.

See also

google.cloud.automl.v1beta1.ListModelsResponse definition in proto format

ListTableSpecsRequest

static

Request message for AutoMl.ListTableSpecs.

Properties

Parameter

parent

string

The resource name of the dataset to list table specs from.

fieldMask

Object

Mask specifying which fields to read.

This object should have the same structure as FieldMask

filter

string

Filter expression, see go/filtering.

pageSize

number

Requested page size. The server can return fewer results than requested. If unspecified, the server will pick a default size.

pageToken

string

A token identifying a page of results for the server to return. Typically obtained from the ListTableSpecsResponse.next_page_token field of the previous AutoMl.ListTableSpecs call.

See also

google.cloud.automl.v1beta1.ListTableSpecsRequest definition in proto format

ListTableSpecsResponse

static

Response message for AutoMl.ListTableSpecs.

Properties

Parameter

tableSpecs

Array of Object

The table specs read.

This object should have the same structure as TableSpec

nextPageToken

string

A token to retrieve next page of results. Pass to ListTableSpecsRequest.page_token to obtain that page.

See also

google.cloud.automl.v1beta1.ListTableSpecsResponse definition in proto format

Model

static

API proto representing a trained machine learning model.

Properties

Parameter

translationModelMetadata

Object

Metadata for translation models.

This object should have the same structure as TranslationModelMetadata

imageClassificationModelMetadata

Object

Metadata for image classification models.

This object should have the same structure as ImageClassificationModelMetadata

textClassificationModelMetadata

Object

Metadata for text classification models.

This object should have the same structure as TextClassificationModelMetadata

imageObjectDetectionModelMetadata

Object

Metadata for image object detection models.

This object should have the same structure as ImageObjectDetectionModelMetadata

videoClassificationModelMetadata

Object

Metadata for video classification models.

This object should have the same structure as VideoClassificationModelMetadata

textExtractionModelMetadata

Object

Metadata for text extraction models.

This object should have the same structure as TextExtractionModelMetadata

tablesModelMetadata

Object

Metadata for Tables models.

This object should have the same structure as TablesModelMetadata

textSentimentModelMetadata

Object

Metadata for text sentiment models.

This object should have the same structure as TextSentimentModelMetadata

name

string

Output only. Resource name of the model. Format: projects/{project_id}/locations/{location_id}/models/{model_id}

displayName

string

Required. The name of the model to show in the interface. The name can be up to 32 characters long and can consist only of ASCII Latin letters A-Z and a-z, underscores (_), and ASCII digits 0-9. It must start with a letter.

datasetId

string

Required. The resource ID of the dataset used to create the model. The dataset must come from the same ancestor project and location.

createTime

Object

Output only. Timestamp when this model was created.

This object should have the same structure as Timestamp

updateTime

Object

Output only. Timestamp when this model was last updated.

This object should have the same structure as Timestamp

deploymentState

number

Output only. Deployment state of the model. A model can only serve prediction requests after it gets deployed.

The number should be among the values of DeploymentState

See also

google.cloud.automl.v1beta1.Model definition in proto format

ModelEvaluation

static

Evaluation results of a model.

Properties

Parameter

classificationEvaluationMetrics

Object

Model evaluation metrics for image, text, video and tables classification. Tables problem is considered a classification when the target column has either CATEGORY or ARRAY(CATEGORY) DataType.

This object should have the same structure as ClassificationEvaluationMetrics

regressionEvaluationMetrics

Object

Model evaluation metrics for Tables regression. Tables problem is considered a regression when the target column has FLOAT64 DataType.

This object should have the same structure as RegressionEvaluationMetrics

translationEvaluationMetrics

Object

Model evaluation metrics for translation.

This object should have the same structure as TranslationEvaluationMetrics

imageObjectDetectionEvaluationMetrics

Object

Model evaluation metrics for image object detection.

This object should have the same structure as ImageObjectDetectionEvaluationMetrics

textSentimentEvaluationMetrics

Object

Evaluation metrics for text sentiment models.

This object should have the same structure as TextSentimentEvaluationMetrics

textExtractionEvaluationMetrics

Object

Evaluation metrics for text extraction models.

This object should have the same structure as TextExtractionEvaluationMetrics

name

string

Output only. Resource name of the model evaluation. Format:

projects/{project_id}/locations/{location_id}/models/{model_id}/modelEvaluations/{model_evaluation_id}

annotationSpecId

string

Output only. The ID of the annotation spec that the model evaluation applies to. The The ID is empty for the overall model evaluation. For Tables classification these are the distinct values of the target column at the moment of the evaluation; for this problem annotation specs in the dataset do not exist. NOTE: Currently there is no way to obtain the display_name of the annotation spec from its ID. To see the display_names, review the model evaluations in the UI.

displayName

string

Output only. The value of AnnotationSpec.display_name when the model was trained. Because this field returns a value at model training time, for different models trained using the same dataset, the returned value could be different as model owner could update the display_name between any two model training. The display_name is empty for the overall model evaluation.

createTime

Object

Output only. Timestamp when this model evaluation was created.

This object should have the same structure as Timestamp

evaluatedExampleCount

number

Output only. The number of examples used for model evaluation, i.e. for which ground truth from time of model creation is compared against the predicted annotations created by the model. For overall ModelEvaluation (i.e. with annotation_spec_id not set) this is the total number of all examples used for evaluation. Otherwise, this is the count of examples that according to the ground truth were annotated by the

annotation_spec_id.

See also

google.cloud.automl.v1beta1.ModelEvaluation definition in proto format

ModelExportOutputConfig

static

Output configuration for ModelExport Action.

Properties

Parameter

gcsDestination

Object

The Google Cloud Storage location where the model is to be written to. This location may only be set for the following model formats: "tflite", "edgetpu_tflite", "core_ml", "docker".

Under the directory given as the destination a new one with name "model-export--", where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format, will be created. Inside the model and any of its supporting files will be written.

This object should have the same structure as GcsDestination

gcrDestination

Object

The GCR location where model image is to be pushed to. This location may only be set for the following model formats: "docker".

The model image will be created under the given URI.

This object should have the same structure as GcrDestination

modelFormat

string

The format in which the model must be exported. The available, and default, formats depend on the problem and model type (if given problem and type combination doesn't have a format listed, it means its models are not exportable):

  • For Image Classification mobile-low-latency-1, mobile-versatile-1,

    mobile-high-accuracy-1:

    "tflite" (default), "edgetpu_tflite", "tf_saved_model", "docker".

  • For Image Classification mobile-core-ml-low-latency-1,

    mobile-core-ml-versatile-1, mobile-core-ml-high-accuracy-1:

    "core_ml" (default). Formats description:

  • tflite - Used for Android mobile devices.

  • edgetpu_tflite - Used for Edge TPU
                 devices.
  • tf_saved_model - A tensorflow model in SavedModel format.
  • docker - Used for Docker containers. Use the params field to customize
         the container. The container is verified to work correctly on
         ubuntu 16.04 operating system.
  • core_ml - Used for iOS mobile devices.

params

Object with string properties

Additional model-type and format specific parameters describing the requirements for the to be exported model files, any string must be up to 25000 characters long.

  • For docker format: cpu_architecture - (string) "x86_64" (default). gpu_architecture - (string) "none" (default), "nvidia".
See also

google.cloud.automl.v1beta1.ModelExportOutputConfig definition in proto format

NormalizedVertex

static

A vertex represents a 2D point in the image. The normalized vertex coordinates are between 0 to 1 fractions relative to the original plane (image, video). E.g. if the plane (e.g. whole image) would have size 10 x 20 then a point with normalized coordinates (0.1, 0.3) would be at the position (1, 6) on that plane.

Properties

Parameter

x

number

Required. Horizontal coordinate.

y

number

Required. Vertical coordinate.

See also

google.cloud.automl.v1beta1.NormalizedVertex definition in proto format

OutputConfig

static

Output configuration for ExportData.

As destination the gcs_destination must be set unless specified otherwise for a domain. Only ground truth annotations are exported (not approved annotations are not exported).

The outputs correspond to how the data was imported, and may be used as input to import data. The output formats are represented as EBNF with literal commas and same non-terminal symbols definitions are these in import data's InputConfig:

  • For Image Object Detection:

    CSV file(s) image_object_detection_1.csv,
    image_object_detection_2.csv,...,image_object_detection_N.csv
    with each line in format:
    ML_USE,GCS_FILE_PATH,LABEL,BOUNDING_BOX
    where GCS_FILE_PATHs point at the original, source locations of the
    imported images.
  • For Video Classification:

    CSV file video_classification.csv, with each line in format:
    ML_USE,GCS_FILE_PATH
    (may have muliple lines per a single ML_USE).
    Each GCS_FILE_PATH leads to another .csv file which
    describes examples that have given ML_USE, using the following
    row format:
    GCS_FILE_PATH,LABEL,TIME_SEGMENT_START,TIME_SEGMENT_END
    Here GCS_FILE_PATHs point at the original, source locations of the
    imported videos.
  • For Text Extraction:

    CSV file text_extraction.csv, with each line in format:
    ML_USE,GCS_FILE_PATH
    GCS_FILE_PATH leads to a .JSONL (i.e. JSON Lines) file which
    contains, per line, a proto that wraps a TextSnippet proto (in json
    representation) followed by AnnotationPayload protos (called
    annotations). If initially documents had been imported, corresponding
    OCR-ed representation is returned.
    • For Tables: Output depends on whether the dataset was imported from GCS or BigQuery. GCS case:

gcs_destination must be set. Exported are CSV file(s) tables_1.csv, tables_2.csv,...,tables_N.csv with each having as header line the table's column names, and all other lines contain values for the header columns. BigQuery case:

bigquery_destination pointing to a BigQuery project must be set. In the given project a new dataset will be created with name

export_data_<automl-dataset-display-name>_<timestamp-of-export-call> where will be made BigQuery-dataset-name compatible (e.g. most special characters will become underscores), and timestamp will be in YYYY_MM_DDThh_mm_ss_sssZ "based on ISO-8601" format. In that dataset a new table called primary_table will be created, and filled with precisely the same data as this obtained on import.

Properties

Parameter

gcsDestination

Object

The Google Cloud Storage location where the output is to be written to. For Image Object Detection, Text Extraction, Video Classification and Tables, in the given directory a new directory will be created with name: export_data-- where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. All export output will be written into that directory.

This object should have the same structure as GcsDestination

bigqueryDestination

Object

The BigQuery location where the output is to be written to.

This object should have the same structure as BigQueryDestination

See also

google.cloud.automl.v1beta1.OutputConfig definition in proto format

PredictRequest

static

Request message for PredictionService.Predict.

Properties

Parameter

name

string

Name of the model requested to serve the prediction.

payload

Object

Required. Payload to perform a prediction on. The payload must match the problem type that the model was trained to solve.

This object should have the same structure as ExamplePayload

params

Object with string properties

Additional domain-specific parameters, any string must be up to 25000 characters long.

  • For Image Classification:

    score_threshold - (float) A value from 0.0 to 1.0. When the model makes predictions for an image, it will only produce results that have at least this confidence score. The default is 0.5.

    • For Image Object Detection: score_threshold - (float) When Model detects objects on the image, it will only produce bounding boxes which have at least this confidence score. Value in 0 to 1 range, default is 0.5. max_bounding_box_count - (int64) No more than this number of bounding boxes will be returned in the response. Default is 100, the requested value may be limited by server.
See also

google.cloud.automl.v1beta1.PredictRequest definition in proto format

PredictResponse

static

Response message for PredictionService.Predict.

Properties

Parameter

payload

Array of Object

Prediction result. Translation and Text Sentiment will return precisely one payload.

This object should have the same structure as AnnotationPayload

metadata

Object with string properties

Additional domain-specific prediction response metadata.

  • For Image Object Detection: max_bounding_box_count - (int64) At most that many bounding boxes per image could have been returned.

  • For Text Sentiment: sentiment_score - (float, deprecated) A value between -1 and 1, -1 maps to least positive sentiment, while 1 maps to the most positive one and the higher the score, the more positive the sentiment in the document is. Yet these values are relative to the training data, so e.g. if all data was positive then -1 will be also positive (though the least). The sentiment_score shouldn't be confused with "score" or "magnitude" from the previous Natural Language Sentiment Analysis API.

See also

google.cloud.automl.v1beta1.PredictResponse definition in proto format

RegressionEvaluationMetrics

static

Metrics for regression problems.

Properties

Parameter

rootMeanSquaredError

number

Output only. Root Mean Squared Error (RMSE).

meanAbsoluteError

number

Output only. Mean Absolute Error (MAE).

meanAbsolutePercentageError

number

Output only. Mean absolute percentage error. Only set if all ground truth values are are positive.

rSquared

number

Output only. R squared.

See also

google.cloud.automl.v1beta1.RegressionEvaluationMetrics definition in proto format

Row

static

A representation of a row in a relational table.

Properties

Parameter

columnSpecIds

Array of string

The resource IDs of the column specs describing the columns of the row. If set must contain, but possibly in a different order, all input feature

column_spec_ids of the Model this row is being passed to. Note: The below values field must match order of this field, if this field is set.

values

Array of Object

Required. The values of the row cells, given in the same order as the column_spec_ids, or, if not set, then in the same order as input feature

column_specs of the Model this row is being passed to.

This object should have the same structure as Value

See also

google.cloud.automl.v1beta1.Row definition in proto format

Row

static

Output only. A row in the confusion matrix.

Property

Parameter

exampleCount

Array of number

Output only. Value of the specific cell in the confusion matrix. The number of values each row has (i.e. the length of the row) is equal to the length of the annotation_spec_id field.

See also

google.cloud.automl.v1beta1.ClassificationEvaluationMetrics.ConfusionMatrix.Row definition in proto format

SingleCategoryStats

static

The statistics of a single CATEGORY value.

Properties

Parameter

value

string

The CATEGORY value.

count

number

The number of occurrences of this value in the series.

See also

google.cloud.automl.v1beta1.CategoryStats.SingleCategoryStats definition in proto format

StringStats

static

The data statistics of a series of STRING values.

Property

Parameter

topUnigramStats

Array of Object

The statistics of the top 20 unigrams, ordered by count.

This object should have the same structure as UnigramStats

See also

google.cloud.automl.v1beta1.StringStats definition in proto format

StructStats

static

The data statistics of a series of STRUCT values.

Property

Parameter

fieldStats

Object with Object properties

Map from a field name of the struct to data stats aggregated over series of all data in that field across all the structs.

See also

google.cloud.automl.v1beta1.StructStats definition in proto format

StructType

static

StructType defines the DataType-s of a STRUCT type.

Property

Parameter

fields

Object with Object properties

Unordered map of struct field names to their data types. Fields cannot be added or removed via Update. Their names and data types are still mutable.

See also

google.cloud.automl.v1beta1.StructType definition in proto format

TablesAnnotation

static

Contains annotation details specific to Tables.

Properties

Parameter

score

number

Output only. A confidence estimate between 0.0 and 1.0, inclusive. A higher value means greater confidence in the returned value. For

target_column_spec of ARRAY(CATEGORY) data type, this is a confidence that one of the values in the ARRAY would be the provided value. For

target_column_spec of FLOAT64 data type the score is not populated.

predictionInterval

Object

Output only. Only populated when

target_column_spec has FLOAT64 data type (i.e. for regression predictions). An interval in which the exactly correct target value has 95% chance to be in.

This object should have the same structure as DoubleRange

value

Object

The predicted value of the row's

target_column. The value depends on the column's DataType: CATEGORY - the predicted (with the above confidence score) CATEGORY value. FLOAT64 - the predicted (with the above confidence score) FLOAT64 value. ARRAY(CATEGORY) - CATEGORY value meaning that this value would be in the ARRAY in that column (with the above confidence score).

This object should have the same structure as Value

tablesModelColumnInfo

Array of Object

Output only. Auxiliary information for each of the model's

input_feature_column_specs' with respect to this particular prediction.

This object should have the same structure as TablesModelColumnInfo

See also

google.cloud.automl.v1beta1.TablesAnnotation definition in proto format

TablesDatasetMetadata

static

Metadata for a dataset used for AutoML Tables.

Properties

Parameter

primaryTableSpecId

string

Output only. The table_spec_id of the primary table of this dataset.

targetColumnSpecId

string

column_spec_id of the primary table's column that should be used as the training & prediction target. This column must be non-nullable and have one of following data types (otherwise model creation will error):

  • CATEGORY
  • ARRAY(CATEGORY)
  • FLOAT64 Furthermore, if the type is CATEGORY or ARRAY(CATEGORY), then only up to 40 unique values may exist in that column across all rows, but for ARRAY(CATEGORY) unique values are counted as elements of the ARRAY (i.e. following 3 ARRAY-s: [A, B], [A], [B] are counted as having 2 unique values).

    NOTE: Updates of this field will instantly affect any other users concurrently working with the dataset.

weightColumnSpecId

string

column_spec_id of the primary table's column that should be used as the weight column, i.e. the higher the value the more important the row will be during model training. Required type: FLOAT64. Allowed values: 0 to 10000, inclusive on both ends; 0 means the row is ignored for training. If not set all rows are assumed to have equal weight of 1. NOTE: Updates of this field will instantly affect any other users concurrently working with the dataset.

mlUseColumnSpecId

string

column_spec_id of the primary table column which specifies a possible ML use of the row, i.e. the column will be used to split the rows into TRAIN, VALIDATE and TEST sets. Required type: STRING. This column, if set, must either have all of TRAIN, VALIDATE, TEST among its values, or only have TEST, UNASSIGNED values. In the latter case the rows with UNASSIGNED value will be assigned by AutoML. Note that if a given ml use distribution makes it impossible to create a "good" model, that call will error describing the issue. If both this column_spec_id and primary table's time_column_spec_id are not set, then all rows are treated as UNASSIGNED. NOTE: Updates of this field will instantly affect any other users concurrently working with the dataset.

targetColumnCorrelations

Object with Object properties

Output only. Correlations between

target_column, and other columns of the

primary_table. Only set if the target column is set. Mapping from other column spec id to its CorrelationStats with the target column. This field may be stale, see the stats_update_time field for for the timestamp at which these stats were last updated.

statsUpdateTime

Object

The most recent timestamp when target_column_correlations field and all descendant ColumnSpec.data_stats and ColumnSpec.top_correlated_columns fields were last (re-)generated. Any changes that happened to the dataset afterwards are not reflected in these fields values. The regeneration happens in the background on a best effort basis.

This object should have the same structure as Timestamp

See also

google.cloud.automl.v1beta1.TablesDatasetMetadata definition in proto format

TablesModelColumnInfo

static

An information specific to given column and Tables Model, in context of the Model and the predictions created by it.

Properties

Parameter

columnSpecName

string

Output only. The name of the ColumnSpec describing the column. Not populated when this proto is outputted to BigQuery.

columnDisplayName

string

Output only. The display name of the column (same as the display_name of its ColumnSpec).

featureImportance

number

Output only.

When given as part of a Model: Measurement of how much model predictions correctness on the TEST data depend on values in this column. A value between 0 and 1, higher means higher influence. These values are normalized - for all input feature columns of a given model they add to 1.

When given back by Predict or Batch Predict: Measurement of how impactful for the prediction returned for the given row the value in this column was. A value between 0 and 1, higher means larger impact. These values are normalized - for all input feature columns of a single predicted row they add to 1.

See also

google.cloud.automl.v1beta1.TablesModelColumnInfo definition in proto format

TablesModelMetadata

static

Model metadata specific to AutoML Tables.

Properties

Parameter

targetColumnSpec

Object

Column spec of the dataset's primary table's column the model is predicting. Snapshotted when model creation started. Only 3 fields are used: name - May be set on CreateModel, if it's not then the ColumnSpec corresponding to the current target_column_spec_id of the dataset the model is trained from is used. If neither is set, CreateModel will error. display_name - Output only. data_type - Output only.

This object should have the same structure as ColumnSpec

inputFeatureColumnSpecs

Array of Object

Column specs of the dataset's primary table's columns, on which the model is trained and which are used as the input for predictions. The

target_column as well as, according to dataset's state upon model creation,

weight_column, and

ml_use_column must never be included here. Only 3 fields are used: name - May be set on CreateModel, if set only the columns specified are used, otherwise all primary table's columns (except the ones listed above) are used for the training and prediction input. display_name - Output only. data_type - Output only.

This object should have the same structure as ColumnSpec

optimizationObjective

string

Objective function the model is optimizing towards. The training process creates a model that maximizes/minimizes the value of the objective function over the validation set.

The supported optimization objectives depend on the prediction_type. If the field is not set, a default objective function is used.

CLASSIFICATION_BINARY: "MAXIMIZE_AU_ROC" (default) - Maximize the area under the receiver operating characteristic (ROC) curve. "MINIMIZE_LOG_LOSS" - Minimize log loss. "MAXIMIZE_AU_PRC" - Maximize the area under the precision-recall curve.

CLASSIFICATION_MULTI_CLASS : "MINIMIZE_LOG_LOSS" (default) - Minimize log loss.

CLASSIFICATION_MULTI_LABEL: "MINIMIZE_LOG_LOSS" (default) - Minimize log loss.

REGRESSION: "MINIMIZE_RMSE" (default) - Minimize root-mean-squared error (RMSE). "MINIMIZE_MAE" - Minimize mean-absolute error (MAE). "MINIMIZE_RMSLE" - Minimize root-mean-squared log error (RMSLE).

FORECASTING: "MINIMIZE_RMSE" (default) - Minimize root-mean-squared error (RMSE). "MINIMIZE_MAE" - Minimize mean-absolute error (MAE).

tablesModelColumnInfo

Array of Object

Output only. Auxiliary information for each of the input_feature_column_specs, with respect to this particular model.

This object should have the same structure as TablesModelColumnInfo

trainBudgetMilliNodeHours

number

The train budget of creating this model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour.

The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend's discretion. This especially may happen when further model training ceases to provide any improvements.

If the budget is set to a value known to be insufficient to train a model for the given dataset, the training won't be attempted and will error.

trainCostMilliNodeHours

number

Output only. The actual training cost of the model, expressed in milli node hours, i.e. 1,000 value in this field means 1 node hour. Guaranteed to not exceed the train budget.

See also

google.cloud.automl.v1beta1.TablesModelMetadata definition in proto format

TableSpec

static

A specification of a relational table. The table's schema is represented via its child column specs. It is pre-populated as part of ImportData by schema inference algorithm, the version of which is a required parameter of ImportData InputConfig. Note: While working with a table, at times the schema may be inconsistent with the data in the table (e.g. string in a FLOAT64 column). The consistency validation is done upon creation of a model. Used by:

  • Tables

Properties

Parameter

name

string

Output only. The resource name of the table spec. Form:

projects/{project_id}/locations/{location_id}/datasets/{dataset_id}/tableSpecs/{table_spec_id}

timeColumnSpecId

string

column_spec_id of the time column. Only used if the parent dataset's ml_use_column_spec_id is not set. Used to split rows into TRAIN, VALIDATE and TEST sets such that oldest rows go to TRAIN set, newest to TEST, and those in between to VALIDATE. Required type: TIMESTAMP. If both this column and ml_use_column are not set, then ML use of all rows will be assigned by AutoML. NOTE: Updates of this field will instantly affect any other users concurrently working with the dataset.

rowCount

number

Output only. The number of rows (i.e. examples) in the table.

columnCount

number

Output only. The number of columns of the table. That is, the number of child ColumnSpec-s.

inputConfigs

Array of Object

Output only. Input configs via which data currently residing in the table had been imported.

This object should have the same structure as InputConfig

etag

string

Used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.

See also

google.cloud.automl.v1beta1.TableSpec definition in proto format

TextClassificationDatasetMetadata

static

Dataset metadata for classification.

Property

Parameter

classificationType

number

Required. Type of the classification problem.

The number should be among the values of ClassificationType

See also

google.cloud.automl.v1beta1.TextClassificationDatasetMetadata definition in proto format

TextClassificationModelMetadata

static

Model metadata that is specific to text classification.

See also

google.cloud.automl.v1beta1.TextClassificationModelMetadata definition in proto format

TextExtractionAnnotation

static

Annotation for identifying spans of text.

Properties

Parameter

score

number

Output only. A confidence estimate between 0.0 and 1.0. A higher value means greater confidence in correctness of the annotation.

textSegment

Object

Required. The part of the original text to which this annotation pertains.

This object should have the same structure as TextSegment

See also

google.cloud.automl.v1beta1.TextExtractionAnnotation definition in proto format

TextExtractionDatasetMetadata

static

Dataset metadata that is specific to text extraction

See also

google.cloud.automl.v1beta1.TextExtractionDatasetMetadata definition in proto format

TextExtractionEvaluationMetrics

static

Model evaluation metrics for text extraction problems.

Properties

Parameter

auPrc

number

Output only. The Area under precision recall curve metric.

confidenceMetricsEntries

Array of Object

Output only. Metrics that have confidence thresholds. Precision-recall curve can be derived from it.

This object should have the same structure as ConfidenceMetricsEntry

See also

google.cloud.automl.v1beta1.TextExtractionEvaluationMetrics definition in proto format

TextExtractionModelMetadata

static

Model metadata that is specific to text extraction.

See also

google.cloud.automl.v1beta1.TextExtractionModelMetadata definition in proto format

TextSegment

static

A contiguous part of a text (string), assuming it has an UTF-8 NFC encoding. .

Properties

Parameter

content

string

Output only. The content of the TextSegment.

startOffset

number

Required. Zero-based character index of the first character of the text segment (counting characters from the beginning of the text).

endOffset

number

Required. Zero-based character index of the first character past the end of the text segment (counting character from the beginning of the text). The character at the end_offset is NOT included in the text segment.

See also

google.cloud.automl.v1beta1.TextSegment definition in proto format

TextSentimentAnnotation

static

Contains annotation details specific to text sentiment.

Property

Parameter

sentiment

number

Output only. The sentiment with the semantic, as given to the AutoMl.ImportData when populating the dataset from which the model used for the prediction had been trained. The sentiment values are between 0 and Dataset.text_sentiment_dataset_metadata.sentiment_max (inclusive), with higher value meaning more positive sentiment. They are completely relative, i.e. 0 means least positive sentiment and sentiment_max means the most positive from the sentiments present in the train data. Therefore e.g. if train data had only negative sentiment, then sentiment_max, would be still negative (although least negative). The sentiment shouldn't be confused with "score" or "magnitude" from the previous Natural Language Sentiment Analysis API.

See also

google.cloud.automl.v1beta1.TextSentimentAnnotation definition in proto format

TextSentimentDatasetMetadata

static

Dataset metadata for text sentiment.

Property

Parameter

sentimentMax

number

Required. A sentiment is expressed as an integer ordinal, where higher value means a more positive sentiment. The range of sentiments that will be used is between 0 and sentiment_max (inclusive on both ends), and all the values in the range must be represented in the dataset before a model can be created. sentiment_max value must be between 1 and 10 (inclusive).

See also

google.cloud.automl.v1beta1.TextSentimentDatasetMetadata definition in proto format

TextSentimentEvaluationMetrics

static

Model evaluation metrics for text sentiment problems.

Properties

Parameter

precision

number

Output only. Precision.

recall

number

Output only. Recall.

f1Score

number

Output only. The harmonic mean of recall and precision.

meanAbsoluteError

number

Output only. Mean absolute error. Only set for the overall model evaluation, not for evaluation of a single annotation spec.

meanSquaredError

number

Output only. Mean squared error. Only set for the overall model evaluation, not for evaluation of a single annotation spec.

linearKappa

number

Output only. Linear weighted kappa. Only set for the overall model evaluation, not for evaluation of a single annotation spec.

quadraticKappa

number

Output only. Quadratic weighted kappa. Only set for the overall model evaluation, not for evaluation of a single annotation spec.

confusionMatrix

Object

Output only. Confusion matrix of the evaluation. Only set for the overall model evaluation, not for evaluation of a single annotation spec.

This object should have the same structure as ConfusionMatrix

annotationSpecId

Array of string

Output only. The annotation spec ids used for this evaluation. Deprecated, remove after Boq Migration and use then TextSentimentModelMetadata.annotation_spec_count for count, and list all model evaluations to see the exact annotation_spec_ids that were used.

See also

google.cloud.automl.v1beta1.TextSentimentEvaluationMetrics definition in proto format

TextSentimentModelMetadata

static

Model metadata that is specific to text classification.

See also

google.cloud.automl.v1beta1.TextSentimentModelMetadata definition in proto format

TextSnippet

static

A representation of a text snippet.

Properties

Parameter

content

string

Required. The content of the text snippet as a string. Up to 250000 characters long.

mimeType

string

The format of the source text. Currently the only two allowed values are "text/html" and "text/plain". If left blank the format is automatically determined from the type of the uploaded content.

contentUri

string

Output only. HTTP URI where you can download the content.

See also

google.cloud.automl.v1beta1.TextSnippet definition in proto format

TimeSegment

static

A time period inside of an example that has a time dimension (e.g. video).

Properties

Parameter

startTimeOffset

Object

Start of the time segment (inclusive), represented as the duration since the example start.

This object should have the same structure as Duration

endTimeOffset

Object

End of the time segment (exclusive), represented as the duration since the example start.

This object should have the same structure as Duration

See also

google.cloud.automl.v1beta1.TimeSegment definition in proto format

TimestampStats

static

The data statistics of a series of TIMESTAMP values.

Property

Parameter

granularStats

Object with Object properties

The string key is the pre-defined granularity. Currently supported: hour_of_day, day_of_week, month_of_year. Granularities finer that the granularity of timestamp data are not populated (e.g. if timestamps are at day granularity, then hour_of_day is not populated).

See also

google.cloud.automl.v1beta1.TimestampStats definition in proto format

TranslationAnnotation

static

Annotation details specific to translation.

Property

Parameter

translatedContent

Object

Output only . The translated content.

This object should have the same structure as TextSnippet

See also

google.cloud.automl.v1beta1.TranslationAnnotation definition in proto format

TranslationDatasetMetadata

static

Dataset metadata that is specific to translation.

Properties

Parameter

sourceLanguageCode

string

Required. The BCP-47 language code of the source language.

targetLanguageCode

string

Required. The BCP-47 language code of the target language.

See also

google.cloud.automl.v1beta1.TranslationDatasetMetadata definition in proto format

TranslationEvaluationMetrics

static

Evaluation metrics for the dataset.

Properties

Parameter

bleuScore

number

Output only. BLEU score.

baseBleuScore

number

Output only. BLEU score for base model.

See also

google.cloud.automl.v1beta1.TranslationEvaluationMetrics definition in proto format

TranslationModelMetadata

static

Model metadata that is specific to translation.

Properties

Parameter

baseModel

string

The resource name of the model to use as a baseline to train the custom model. If unset, we use the default base model provided by Google Translate. Format: projects/{project_id}/locations/{location_id}/models/{model_id}

sourceLanguageCode

string

Output only. Inferred from the dataset. The source languge (The BCP-47 language code) that is used for training.

targetLanguageCode

string

Output only. The target languge (The BCP-47 language code) that is used for training.

See also

google.cloud.automl.v1beta1.TranslationModelMetadata definition in proto format

UndeployModelRequest

static

Request message for AutoMl.UndeployModel.

Property

Parameter

name

string

Resource name of the model to undeploy.

See also

google.cloud.automl.v1beta1.UndeployModelRequest definition in proto format

UnigramStats

static

The statistics of a unigram.

Properties

Parameter

value

string

The unigram.

count

number

The number of occurrences of this unigram in the series.

See also

google.cloud.automl.v1beta1.StringStats.UnigramStats definition in proto format

UpdateColumnSpecRequest

static

Request message for AutoMl.UpdateColumnSpec

Properties

Parameter

columnSpec

Object

The column spec which replaces the resource on the server.

This object should have the same structure as ColumnSpec

updateMask

Object

The update mask applies to the resource. For the FieldMask definition, see

https: //developers.google.com/protocol-buffers // /docs/reference/google.protobuf#fieldmask

This object should have the same structure as FieldMask

See also

google.cloud.automl.v1beta1.UpdateColumnSpecRequest definition in proto format

UpdateDatasetRequest

static

Request message for AutoMl.UpdateDataset

Properties

Parameter

dataset

Object

The dataset which replaces the resource on the server.

This object should have the same structure as Dataset

updateMask

Object

The update mask applies to the resource. For the FieldMask definition, see

https: //developers.google.com/protocol-buffers // /docs/reference/google.protobuf#fieldmask

This object should have the same structure as FieldMask

See also

google.cloud.automl.v1beta1.UpdateDatasetRequest definition in proto format

UpdateTableSpecRequest

static

Request message for AutoMl.UpdateTableSpec

Properties

Parameter

tableSpec

Object

The table spec which replaces the resource on the server.

This object should have the same structure as TableSpec

updateMask

Object

The update mask applies to the resource. For the FieldMask definition, see

https: //developers.google.com/protocol-buffers // /docs/reference/google.protobuf#fieldmask

This object should have the same structure as FieldMask

See also

google.cloud.automl.v1beta1.UpdateTableSpecRequest definition in proto format

VideoClassificationAnnotation

static

Contains annotation details specific to video classification.

Properties

Parameter

type

string

Output only. Expresses the type of video classification. Possible values:

  • segment - Classification done on a specified by user

    time segment of a video. AnnotationSpec is answered to be present
    in that time segment, if it is present in any part of it. The video
    ML model evaluations are done only for this type of classification.
  • shot- Shot-level classification.

    AutoML Video Intelligence determines the boundaries
    for each camera shot in the entire segment of the video that user
    specified in the request configuration. AutoML Video Intelligence
    then returns labels and their confidence scores for each detected
    shot, along with the start and end time of the shot.
    WARNING: Model evaluation is not done for this classification type,
    the quality of it depends on training data, but there are no
    metrics provided to describe that quality.
  • 1s_interval - AutoML Video Intelligence returns labels and their

    confidence scores for each second of the entire segment of the video
    that user specified in the request configuration.
    WARNING: Model evaluation is not done for this classification type,
    the quality of it depends on training data, but there are no
    metrics provided to describe that quality.

classificationAnnotation

Object

Output only . The classification details of this annotation.

This object should have the same structure as ClassificationAnnotation

timeSegment

Object

Output only . The time segment of the video to which the annotation applies.

This object should have the same structure as TimeSegment

See also

google.cloud.automl.v1beta1.VideoClassificationAnnotation definition in proto format

VideoClassificationDatasetMetadata

static

Dataset metadata specific to video classification. All Video Classification datasets are treated as multi label.

See also

google.cloud.automl.v1beta1.VideoClassificationDatasetMetadata definition in proto format

VideoClassificationModelMetadata

static

Model metadata specific to video classification.

See also

google.cloud.automl.v1beta1.VideoClassificationModelMetadata definition in proto format