REST Resource: projects.locations.models.modelEvaluations

Resource: ModelEvaluation

Evaluation results of a model.

JSON representation 
{
  "name": string,
  "annotationSpecId": string,
  "createTime": string,
  "evaluatedExampleCount": number,

  // Union field metrics can be only one of the following:
  "classificationEvaluationMetrics": {
    object(ClassificationEvaluationMetrics)
  },
  "translationEvaluationMetrics": {
    object(TranslationEvaluationMetrics)
  }
  // End of list of possible types for union field metrics.
}
Fields
name

string

Output only. Resource name of the model evaluation. Format:

projects/{project_id}/locations/{locationId}/models/{modelId}/modelEvaluations/{model_evaluation_id}
annotationSpecId

string

Output only. The ID of the annotation spec that the model evaluation applies to. The ID is empty for overall model evaluation. NOTE: Currently there is no way to obtain the displayName of the annotation spec from its ID. To see the display_names, review the model evaluations in the UI.

createTime

string (Timestamp format)

Output only. Timestamp when this model evaluation was created.

A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2014-10-02T15:01:23.045123456Z".
evaluatedExampleCount

number

Output only. The number of examples used for model evaluation.

Union field metrics. Output only. Problem type specific evaluation metrics. metrics can be only one of the following:
classificationEvaluationMetrics

object(ClassificationEvaluationMetrics)

Evaluation metrics for models that classify items.

translationEvaluationMetrics

object(TranslationEvaluationMetrics)

Evaluation metrics for translation models.

ClassificationEvaluationMetrics

Model evaluation metrics for classification problems. Visible only to v1beta1

JSON representation 
{
  "auPrc": number,
  "baseAuPrc": number,
  "confidenceMetricsEntry": [
    {
      object(ConfidenceMetricsEntry)
    }
  ],
  "confusionMatrix": {
    object(ConfusionMatrix)
  },
  "annotationSpecId": [
    string
  ]
}
Fields
auPrc

number

Output only. The Area under precision recall curve metric.

baseAuPrc

number

Output only. The Area under precision recall curve metric based on priors.

confidenceMetricsEntry[]

object(ConfidenceMetricsEntry)

Output only. Metrics that have confidence thresholds. Precision-recall curve can be derived from it.

confusionMatrix

object(ConfusionMatrix)

Output only. Confusion matrix of the evaluation. Only set for MULTICLASS classification problems where number of labels is no more than 10. Only set for model level evaluation, not for evaluation per label.

annotationSpecId[]

string

Output only. The annotation spec ids used for this evaluation.

ConfidenceMetricsEntry

Metrics for a single confidence threshold.

JSON representation 
{
  "confidenceThreshold": number,
  "recall": number,
  "precision": number,
  "f1Score": number,
  "recallAt1": number,
  "precisionAt1": number,
  "f1ScoreAt1": number
}
Fields
confidenceThreshold

number

Output only. The confidence threshold value used to compute the metrics.

recall

number

Output only. Recall under the given confidence threshold.

precision

number

Output only. Precision under the given confidence threshold.

f1Score

number

Output only. The harmonic mean of recall and precision.

recallAt1

number

Output only. The recall when only considering the label that has the highest prediction score and not below the confidence threshold for each example.

precisionAt1

number

Output only. The precision when only considering the label that has the highest predictionscore and not below the confidence threshold for each example.

f1ScoreAt1

number

Output only. The harmonic mean of recallAt1 and precisionAt1.

ConfusionMatrix

Confusion matrix of the model running the classification.

JSON representation 
{
  "annotationSpecId": [
    string
  ],
  "row": [
    {
      object(Row)
    }
  ]
}
Fields
annotationSpecId[]

string

Output only. IDs of the annotation specs used in the confusion matrix.

row[]

object(Row)

Output only. Rows in the confusion matrix. The number of rows is equal to the size of annotationSpecId. row[i].value[j] is the number of examples that have ground truth of the annotationSpecId[i] and are predicted as annotationSpecId[j] by the model being evaluated.

Row

Output only. A row in the confusion matrix.

JSON representation 
{
  "exampleCount": [
    number
  ]
}
Fields
exampleCount[]

number

Output only. Value of the specific cell in the confusion matrix. The number of values each row is equal to the size of annotatin_spec_id.

TranslationEvaluationMetrics

Evaluation metrics for the dataset.

JSON representation 
{
  "bleuScore": number,
  "baseBleuScore": number
}
Fields
bleuScore

number

Output only. BLEU score.

baseBleuScore

number

Output only. BLEU score for base model.

Methods

get

 Gets a model evaluation.

list

 Lists model evaluations.