Google Cloud Datalabeling V1beta1 Client - Class EvaluationJobConfig (0.5.4)

Reference documentation and code samples for the Google Cloud Datalabeling V1beta1 Client class EvaluationJobConfig.

Configures specific details of how a continuous evaluation job works. Provide this configuration when you create an EvaluationJob.

Generated from protobuf message google.cloud.datalabeling.v1beta1.EvaluationJobConfig

Namespace

Google \ Cloud \ DataLabeling \ V1beta1

Methods

__construct

Constructor.

Parameters
Name Description
data array

Optional. Data for populating the Message object.
↳ image_classification_config Google\Cloud\DataLabeling\V1beta1\ImageClassificationConfig

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.
↳ bounding_poly_config Google\Cloud\DataLabeling\V1beta1\BoundingPolyConfig

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.
↳ text_classification_config Google\Cloud\DataLabeling\V1beta1\TextClassificationConfig

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.
↳ input_config Google\Cloud\DataLabeling\V1beta1\InputConfig

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).
↳ evaluation_config Google\Cloud\DataLabeling\V1beta1\EvaluationConfig

Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.
↳ human_annotation_config Google\Cloud\DataLabeling\V1beta1\HumanAnnotationConfig

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.
↳ bigquery_import_keys array|Google\Protobuf\Internal\MapField

Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.
↳ example_count int

Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.
↳ example_sample_percentage float

Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.
↳ evaluation_job_alert_config Google\Cloud\DataLabeling\V1beta1\EvaluationJobAlertConfig

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

getImageClassificationConfig

Specify this field if your model version performs image classification or general classification.

annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\ImageClassificationConfig|null

hasImageClassificationConfig

setImageClassificationConfig

Specify this field if your model version performs image classification or general classification.

annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\ImageClassificationConfig
Returns
Type Description
$this

getBoundingPolyConfig

Specify this field if your model version performs image object detection (bounding box detection).

annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\BoundingPolyConfig|null

hasBoundingPolyConfig

setBoundingPolyConfig

Specify this field if your model version performs image object detection (bounding box detection).

annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\BoundingPolyConfig
Returns
Type Description
$this

getTextClassificationConfig

Specify this field if your model version performs text classification.

annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\TextClassificationConfig|null

hasTextClassificationConfig

setTextClassificationConfig

Specify this field if your model version performs text classification.

annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\TextClassificationConfig
Returns
Type Description
$this

getInputConfig

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:

  • dataType must be one of IMAGE, TEXT, or GENERAL_DATA.

  • annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection).

  • If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel.
  • You must specify bigquerySource (not gcsSource).
Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\InputConfig|null

hasInputConfig

clearInputConfig

setInputConfig

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:

  • dataType must be one of IMAGE, TEXT, or GENERAL_DATA.

  • annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection).

  • If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel.
  • You must specify bigquerySource (not gcsSource).
Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\InputConfig
Returns
Type Description
$this

getEvaluationConfig

Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\EvaluationConfig|null

hasEvaluationConfig

clearEvaluationConfig

setEvaluationConfig

Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\EvaluationConfig
Returns
Type Description
$this

getHumanAnnotationConfig

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field.

Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\HumanAnnotationConfig|null

hasHumanAnnotationConfig

clearHumanAnnotationConfig

setHumanAnnotationConfig

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field.

Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\HumanAnnotationConfig
Returns
Type Description
$this

getBigqueryImportKeys

Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON.

You can provide the following entries in this field:

  • data_json_key: the data key for prediction input. You must provide either this key or reference_json_key.
  • reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key.
  • label_json_key: the label key for prediction output. Required.
  • label_score_json_key: the score key for prediction output. Required.
  • bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.
Returns
Type Description
Google\Protobuf\Internal\MapField

setBigqueryImportKeys

Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON.

You can provide the following entries in this field:

  • data_json_key: the data key for prediction input. You must provide either this key or reference_json_key.
  • reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key.
  • label_json_key: the label key for prediction output. Required.
  • label_score_json_key: the score key for prediction output. Required.
  • bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.
Parameter
Name Description
var array|Google\Protobuf\Internal\MapField
Returns
Type Description
$this

getExampleCount

Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Returns
Type Description
int

setExampleCount

Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Parameter
Name Description
var int
Returns
Type Description
$this

getExampleSamplePercentage

Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Returns
Type Description
float

setExampleSamplePercentage

Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Parameter
Name Description
var float
Returns
Type Description
$this

getEvaluationJobAlertConfig

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Returns
Type Description
Google\Cloud\DataLabeling\V1beta1\EvaluationJobAlertConfig|null

hasEvaluationJobAlertConfig

clearEvaluationJobAlertConfig

setEvaluationJobAlertConfig

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Parameter
Name Description
var Google\Cloud\DataLabeling\V1beta1\EvaluationJobAlertConfig
Returns
Type Description
$this

getHumanAnnotationRequestConfig

Returns
Type Description
string