Configures specific details of how a continuous evaluation job works. Provide this configuration when you create an EvaluationJob.
Specify this field if your model version performs image
classification or general classification.
annotationSpecSet in this configuration must match [Evalua
tionJob.annotationSpecSet][google.cloud.datalabeling.v1beta1.E
valuationJob.annotation_spec_set]. allowMultiLabel in this
configuration must match
classificationMetadata.isMultiLabel in [input_config][goog
le.cloud.datalabeling.v1beta1.EvaluationJobConfig.input_config
].
Specify this field if your model version performs text
classification. annotationSpecSet in this configuration
must match [EvaluationJob.annotationSpecSet][google.cloud.data
labeling.v1beta1.EvaluationJob.annotation_spec_set].
allowMultiLabel in this configuration must match
classificationMetadata.isMultiLabel in [input_config][goog
le.cloud.datalabeling.v1beta1.EvaluationJobConfig.input_config
].
Required. Details for calculating evaluation metrics and
creating
Evaulations.
If your model version performs image object detection, you
must specify the boundingBoxEvaluationOptions field within
this configuration. Otherwise, provide an empty object for
this configuration.
Required. Prediction keys that tell Data Labeling Service
where to find the data for evaluation in your BigQuery table.
When the service samples prediction input and output from your
model version and saves it to BigQuery, the data gets stored
as JSON strings in the BigQuery table. These keys tell Data
Labeling Service how to parse the JSON. You can provide the
following entries in this field: - data_json_key: the
data key for prediction input. You must provide either this
key or reference_json_key. - reference_json_key: the
data reference key for prediction input. You must provide
either this key or data_json_key. - label_json_key:
the label key for prediction output. Required. -
label_score_json_key: the score key for prediction output.
Required. - bounding_box_json_key: the bounding box key
for prediction output. Required if your model version
perform image object detection. Learn how to configure
prediction keys </ml-engine/docs/continuous-evaluation/create-
job#prediction-keys>__.
Required. Fraction of predictions to sample and save to BigQuery during each [evaluation interval][google.cloud.datala beling.v1beta1.EvaluationJob.schedule]. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.
Inheritance
builtins.object > google.protobuf.pyext._message.CMessage > builtins.object > google.protobuf.message.Message > EvaluationJobConfigClasses
BigqueryImportKeysEntry
API documentation for datalabeling_v1beta1.types.EvaluationJobConfig.BigqueryImportKeysEntry class.