Class EvaluationJobConfig (1.6.0)

Version 1.6.0

• 1.10.3 (latest)
• 1.10.2
• 1.9.0
• 1.8.3
• 1.7.0
• 1.6.3
• 1.5.2
• 1.4.0
• 1.3.0
• 1.2.4
• 1.1.0
• 1.0.0
• 0.4.2
• 0.3.0
• 0.2.1

EvaluationJobConfig(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configures specific details of how a continuous evaluation job works. Provide this configuration when you create an EvaluationJob.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Attributes

Name	Description
image_classification_config	google.cloud.datalabeling_v1beta1.types.ImageClassificationConfig Specify this field if your model version performs image classification or general classification. ``annotationSpecSet`` in this configuration must match EvaluationJob.annotationSpecSet. ``allowMultiLabel`` in this configuration must match ``classificationMetadata.isMultiLabel`` in input_config. This field is a member of `oneof`_ ``human_annotation_request_config``.
bounding_poly_config	google.cloud.datalabeling_v1beta1.types.BoundingPolyConfig Specify this field if your model version performs image object detection (bounding box detection). ``annotationSpecSet`` in this configuration must match EvaluationJob.annotationSpecSet. This field is a member of `oneof`_ ``human_annotation_request_config``.
text_classification_config	google.cloud.datalabeling_v1beta1.types.TextClassificationConfig Specify this field if your model version performs text classification. ``annotationSpecSet`` in this configuration must match EvaluationJob.annotationSpecSet. ``allowMultiLabel`` in this configuration must match ``classificationMetadata.isMultiLabel`` in input_config. This field is a member of `oneof`_ ``human_annotation_request_config``.
input_config	google.cloud.datalabeling_v1beta1.types.InputConfig Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: - ``dataType`` must be one of ``IMAGE``, ``TEXT``, or ``GENERAL_DATA``. - ``annotationType`` must be one of ``IMAGE_CLASSIFICATION_ANNOTATION``, ``TEXT_CLASSIFICATION_ANNOTATION``, ``GENERAL_CLASSIFICATION_ANNOTATION``, or ``IMAGE_BOUNDING_BOX_ANNOTATION`` (image object detection). - If your machine learning model performs classification, you must specify ``classificationMetadata.isMultiLabel``. - You must specify ``bigquerySource`` (not ``gcsSource``).
evaluation_config	google.cloud.datalabeling_v1beta1.types.EvaluationConfig Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the ``boundingBoxEvaluationOptions`` field within this configuration. Otherwise, provide an empty object for this configuration.
human_annotation_config	google.cloud.datalabeling_v1beta1.types.HumanAnnotationConfig Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to ``true`` for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the ``instruction`` field within this configuration.
bigquery_import_keys	Mapping[str, str] Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: - ``data_json_key``: the data key for prediction input. You must provide either this key or ``reference_json_key``. - ``reference_json_key``: the data reference key for prediction input. You must provide either this key or ``data_json_key``. - ``label_json_key``: the label key for prediction output. Required. - ``label_score_json_key``: the score key for prediction output. Required. - ``bounding_box_json_key``: the bounding box key for prediction output. Required if your model version perform image object detection. Learn `how to configure prediction keys `__.
example_count	int Required. The maximum number of predictions to sample and save to BigQuery during each [evaluation interval][google.cloud.datalabeling.v1beta1.EvaluationJob.schedule]. This limit overrides ``example_sample_percentage``: even if the service has not sampled enough predictions to fulfill ``example_sample_perecentage`` during an interval, it stops sampling predictions when it meets this limit.
example_sample_percentage	float Required. Fraction of predictions to sample and save to BigQuery during each [evaluation interval][google.cloud.datalabeling.v1beta1.EvaluationJob.schedule]. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.
evaluation_job_alert_config	google.cloud.datalabeling_v1beta1.types.EvaluationJobAlertConfig Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Inheritance

builtins.object > proto.message.Message > EvaluationJobConfig

Classes

BigqueryImportKeysEntry

BigqueryImportKeysEntry(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The abstract base class for a message.

Parameters

Name	Description
kwargs	dict Keys and values corresponding to the fields of the message.
mapping	Union[dict, `.Message`] A dictionary or message to be used to determine the values for this message.
ignore_unknown_fields	Optional(bool) If True, do not raise errors for unknown fields. Only applied if mapping is a mapping type or there are keyword parameters.