AI Platform Data Labeling Service V1beta1 API - Class Google::Cloud::DataLabeling::V1beta1::EvaluationJobConfig (v0.6.0)

Reference documentation and code samples for the AI Platform Data Labeling Service V1beta1 API class Google::Cloud::DataLabeling::V1beta1::EvaluationJobConfig.

Configures specific details of how a continuous evaluation job works. Provide this configuration when you create an EvaluationJob.

Inherits

  • Object

Extended By

  • Google::Protobuf::MessageExts::ClassMethods

Includes

  • Google::Protobuf::MessageExts

Methods

#bigquery_import_keys

def bigquery_import_keys() -> ::Google::Protobuf::Map{::String => ::String}
Returns
  • (::Google::Protobuf::Map{::String => ::String}) — Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON.

    You can provide the following entries in this field:

    • data_json_key: the data key for prediction input. You must provide either this key or reference_json_key.
    • reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key.
    • label_json_key: the label key for prediction output. Required.
    • label_score_json_key: the score key for prediction output. Required.
    • bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection.

    Learn how to configure prediction keys.

#bigquery_import_keys=

def bigquery_import_keys=(value) -> ::Google::Protobuf::Map{::String => ::String}
Parameter
  • value (::Google::Protobuf::Map{::String => ::String}) — Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON.

    You can provide the following entries in this field:

    • data_json_key: the data key for prediction input. You must provide either this key or reference_json_key.
    • reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key.
    • label_json_key: the label key for prediction output. Required.
    • label_score_json_key: the score key for prediction output. Required.
    • bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection.

    Learn how to configure prediction keys.

Returns
  • (::Google::Protobuf::Map{::String => ::String}) — Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON.

    You can provide the following entries in this field:

    • data_json_key: the data key for prediction input. You must provide either this key or reference_json_key.
    • reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key.
    • label_json_key: the label key for prediction output. Required.
    • label_score_json_key: the score key for prediction output. Required.
    • bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection.

    Learn how to configure prediction keys.

#bounding_poly_config

def bounding_poly_config() -> ::Google::Cloud::DataLabeling::V1beta1::BoundingPolyConfig
Returns

#bounding_poly_config=

def bounding_poly_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::BoundingPolyConfig
Parameter
Returns

#evaluation_config

def evaluation_config() -> ::Google::Cloud::DataLabeling::V1beta1::EvaluationConfig
Returns
  • (::Google::Cloud::DataLabeling::V1beta1::EvaluationConfig) — Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

#evaluation_config=

def evaluation_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::EvaluationConfig
Parameter
  • value (::Google::Cloud::DataLabeling::V1beta1::EvaluationConfig) — Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.
Returns
  • (::Google::Cloud::DataLabeling::V1beta1::EvaluationConfig) — Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

#evaluation_job_alert_config

def evaluation_job_alert_config() -> ::Google::Cloud::DataLabeling::V1beta1::EvaluationJobAlertConfig
Returns

#evaluation_job_alert_config=

def evaluation_job_alert_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::EvaluationJobAlertConfig
Parameter
Returns

#example_count

def example_count() -> ::Integer
Returns
  • (::Integer) — Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

#example_count=

def example_count=(value) -> ::Integer
Parameter
  • value (::Integer) — Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.
Returns
  • (::Integer) — Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

#example_sample_percentage

def example_sample_percentage() -> ::Float
Returns
  • (::Float) — Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

#example_sample_percentage=

def example_sample_percentage=(value) -> ::Float
Parameter
  • value (::Float) — Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.
Returns
  • (::Float) — Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

#human_annotation_config

def human_annotation_config() -> ::Google::Cloud::DataLabeling::V1beta1::HumanAnnotationConfig
Returns
  • (::Google::Cloud::DataLabeling::V1beta1::HumanAnnotationConfig) — Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field.

    Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

#human_annotation_config=

def human_annotation_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::HumanAnnotationConfig
Parameter
  • value (::Google::Cloud::DataLabeling::V1beta1::HumanAnnotationConfig) — Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field.

    Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Returns
  • (::Google::Cloud::DataLabeling::V1beta1::HumanAnnotationConfig) — Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field.

    Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

#image_classification_config

def image_classification_config() -> ::Google::Cloud::DataLabeling::V1beta1::ImageClassificationConfig
Returns

#image_classification_config=

def image_classification_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::ImageClassificationConfig
Parameter
Returns

#input_config

def input_config() -> ::Google::Cloud::DataLabeling::V1beta1::InputConfig
Returns
  • (::Google::Cloud::DataLabeling::V1beta1::InputConfig) —

    Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:

    • dataType must be one of IMAGE, TEXT, or GENERAL_DATA.
    • annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection).
    • If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel.
    • You must specify bigquerySource (not gcsSource).

#input_config=

def input_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::InputConfig
Parameter
  • value (::Google::Cloud::DataLabeling::V1beta1::InputConfig) —

    Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:

    • dataType must be one of IMAGE, TEXT, or GENERAL_DATA.
    • annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection).
    • If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel.
    • You must specify bigquerySource (not gcsSource).
Returns
  • (::Google::Cloud::DataLabeling::V1beta1::InputConfig) —

    Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:

    • dataType must be one of IMAGE, TEXT, or GENERAL_DATA.
    • annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection).
    • If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel.
    • You must specify bigquerySource (not gcsSource).

#text_classification_config

def text_classification_config() -> ::Google::Cloud::DataLabeling::V1beta1::TextClassificationConfig
Returns

#text_classification_config=

def text_classification_config=(value) -> ::Google::Cloud::DataLabeling::V1beta1::TextClassificationConfig
Parameter
Returns