Imports a list of externally generated EvaluatedAnnotations.
HTTP request
POST https://{service-endpoint}/v1/{parent}:batchImport
Where {service-endpoint}
is one of the supported service endpoints.
Path parameters
Parameters | |
---|---|
parent |
Required. The name of the parent ModelEvaluationSlice resource. Format: |
Request body
The request body contains data with the following structure:
JSON representation |
---|
{
"evaluatedAnnotations": [
{
object ( |
Fields | |
---|---|
evaluatedAnnotations[] |
Required. Evaluated annotations resource to be imported. |
Response body
Response message for ModelService.BatchImportEvaluatedAnnotations
If successful, the response body contains data with the following structure:
JSON representation |
---|
{ "importedEvaluatedAnnotationsCount": integer } |
Fields | |
---|---|
importedEvaluatedAnnotationsCount |
Output only. Number of EvaluatedAnnotations imported. |
Authorization scopes
Requires the following OAuth scope:
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview.
IAM Permissions
Requires the following IAM permission on the parent
resource:
aiplatform.modelEvaluationSlices.import
For more information, see the IAM documentation.
EvaluatedAnnotation
True positive, false positive, or false negative.
EvaluatedAnnotation is only available under ModelEvaluationSlice with slice of annotationSpec
dimension.
JSON representation |
---|
{ "type": enum ( |
Fields | |
---|---|
type |
Output only. type of the EvaluatedAnnotation. |
predictions[] |
Output only. The model predicted annotations. For true positive, there is one and only one prediction, which matches the only one ground truth annotation in For false positive, there is one and only one prediction, which doesn't match any ground truth annotation of the corresponding [data_item_view_id][EvaluatedAnnotation.data_item_view_id]. For false negative, there are zero or more predictions which are similar to the only ground truth annotation in The schema of the prediction is stored in |
groundTruths[] |
Output only. The ground truth Annotations, i.e. the Annotations that exist in the test data the Model is evaluated on. For true positive, there is one and only one ground truth annotation, which matches the only prediction in For false positive, there are zero or more ground truth annotations that are similar to the only prediction in For false negative, there is one and only one ground truth annotation, which doesn't match any predictions created by the model. The schema of the ground truth is stored in |
dataItemPayload |
Output only. The data item payload that the Model predicted this EvaluatedAnnotation on. |
evaluatedDataItemViewId |
Output only. id of the EvaluatedDataItemView under the same ancestor ModelEvaluation. The EvaluatedDataItemView consists of all ground truths and predictions on |
explanations[] |
Explanations of The attributions list in the |
errorAnalysisAnnotations[] |
Annotations of model error analysis results. |
EvaluatedAnnotationType
Describes the type of the EvaluatedAnnotation. The type is determined
Enums | |
---|---|
EVALUATED_ANNOTATION_TYPE_UNSPECIFIED |
Invalid value. |
TRUE_POSITIVE |
The EvaluatedAnnotation is a true positive. It has a prediction created by the Model and a ground truth Annotation which the prediction matches. |
FALSE_POSITIVE |
The EvaluatedAnnotation is false positive. It has a prediction created by the Model which does not match any ground truth annotation. |
FALSE_NEGATIVE |
The EvaluatedAnnotation is false negative. It has a ground truth annotation which is not matched by any of the model created predictions. |
EvaluatedAnnotationExplanation
Explanation result of the prediction produced by the Model.
JSON representation |
---|
{
"explanationType": string,
"explanation": {
object ( |
Fields | |
---|---|
explanationType |
Explanation type. For AutoML Image Classification models, possible values are:
|
explanation |
Explanation attribution response details. |
ErrorAnalysisAnnotation
Model error analysis for each annotation.
JSON representation |
---|
{ "attributedItems": [ { object ( |
Fields | |
---|---|
attributedItems[] |
Attributed items for a given annotation, typically representing neighbors from the training sets constrained by the query type. |
queryType |
The query type used for finding the attributed items. |
outlierScore |
The outlier score of this annotated item. Usually defined as the min of all distances from attributed items. |
outlierThreshold |
The threshold used to determine if this annotation is an outlier or not. |
AttributedItem
Attributed items for a given annotation, typically representing neighbors from the training sets constrained by the query type.
JSON representation |
---|
{ "annotationResourceName": string, "distance": number } |
Fields | |
---|---|
annotationResourceName |
The unique id for each annotation. Used by FE to allocate the annotation in DB. |
distance |
The distance of this item to the annotation. |
QueryType
The query type used for finding the attributed items.
Enums | |
---|---|
QUERY_TYPE_UNSPECIFIED |
Unspecified query type for model error analysis. |
ALL_SIMILAR |
Query similar samples across all classes in the dataset. |
SAME_CLASS_SIMILAR |
Query similar samples from the same class of the input sample. |
SAME_CLASS_DISSIMILAR |
Query dissimilar samples from the same class of the input sample. |