Package google.cloud.aiplatform.v1beta1.schema.predict.prediction

Index

• NamedBoundingBox (message)
• SafetyAttributes (message)
• SafetyAttributes.DetectedLabels (message)
• SafetyAttributes.DetectedLabels.BoundingBox (message)
• SafetyAttributes.DetectedLabels.Entity (message)
• SemanticFilterResponse (message)
• TextEmbedding (message)
• TextEmbedding.Statistics (message)
• TextEmbeddingPredictionResult (message)
• VideoGenerationModelResult (message)
• VirtualTryOnModelResultProto (message)
• VirtualTryOnModelResultProto.Image (message)
• VisionEmbeddingModelResult (message)
• VisionEmbeddingModelResult.VideoEmbedding (message)
• VisionGenerativeModelResult (message)
• VisionGenerativeModelResult.Image (message)
• VisionReasoningModelResult (message)

NamedBoundingBox

NamedBoundingBox to track an annotated bounding box.

Fields
classes[]	string Annotated classes.
entities[]	string Annotated entities.
scores[]	float Annotated scores. Scores are normalized between [0, 1].
x1	float The top-left (x1, y1) corner's unnormalized coordinate.
x2	float
y1	float The bottom-right (y1, y2) corner's unnormalized coordinate.
y2	float

SafetyAttributes

Fields
categories[]	string List of RAI categories.
scores[]	float List of RAI scores.
detected_labels[]	DetectedLabels List of detected labels

DetectedLabels

Filters which return labels with confidence scores.

Fields
entities[]	Entity The list of detected entities for the rai signal.
rai_category	string The RAI category for the deteceted labels.

BoundingBox

An integer bounding box of the original pixel size for the detected labels.

Fields
x1	int32 The X coordinate of the top-left corner, in pixels.
y1	int32 The Y coordinate of the top-left corner, in pixels.
x2	int32 The X coordinate of the bottom-right corner, in pixels.
y2	int32 The Y coordinate of the bottom-right corner, in pixels.

Entity

The properties for a detected entity from the rai signal.

Fields
mid	string MID of the label
description	string Description of the label
score	float Confidence score of the label
bounding_box	BoundingBox Bounding box of the label
iou_score	float The intersection ratio between the detection bounding box and the mask.

SemanticFilterResponse

SemanticFilterResponse tracks the semantic filtering results if user turns on the semantic filtering in LVM image editing's editConfig.

Fields
named_bounding_boxes[]	NamedBoundingBox If semantic filtering is not passed, a list of named bounding boxes will be populated to report users the detected objects that failed semantic filtering.
passed_semantic_filter	bool Whether the semantic filtering is passed.

TextEmbedding

Fields
values[]	float The values field contains the embedding vectors corresponding to the words in the input text.
statistics	Statistics The statistics computed from the input text.

Statistics

Fields
token_count	int32 Number of tokens of the input text.
truncated	bool Indicates if the input text was longer than max allowed tokens and truncated.

TextEmbeddingPredictionResult

Prediction output format for Text Embedding. LINT.IfChange

Fields
embeddings	TextEmbedding The embeddings generated from the input text.

VideoGenerationModelResult

Prediction format for phenaki model.

Fields
gcs_uris[]	string List of gcs uris of the generated videos.

VirtualTryOnModelResultProto

Prediction format for the Virtual Try On model.

Fields
images[]	Image List of images bytes or gcs uris of the generated images.

Image

The generated image and metadata.

Fields
mime_type	string The MIME type of the content of the image. Only the images in the MIME types below are supported. - image/jpeg - image/png
Union field data. The image bytes or Cloud Storage URI to make the prediction on. data can be only one of the following:
bytes_base64_encoded	string Base64 encoded bytes string representing the image.
gcs_uri	string The Cloud Storage URI of the image.
rai_filtered_reason	string The reason generated images get filtered.

VisionEmbeddingModelResult

Prediction format for large vision model embedding api.

Fields
image_embedding	ListValue The 1024 dimension image embedding result from the provided image.
text_embedding	ListValue The 1024 dimension text embedding result from the provided text.
video_embeddings[]	VideoEmbedding Video embeddings.

VideoEmbedding

The video embedding message.

Fields
start_offset_sec	int32 The start offset of the video.
end_offset_sec	int32 The end offset of the video.
embedding	ListValue The 1024 dimension video embedding result from the provided video.

VisionGenerativeModelResult

Fields
images[]	Image List of images bytes or gcs uris of the generated images.

Image

Fields
mime_type	string The MIME type of the content of the image. Only the images in below listed MIME types are supported. - image/jpeg - image/gif - image/png - image/webp - image/bmp - image/tiff - image/vnd.microsoft.icon
prompt	string The rewritten prompt used for the image generation.
Union field data. The image bytes or Cloud Storage URI to make the prediction on. data can be only one of the following:
bytes_base64_encoded	string Base64 encoded bytes string representing the image.
gcs_uri	string
rai_filtered_reason	string The reason generated images get filtered.
content_type	string Input object content type
semantic_filter_response	SemanticFilterResponse Semantic filter results. This will report to users when semantic filter is turned on in editConfig and used for image inpainting.
safety_attributes	SafetyAttributes Safety attributes scores of the content.

VisionReasoningModelResult

The response format for lvm image and video captioning is as follows: 1. Image captioning: From the lvm image2text(PaLi) model, the responses are descriptions of the same image. 2. Video captioning: From the lvm video2text(Penguin) model, the responses are different segments within the same video. The response also contains the start and end offsets of the video segment. Video captioning response format: "[start_offset, end_offset) - text_response".

Fields
text_responses[]	string List of text responses in the given text language.