Package cloud.ai.large_models.vision

Index

EmbedVideoResponse

Video embedding response.

Fields
video_embeddings[]

Value

The embedding vector for the video.

FilteredText

Details for filtered input text.

Fields
prompt

string

Input prompt

confidence

Confidence

Filtered category

score

double

Score for category

category

RaiCategory

Confidence level

Confidence

Enums
CONFIDENCE_UNSPECIFIED
CONFIDENCE_LOW
CONFIDENCE_MEDIUM
CONFIDENCE_HIGH

GenerateVideoResponse

Generate video response.

Fields
generated_samples[]

Media

The generates samples.

rai_media_filtered_count

int32

Returns if any videos were filtered due to RAI policies.

rai_media_filtered_reasons[]

string

Returns rai failure reasons if any.

rai_text_filtered_reason

FilteredText

Returns filtered text rai info.

rai_error_message

string

Returns rai error message for filtered videos.

Image

Image.

Fields
encoding

string

Image encoding, encoded as "image/png" or "image/jpg".

image_rai_scores

ImageRAIScores

RAI scores for generated image.

rai_info

RaiInfo

RAI info for image.

semantic_filter_response

SemanticFilterResponse

Semantic filter info for image.

text

string

Text/Expanded text input for imagen.

Union field content.

content can be only one of the following:

image

bytes

Raw bytes.

uri

string

Path to another storage (typically Google Cloud Storage).

ImageRAIScores

RAI scores for generated image returned.

Fields
agile_watermark_detection_score

double

Agile watermark score for image.

Media

Media.

Fields

Union field type.

type can be only one of the following:

image

Image

Image.

video

Video

Video

MediaGenerateContentResponse

Generate media content response

Fields
response

GenerateMultiModalResponse

Response to the user's request.

NamedBoundingBox

Fields
x1

float

x2

float

y1

float

y2

float

classes[]

string

entities[]

string

scores[]

float

RaiCategory

Enums
RAI_CATEGORY_UNSPECIFIED
OBSCENE
SEXUALLY_EXPLICIT Porn
IDENTITY_ATTACK Hate
VIOLENCE_ABUSE
CSAI
SPII
CELEBRITY
FACE_IMG
WATERMARK_IMG
MEMORIZATION_IMG
CSAI_IMG
PORN_IMG
VIOLENCE_IMG
CHILD_IMG
TOXIC
SENSITIVE_WORD
PERSON_IMG
ICA_IMG
SEXUAL_IMG
IU_IMG
RACY_IMG
PEDO_IMG
DEATH_HARM_TRAGEDY SafetyAttributes returned but not filtered on
HEALTH
FIREARMS_WEAPONS
RELIGIOUS_BELIEF
ILLICIT_DRUGS
WAR_CONFLICT
POLITICS
HATE_SYMBOL_IMG End of list
CHILD_TEXT
DANGEROUS_CONTENT Text category from SafetyCat v3
RECITATION_TEXT
CELEBRITY_IMG

RaiInfo

Fields
rai_categories[]

string

List of rai categories' information to return

scores[]

float

List of rai scores mapping to the rai categories. Rounded to 1 decimal place.

ReasonVideoResponse

Video reasoning response.

Fields
responses[]

TextResponse

Generated text responses. The generated responses for different segments within the same video.

TextResponse

Contains text that is the response of the video captioning.

Fields
text

string

Text information

relative_temporal_partition

RelativeTemporalPartition

Partition of the caption's video in time. This field is intended for video captioning. To represent the start time and end time of the caption's video.

RelativeTemporalPartition

For ease of use, assume that the start_offset is inclusive and the end_offset is exclusive. In mathematical terms, the partition would be written as [start_offset, end_offset).

Fields
start_offset

Duration

Start time offset of the partition.

end_offset

Duration

End time offset of the partition.

SemanticFilterResponse

Fields
passed_semantic_filter

bool

This response is added when semantic filter config is turned on in EditConfig. It reports if this image is passed semantic filter response. If passed_semantic_filter is false, the bounding box information will be populated for user to check what caused the semantic filter to fail.

named_bounding_boxes[]

NamedBoundingBox

Class labels of the bounding boxes that failed the semantic filtering. Bounding box coordinates.

Video

Video

Fields

Union field content.

content can be only one of the following:

video

bytes

Raw bytes.

uri

string

Path to another storage (typically Google Cloud Storage).