Try Gemini 1.5 models, our newest multimodal models in Vertex AI, and see what you can build with a 1M token context window. Try Gemini 1.5 models, our newest multimodal models in Vertex AI, and see what you can build with a 1M token context window.

Package cloud.ai.large_models.vision

Index

FilteredText (message)
FilteredText.Confidence (enum)
GenerateVideoResponse (message)
Image (message)
ImageRAIScores (message)
Media (message)
NamedBoundingBox (message)
RaiCategory (enum)
RaiInfo (message)
SemanticFilterResponse (message)
Video (message)

FilteredText

Details for filtered input text.

Fields
`prompt`	`string` Input prompt
`confidence`	`Confidence` Filtered category
`score`	`double` Score for category
`category`	`RaiCategory` Confidence level

Confidence

Enums
`CONFIDENCE_UNSPECIFIED`
`CONFIDENCE_LOW`
`CONFIDENCE_MEDIUM`
`CONFIDENCE_HIGH`

GenerateVideoResponse

Generate video response.

Fields
`generated_samples[]`	`Media` The generates samples.
`rai_media_filtered_count`	`int32` Returns if any videos were filtered due to RAI policies.
`rai_media_filtered_reasons[]`	`string` Returns rai failure reasons if any.
`rai_text_filtered_reason`	`FilteredText` Returns filtered text rai info.
`rai_error_message`	`string` Returns rai error message for filtered videos.

Image

Image.

Fields
`encoding`	`string` Image encoding, encoded as "image/png" or "image/jpg".
`image_rai_scores`	`ImageRAIScores` RAI scores for generated image.
`rai_info`	`RaiInfo` RAI info for image.
`semantic_filter_response`	`SemanticFilterResponse` Semantic filter info for image.
`text`	`string` Text/Expanded text input for imagen.
Union field `content`. `content` can be only one of the following:
`image`	`bytes` Raw bytes.
`uri`	`string` Path to another storage (typically Google Cloud Storage).

ImageRAIScores

RAI scores for generated image returned.

Fields

Fields
`agile_watermark_detection_score`	`double` Agile watermark score for image.

agile_watermark_detection_score

double

Agile watermark score for image.

Media

Media.

Fields

Fields
Union field `type`. `type` can be only one of the following:
`image`	`Image` Image.
`video`	`Video` Video

Union field type.

type can be only one of the following:

image

Image

Image.

video

Video

Video

NamedBoundingBox

Fields
`x1`	`float`
`x2`	`float`
`y1`	`float`
`y2`	`float`
`classes[]`	`string`
`entities[]`	`string`
`scores[]`	`float`

RaiCategory

Enums
`RAI_CATEGORY_UNSPECIFIED`
`OBSCENE`
`SEXUALLY_EXPLICIT`	Porn
`IDENTITY_ATTACK`	Hate
`VIOLENCE_ABUSE`
`CSAI`
`SPII`
`CELEBRITY`
`FACE_IMG`
`WATERMARK_IMG`
`MEMORIZATION_IMG`
`CSAI_IMG`
`PORN_IMG`
`VIOLENCE_IMG`
`CHILD_IMG`
`TOXIC`
`SENSITIVE_WORD`
`PERSON_IMG`
`ICA_IMG`
`SEXUAL_IMG`
`IU_IMG`
`RACY_IMG`
`PEDO_IMG`
`DEATH_HARM_TRAGEDY`	SafetyAttributes returned but not filtered on
`HEALTH`
`FIREARMS_WEAPONS`
`RELIGIOUS_BELIEF`
`ILLICIT_DRUGS`
`WAR_CONFLICT`
`POLITICS`
`HATE_SYMBOL_IMG`	End of list
`CHILD_TEXT`
`DANGEROUS_CONTENT`	Text category from SafetyCat v3
`RECITATION_TEXT`
`CELEBRITY_IMG`
`WATERMARK_IMG_REMOVAL`	Error message when user attempts to remove watermark from editing image

RaiInfo

Fields

Fields
`rai_categories[]`	`string` List of rai categories' information to return
`scores[]`	`float` List of rai scores mapping to the rai categories. Rounded to 1 decimal place.

rai_categories[]

string

List of rai categories' information to return

scores[]

float

List of rai scores mapping to the rai categories. Rounded to 1 decimal place.

SemanticFilterResponse

Fields

Fields
`passed_semantic_filter`	`bool` This response is added when semantic filter config is turned on in EditConfig. It reports if this image is passed semantic filter response. If passed_semantic_filter is false, the bounding box information will be populated for user to check what caused the semantic filter to fail.
`named_bounding_boxes[]`	`NamedBoundingBox` Class labels of the bounding boxes that failed the semantic filtering. Bounding box coordinates.

passed_semantic_filter

bool

This response is added when semantic filter config is turned on in EditConfig. It reports if this image is passed semantic filter response. If passed_semantic_filter is false, the bounding box information will be populated for user to check what caused the semantic filter to fail.

named_bounding_boxes[]

NamedBoundingBox

Class labels of the bounding boxes that failed the semantic filtering. Bounding box coordinates.

Video

Fields

Fields
Union field `content`. `content` can be only one of the following:
`video`	`bytes` Raw bytes.
`uri`	`string` Path to another storage (typically Google Cloud Storage).

Union field content.

content can be only one of the following:

video

bytes

Raw bytes.

uri

string

Path to another storage (typically Google Cloud Storage).