Summary of entries of Classes for aiplatform.
Classes
Candidate
A response candidate generated by the model.
ChatSession
Chat session holds the chat history.
Content
The multi-part content of a message.
Usage:
response = model.generate_content(contents=[
Content(role="user", parts=[Part.from_text("Why is sky blue?")])
])
```
FinishReason
The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.
FunctionDeclaration
A representation of a function declaration.
Usage: Create function declaration and tool:
get_current_weather_func = generative_models.FunctionDeclaration(
name="get_current_weather",
description="Get the current weather in a given location",
parameters={
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": [
"celsius",
"fahrenheit",
]
}
},
"required": [
"location"
]
},
)
weather_tool = generative_models.Tool(
function_declarations=[get_current_weather_func],
)
```
Use tool in `GenerativeModel.generate_content`:
```
model = GenerativeModel("gemini-pro")
print(model.generate_content(
"What is the weather like in Boston?",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
))
```
Use tool in chat:
```
model = GenerativeModel(
"gemini-pro",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
)
chat = model.start_chat()
print(chat.send_message("What is the weather like in Boston?"))
print(chat.send_message(
Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
),
))
```
GenerationConfig
Parameters for the generation.
GenerationResponse
The response from the model.
GenerativeModel
Initializes GenerativeModel.
Usage:
model = GenerativeModel("gemini-pro")
print(model.generate_content("Hello"))
```
HarmBlockThreshold
Probability based thresholds levels for blocking.
HarmCategory
Harm categories that will block the content.
Image
The image that can be sent to a generative model.
Part
A part of a multi-part Content message.
Usage:
text_part = Part.from_text("Why is sky blue?")
image_part = Part.from_image(Image.load_from_file("image.jpg"))
video_part = Part.from_uri(uri="gs://.../video.mp4", mime_type="video/mp4")
function_response_part = Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
)
response1 = model.generate_content([text_part, image_part])
response2 = model.generate_content(video_part)
response3 = chat.send_message(function_response_part)
```
ResponseValidationError
Common base class for all non-exit exceptions.
SafetySetting
Parameters for the generation.
HarmBlockMethod
Probability vs severity.
HarmBlockThreshold
Probability based thresholds levels for blocking.
HarmCategory
Harm categories that will block the content.
Tool
A collection of functions that the model may use to generate response.
Usage: Create tool from function declarations:
get_current_weather_func = generative_models.FunctionDeclaration(...)
weather_tool = generative_models.Tool(
function_declarations=[get_current_weather_func],
)
```
Use tool in `GenerativeModel.generate_content`:
```
model = GenerativeModel("gemini-pro")
print(model.generate_content(
"What is the weather like in Boston?",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
))
```
Use tool in chat:
```
model = GenerativeModel(
"gemini-pro",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
)
chat = model.start_chat()
print(chat.send_message("What is the weather like in Boston?"))
print(chat.send_message(
Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
),
))
```
ToolConfig
Config shared for all tools provided in the request.
Usage: Create ToolConfig
tool_config = ToolConfig(
function_calling_config=ToolConfig.FunctionCallingConfig(
mode=ToolConfig.FunctionCallingConfig.Mode.ANY,
allowed_function_names=["get_current_weather_func"],
))
```
Use ToolConfig in `GenerativeModel.generate_content`:
```
model = GenerativeModel("gemini-pro")
print(model.generate_content(
"What is the weather like in Boston?",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
tool_config=tool_config,
))
```
Use ToolConfig in chat:
```
model = GenerativeModel(
"gemini-pro",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
tool_config=tool_config,
)
chat = model.start_chat()
print(chat.send_message("What is the weather like in Boston?"))
print(chat.send_message(
Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
),
))
```
grounding
Grounding namespace.
GoogleSearchRetrieval
Tool to retrieve public web data for grounding, powered by Google Search.
ChatMessage
A chat message.
ChatModel
ChatModel represents a language model that is capable of chat.
Examples::
chat_model = ChatModel.from_pretrained("chat-bison@001")
chat = chat_model.start_chat(
context="My name is Ned. You are my personal assistant. My favorite movies are Lord of the Rings and Hobbit.",
examples=[
InputOutputTextPair(
input_text="Who do you work for?",
output_text="I work for Ned.",
),
InputOutputTextPair(
input_text="What do I like?",
output_text="Ned likes watching movies.",
),
],
temperature=0.3,
)
chat.send_message("Do you know any cool events this weekend?")
ChatSession
ChatSession represents a chat session with a language model.
Within a chat session, the model keeps context and remembers the previous conversation.
CodeChatModel
CodeChatModel represents a model that is capable of completing code.
.. rubric:: Examples
code_chat_model = CodeChatModel.from_pretrained("codechat-bison@001")
code_chat = code_chat_model.start_chat( context="I'm writing a large-scale enterprise application.", max_output_tokens=128, temperature=0.2, )
code_chat.send_message("Please help write a function to calculate the min of two numbers")
CodeChatSession
CodeChatSession represents a chat session with code chat language model.
Within a code chat session, the model keeps context and remembers the previous converstion.
CodeGenerationModel
Creates a LanguageModel.
This constructor should not be called directly.
Use LanguageModel.from_pretrained(model_name=...) instead.
GroundingSource
GroundingSource()
InlineContext
InlineContext represents a grounding source using provided inline context. .. attribute:: inline_context
The content used as inline context.
:type: str
VertexAISearch
VertexAISearchDatastore represents a grounding source using Vertex AI Search datastore .. attribute:: data_store_id
Data store ID of the Vertex AI Search datastore.
:type: str
WebSearch
WebSearch represents a grounding source using public web search. .. attribute:: disable_attribution
If set to True, skip finding claim attributions (i.e not generate grounding citation). Default: False.
:type: bool
InputOutputTextPair
InputOutputTextPair represents a pair of input and output texts.
TextEmbedding
Text embedding vector and statistics.
TextEmbeddingInput
Structural text embedding input.
TextEmbeddingModel
Creates a LanguageModel.
This constructor should not be called directly.
Use LanguageModel.from_pretrained(model_name=...) instead.
TextGenerationModel
Creates a LanguageModel.
This constructor should not be called directly.
Use LanguageModel.from_pretrained(model_name=...) instead.
TextGenerationResponse
TextGenerationResponse represents a response of a language model. .. attribute:: text
The generated text
:type: str
_TunableModelMixin
Model that can be tuned with supervised fine tuning (SFT).
AutomaticFunctionCallingResponder
Responder that automatically responds to model's function calls.
CallableFunctionDeclaration
A function declaration plus a function.
Candidate
A response candidate generated by the model.
ChatSession
Chat session holds the chat history.
Content
The multi-part content of a message.
Usage:
response = model.generate_content(contents=[
Content(role="user", parts=[Part.from_text("Why is sky blue?")])
])
```
FinishReason
The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.
FunctionDeclaration
A representation of a function declaration.
Usage: Create function declaration and tool:
get_current_weather_func = generative_models.FunctionDeclaration(
name="get_current_weather",
description="Get the current weather in a given location",
parameters={
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": [
"celsius",
"fahrenheit",
]
}
},
"required": [
"location"
]
},
)
weather_tool = generative_models.Tool(
function_declarations=[get_current_weather_func],
)
```
Use tool in `GenerativeModel.generate_content`:
```
model = GenerativeModel("gemini-pro")
print(model.generate_content(
"What is the weather like in Boston?",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
))
```
Use tool in chat:
```
model = GenerativeModel(
"gemini-pro",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
)
chat = model.start_chat()
print(chat.send_message("What is the weather like in Boston?"))
print(chat.send_message(
Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
),
))
```
GenerationConfig
Parameters for the generation.
GenerationResponse
The response from the model.
GenerativeModel
Initializes GenerativeModel.
Usage:
model = GenerativeModel("gemini-pro")
print(model.generate_content("Hello"))
```
HarmBlockThreshold
Probability based thresholds levels for blocking.
HarmCategory
Harm categories that will block the content.
Image
The image that can be sent to a generative model.
Part
A part of a multi-part Content message.
Usage:
text_part = Part.from_text("Why is sky blue?")
image_part = Part.from_image(Image.load_from_file("image.jpg"))
video_part = Part.from_uri(uri="gs://.../video.mp4", mime_type="video/mp4")
function_response_part = Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
)
response1 = model.generate_content([text_part, image_part])
response2 = model.generate_content(video_part)
response3 = chat.send_message(function_response_part)
```
ResponseBlockedError
Common base class for all non-exit exceptions.
ResponseValidationError
Common base class for all non-exit exceptions.
SafetySetting
Parameters for the generation.
HarmBlockMethod
Probability vs severity.
HarmBlockThreshold
Probability based thresholds levels for blocking.
HarmCategory
Harm categories that will block the content.
Tool
A collection of functions that the model may use to generate response.
Usage: Create tool from function declarations:
get_current_weather_func = generative_models.FunctionDeclaration(...)
weather_tool = generative_models.Tool(
function_declarations=[get_current_weather_func],
)
```
Use tool in `GenerativeModel.generate_content`:
```
model = GenerativeModel("gemini-pro")
print(model.generate_content(
"What is the weather like in Boston?",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
))
```
Use tool in chat:
```
model = GenerativeModel(
"gemini-pro",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
)
chat = model.start_chat()
print(chat.send_message("What is the weather like in Boston?"))
print(chat.send_message(
Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
),
))
```
ToolConfig
Config shared for all tools provided in the request.
Usage: Create ToolConfig
tool_config = ToolConfig(
function_calling_config=ToolConfig.FunctionCallingConfig(
mode=ToolConfig.FunctionCallingConfig.Mode.ANY,
allowed_function_names=["get_current_weather_func"],
))
```
Use ToolConfig in `GenerativeModel.generate_content`:
```
model = GenerativeModel("gemini-pro")
print(model.generate_content(
"What is the weather like in Boston?",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
tool_config=tool_config,
))
```
Use ToolConfig in chat:
```
model = GenerativeModel(
"gemini-pro",
# You can specify tools when creating a model to avoid having to send them with every request.
tools=[weather_tool],
tool_config=tool_config,
)
chat = model.start_chat()
print(chat.send_message("What is the weather like in Boston?"))
print(chat.send_message(
Part.from_function_response(
name="get_current_weather",
response={
"content": {"weather_there": "super nice"},
}
),
))
```
ChatMessage
A chat message.
CountTokensResponse
The response from a count_tokens request. .. attribute:: total_tokens
The total number of tokens counted across all instances passed to the request.
:type: int
EvaluationClassificationMetric
The evaluation metric response for classification metrics.
EvaluationMetric
The evaluation metric response.
EvaluationQuestionAnsweringSpec
Spec for question answering model evaluation tasks.
EvaluationTextClassificationSpec
Spec for text classification model evaluation tasks.
EvaluationTextGenerationSpec
Spec for text generation model evaluation tasks.
EvaluationTextSummarizationSpec
Spec for text summarization model evaluation tasks.
InputOutputTextPair
InputOutputTextPair represents a pair of input and output texts.
TextEmbedding
Text embedding vector and statistics.
TextEmbeddingInput
Structural text embedding input.
TextGenerationResponse
TextGenerationResponse represents a response of a language model. .. attribute:: text
The generated text
:type: str
TuningEvaluationSpec
Specification for model evaluation to perform during tuning.
LangchainAgent
A Langchain Agent.
Reference:
Queryable
Protocol for Reasoning Engine applications that can be queried.
ReasoningEngine
Represents a Vertex AI Reasoning Engine resource.
GeneratedImage
Generated image.
Image
Image.
ImageCaptioningModel
Generates captions from image.
Examples::
model = ImageCaptioningModel.from_pretrained("imagetext@001")
image = Image.load_from_file("image.png")
captions = model.get_captions(
image=image,
# Optional:
number_of_results=1,
language="en",
)
ImageGenerationModel
Generates images from text prompt.
Examples::
model = ImageGenerationModel.from_pretrained("imagegeneration@002")
response = model.generate_images(
prompt="Astronaut riding a horse",
# Optional:
number_of_images=1,
seed=0,
)
response[0].show()
response[0].save("image1.png")
ImageGenerationResponse
Image generation response.
ImageQnAModel
Answers questions about an image.
Examples::
model = ImageQnAModel.from_pretrained("imagetext@001")
image = Image.load_from_file("image.png")
answers = model.ask_question(
image=image,
question="What color is the car in this image?",
# Optional:
number_of_results=1,
)
ImageTextModel
Generates text from images.
Examples::
model = ImageTextModel.from_pretrained("imagetext@001")
image = Image.load_from_file("image.png")
captions = model.get_captions(
image=image,
# Optional:
number_of_results=1,
language="en",
)
answers = model.ask_question(
image=image,
question="What color is the car in this image?",
# Optional:
number_of_results=1,
)
MultiModalEmbeddingModel
Generates embedding vectors from images and videos.
Examples::
model = MultiModalEmbeddingModel.from_pretrained("multimodalembedding@001")
image = Image.load_from_file("image.png")
video = Video.load_from_file("video.mp4")
embeddings = model.get_embeddings(
image=image,
video=video,
contextual_text="Hello world",
)
image_embedding = embeddings.image_embedding
video_embeddings = embeddings.video_embeddings
text_embedding = embeddings.text_embedding
MultiModalEmbeddingResponse
The multimodal embedding response.
Video
Video.
VideoEmbedding
Embeddings generated from video with offset times.
VideoSegmentConfig
The specific video segments (in seconds) the embeddings are generated for.
WatermarkVerificationModel
Verifies if an image has a watermark.
WatermarkVerificationResponse
WatermarkVerificationResponse(_prediction_response: Any, watermark_verification_result: Optional[str] = None)
ModelMonitor
Initializer for ModelMonitor.
ModelMonitoringJob
Initializer for ModelMonitoringJob.
Example Usage:
my_monitoring_job = aiplatform.ModelMonitoringJob(
model_monitoring_job_name='projects/123/locations/us-central1/modelMonitors/\
my_model_monitor_id/modelMonitoringJobs/my_monitoring_job_id'
)
or
my_monitoring_job = aiplatform.aiplatform.ModelMonitoringJob(
model_monitoring_job_name='my_monitoring_job_id',
model_monitor_id='my_model_monitor_id',
)
DataDriftSpec
Data drift monitoring spec.
Data drift measures the distribution distance between the current dataset and a baseline dataset. A typical use case is to detect data drift between the recent production serving dataset and the training dataset, or to compare the recent production dataset with a dataset from a previous period.
.. rubric:: Example
feature_drift_spec=DataDriftSpec( features=["feature1"] categorical_metric_type="l_infinity", numeric_metric_type="jensen_shannon_divergence", default_categorical_alert_threshold=0.01, default_numeric_alert_threshold=0.02, feature_alert_thresholds={"feature1":0.02, "feature2":0.01}, )
FeatureAttributionSpec
Feature attribution spec.
.. rubric:: Example
feature_attribution_spec=FeatureAttributionSpec( features=["feature1"] default_alert_threshold=0.01, feature_alert_thresholds={"feature1":0.02, "feature2":0.01}, batch_dedicated_resources=BatchDedicatedResources( starting_replica_count=1, max_replica_count=2, machine_spec=my_machine_spec, ), )
FieldSchema
Field Schema.
The class identifies the data type of a single feature, which combines together to form the Schema for different fields in ModelMonitoringSchema.
ModelMonitoringSchema
Initializer for ModelMonitoringSchema.
MonitoringInput
Model monitoring data input spec.
NotificationSpec
Initializer for NotificationSpec.
ObjectiveSpec
Initializer for ObjectiveSpec.
OutputSpec
Initializer for OutputSpec.
TabularObjective
Initializer for TabularObjective.
GeneratedImage
Generated image.
Image
Image.
ImageCaptioningModel
Generates captions from image.
Examples::
model = ImageCaptioningModel.from_pretrained("imagetext@001")
image = Image.load_from_file("image.png")
captions = model.get_captions(
image=image,
# Optional:
number_of_results=1,
language="en",
)
ImageGenerationModel
Generates images from text prompt.
Examples::
model = ImageGenerationModel.from_pretrained("imagegeneration@002")
response = model.generate_images(
prompt="Astronaut riding a horse",
# Optional:
number_of_images=1,
seed=0,
)
response[0].show()
response[0].save("image1.png")
ImageGenerationResponse
Image generation response.
ImageQnAModel
Answers questions about an image.
Examples::
model = ImageQnAModel.from_pretrained("imagetext@001")
image = Image.load_from_file("image.png")
answers = model.ask_question(
image=image,
question="What color is the car in this image?",
# Optional:
number_of_results=1,
)
ImageTextModel
Generates text from images.
Examples::
model = ImageTextModel.from_pretrained("imagetext@001")
image = Image.load_from_file("image.png")
captions = model.get_captions(
image=image,
# Optional:
number_of_results=1,
language="en",
)
answers = model.ask_question(
image=image,
question="What color is the car in this image?",
# Optional:
number_of_results=1,
)
MultiModalEmbeddingModel
Generates embedding vectors from images and videos.
Examples::
model = MultiModalEmbeddingModel.from_pretrained("multimodalembedding@001")
image = Image.load_from_file("image.png")
video = Video.load_from_file("video.mp4")
embeddings = model.get_embeddings(
image=image,
video=video,
contextual_text="Hello world",
)
image_embedding = embeddings.image_embedding
video_embeddings = embeddings.video_embeddings
text_embedding = embeddings.text_embedding
MultiModalEmbeddingResponse
The multimodal embedding response.
Video
Video.
VideoEmbedding
Embeddings generated from video with offset times.
VideoSegmentConfig
The specific video segments (in seconds) the embeddings are generated for.
Modules
_language_models
Classes for working with language models.
generative_models
Classes for working with the Gemini models.
language_models
Classes for working with language models.
vision_models
Classes for working with vision models.