Classes
ClientError
ClientError is thrown when http 4XX status is received. For details please refer to https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#client_error_responses
GoogleAuthError
GoogleAuthError is thrown when there is authentication issue with the request
GoogleGenerativeAIError
GoogleGenerativeAIError is thrown when http response is not ok and status code is not 4XX For details please refer to https://developer.mozilla.org/en-US/docs/Web/HTTP/Status
VertexAI
Base class for authenticating to Vertex, creates the preview namespace. Users use getGenerativeModel method to start using Google's generative models in GA.
Interfaces
BaseModelParams
Base params for initializing a model or calling GenerateContent. {SafetySetting[]} - [safety_settings] Array of SafetySetting {GenerationConfig} - [generation_config] GenerationConfig
BasePart
A part of a turn in a conversation with the model with a fixed MIME type. It has one of the following mutually exclusive fields: 1. text 2. inline_data 3. file_data 4. functionResponse 5. functionCall
CitationMetadata
A collection of source attributions for a piece of content. {CitationSource[]} - citationSources. Array of CitationSource
CitationSource
Source attributions for content. {number} - [startIndex] Start index into the content. {number} - [endIndex] End index into the content. {string} - [url] Url reference of the attribution. {string} - [license] License of the attribution.
Content
The base structured datatype containing multi-part content of a message. {Part[]} - parts. Array of Part {string} - [role]. The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.
CountTokensRequest
Params used to call the countTokens method. {Content[]} - contents. Array of Content
CountTokensResponse
Response returned from countTokens method. {number} - totalTokens. The total number of tokens counted across all instances from the request. {number} - [totalBillableCharacters]. The total number of billable characters counted across all instances from the request.
FileData
URI based data. {string} - mime_type. The IANA standard MIME type of the source data. {string} - file_uri. URI of the file.
FileDataPart
A file data part of a conversation with the model. {never} - [text]. text is not expected for FileDataPart. {never} - [inline_data]. inline_data is not expected for FileDataPart. {FileData} - file_data. Only this property is expected for FileDataPart. FileData {never} - [functionResponse]. functionResponse is not expected for FileDataPart. {never} - [functionCall]. functionCall is not expected for FileDataPart.
FunctionCall
A predicted FunctionCall returned from the model that contains a string representating the FunctionDeclaration.name with the parameters and their values. {string} - name The name of the function specified in FunctionDeclaration.name. {object} - args The arguments to pass to the function.
FunctionCallPart
A function call part of a conversation with the model. {never} - [text]. text is not expected for FunctionResponsePart. {never} - [inline_data]. inline_data is not expected for FunctionResponsePart. {never} - [file_data]. file_data is not expected for FunctionResponsePart. FileData {never} - [functionResponse]. functionResponse is not expected for FunctionResponsePart. {FunctionCall} - functionCall. only functionCall is expected for FunctionCallPart.
FunctionDeclaration
Structured representation of a function declaration as defined by the [OpenAPI 3.0 specification](https://spec.openapis.org/oas/v3.0.3). Included in this declaration are the function name and parameters. This FunctionDeclaration is a representation of a block of code that can be used as a Tool by the model and executed by the client. {string} - name The name of the function to call. Must start with a letter or an underscore. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a max length of 64. {string} - description Description and purpose of the function. Model uses it to decide how and whether to call the function. {FunctionDeclarationSchema} - parameters Describes the parameters to this function in JSON Schema Object format. Reflects the Open API 3.03 Parameter Object. string Key: the name of the parameter. Parameter names are case sensitive. Schema Value: the Schema defining the type used for the parameter. For function with no parameters, this can be left unset. Example with 1 required and 1 optional parameter: type: OBJECT properties:
param1:
type: STRING param2:
type: INTEGER required:
- param1
FunctionDeclarationSchema
Schema for parameters passed to [FunctionDeclaration.parameters]
FunctionDeclarationSchemaProperty
Schema is used to define the format of input/output data. Represents a select subset of an OpenAPI 3.0 schema object. More fields may be added in the future as needed.
FunctionDeclarationsTool
A FunctionDeclarationsTool is a piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. {object} - function_declarations One or more function declarations to be passed to the model along with the current user query. Model may decide to call a subset of these functions by populating [FunctionCall][content.part.function_call] in the response. User should provide a [FunctionResponse][content.part.function_response] for each function call in the next turn. Based on the function responses, Model will generate the final response back to the user. Maximum 64 function declarations can be provided.
FunctionResponse
The result output of a FunctionCall that contains a string representing the FunctionDeclaration.name and a structured JSON object containing any output from the function call. It is used as context to the model. {string} - name The name of the function specified in FunctionDeclaration.name. {object} - response The expected response from the model.
FunctionResponsePart
A function response part of a conversation with the model. {never} - [text]. text is not expected for FunctionResponsePart. {never} - [inline_data]. inline_data is not expected for FunctionResponsePart. {FileData} - [file_data]. file_data is not expected for FunctionResponsePart. FileData {never} - functionResponse. only functionResponse is expected for FunctionResponsePart. {never} - [functionCall]. functionCall is not expected for FunctionResponsePart.
GenerateContentCandidate
A response candidate generated from the model. {Content} - content. Content {number} - [index]. The index of the candidate in the GenerateContentResponse {FinishReason} - [finishReason]. FinishReason {string} - [finishMessage]. {SafetyRating[]} - [safetyRatings]. Array of SafetyRating {CitationMetadata} - [citationMetadata]. CitationMetadata {GroundingMetadata} - [groundingMetadata]. GroundingMetadata
GenerateContentRequest
Params used to call the generateContent method. {Content[]} - contents. Array of Content
GenerateContentResponse
Response from the model supporting multiple candidates {GenerateContentCandidate} - candidates. GenerateContentCandidate {PromptFeedback} - [promptFeedback]. This is only populated if there are no candidates due to a safety block PromptFeedback {UsageMetadata} - [usageMetadata]. UsageMetadata
GenerateContentResult
Wrapper for respones from a generateContent request {GenerateContentResponse} - response. All GenerateContentResponses received so far GenerateContentResponse
GenerationConfig
Configuration options for model generation and outputs {number} - [candidate_count] Number of candidates to generate. {string[]} - [stop_sequences] Stop sequences. {number} - [max_output_tokens] The maximum number of output tokens to generate per message. {number} - [temperature] Controls the randomness of predictions. {number} - [top_p] If specified, nucleus sampling will be used. {number} - [top_k] If specified, top-k sampling will be used.
GenerativeContentBlob
Raw media bytes sent directly in the request. Text should not be sent as raw bytes. {string} - mime_type. The MIME type of the source data. The only accepted values: "image/png" or "image/jpeg". {string} - data. data must be base64 string
GetGenerativeModelParams
{string} model - model name {string} project - project The Google Cloud project to use for the request {string} location - The Google Cloud project location to use for the request {GoogleAuth} googleAuth - GoogleAuth class instance that handles authentication. Details about GoogleAuth is referred to https://github.com/googleapis/google-auth-library-nodejs/blob/main/src/auth/googleauth.ts {string} - [apiEndpoint] The base Vertex AI endpoint to use for the request. If not provided, the default regionalized endpoint (i.e. us-central1-aiplatform.googleapis.com) will be used. {GenerationConfig} [generation_config] - GenerationConfig {SafetySetting[]} [safety_settings] - SafetySetting {Tool[]} [tools] - Tool
GoogleSearchRetrieval
Tool to retrieve public web data for grounding, powered by Google. {boolean} - [disableAttribution] Disable using the result from this tool in detecting grounding attribution. This does not affect how the result is given to the model for generation.
GoogleSearchRetrievalTool
GroundingAttribution
Grounding attribution. {GroundingAttributionWeb} - [web] Attribution from the web. {GroundingAttributionSegment} - [segment] Segment of the content this attribution belongs to. {number} - [confidenceScore] Confidence score of the attribution. Ranges from 0 to 1. 1 is the most confident.
GroundingAttributionSegment
Segment of the content this attribution belongs to. {number} - [part_index] The index of a Part object within its parent Content object. {number} - [startIndex] Start index in the given Part, measured in bytes. Offset from the start of the Part, inclusive, starting at zero. {number} - [endIndex] End index in the given Part, measured in bytes. Offset from the start of the Part, exclusive, starting at zero.
GroundingAttributionWeb
Attribution from the web. {string} - [uri] URI reference of the attribution. {string} - [title] Title of the attribution.
GroundingMetadata
A collection of grounding attributions for a piece of content. {string[]} - [webSearchQueries]. Web search queries for the following-up web search. {GroundingAttribution[]} - [groundingAttributions]. Array of GroundingAttribution
InlineDataPart
An inline data part of a conversation with the model. {never} - [text]. text is not expected for InlineDataPart. {GenerativeContentBlob} - inline_data. Only this property is expected for InlineDataPart. GenerativeContentBlob {never} - [file_data]. file_data is not expected for InlineDataPart. {never} - [functionResponse]. functionResponse is not expected for InlineDataPart. {never} - [functionCall]. functionCall is not expected for InlineDataPart.
ModelParams
Configuration for initializing a model, for example via getGenerativeModel {string} model - model name.
PromptFeedback
Content filter results for a prompt sent in the request. {BlockedReason} - block_reason. {SafetyRating[]} - safety_ratings. Array of SafetyRating {string} - block_reason_message. A readable block reason message.
Retrieval
Defines a retrieval tool that model can call to access external knowledge. {VertexAISearch} - [vertexAiSearch] Set to use data source powered by Vertex AI Search. {boolean} - [disableAttribution] Disable using the result from this tool in detecting grounding attribution. This does not affect how the result is given to the model for generation.
RetrievalTool
SafetyRating
Safety rating corresponding to the generated content. {HarmCategory} - category. HarmCategory {HarmProbability} - probability. HarmProbability
SafetySetting
Safety feedback for an entire request. {HarmCategory} - category. HarmCategory {HarmBlockThreshold} - threshold. HarmBlockThreshold
StartChatParams
Params to initiate a multiturn chat with the model via startChat {Content[]} - [history] history of the chat session. Content {SafetySetting[]} - [safety_settings] Array of SafetySetting {GenerationConfig} - [generation_config] GenerationConfig
StartChatSessionRequest
All params passed to initiate multiturn chat via startChat {string} project - project The Google Cloud project to use for the request {string} location - The Google Cloud project location to use for the request
StreamGenerateContentResult
Wrapper for respones from a generateContent method when steam parameter is true {Promise
TextPart
A text part of a conversation with the model. {string} - text. Only this propery is expected for TextPart. {never} - [inline_data]. inline_data is not expected for TextPart. {never} - [file_data]. file_data is not expected for TextPart. {never} - [functionResponse]. functionResponse is not expected for TextPart. {never} - [functionCall]. functionCall is not expected for TextPart.
UsageMetadata
Usage metadata about response(s). {number} - [prompt_token_count]. Number of tokens in the request. {number} - [candidates_token_count]. Number of tokens in the response(s). {number} - [totalTokenCount]. Total number of tokens.
VertexAISearch
Retrieve from Vertex AI Search datastore for grounding. See https://cloud.google.com/vertex-ai-search-and-conversation {string} - [datastore] Fully-qualified Vertex AI Search's datastore resource ID. projects/<>/locations/<>/collections/<>/dataStores/<>
VertexInit
Params used to initialize the Vertex SDK @param{string} project - the project name of your Google Cloud project. It is not the numeric project ID. @param{string} location - the location of your project. @param{string} [apiEndpoint] - If not specified, a default value will be resolved by SDK.
Enums
BlockedReason
{string} The reason why the reponse is blocked. Values: BLOCKED_REASON_UNSPECIFIED Unspecified blocked reason. SAFETY Candidates blocked due to safety. OTHER Candidates blocked due to other reason.
FinishReason
{string} The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens. Values: FINISH_REASON_UNSPECIFIED The finish reason is unspecified. STOP: Natural stop point of the model or provided stop sequence. MAX_TOKENS: The maximum number of tokens as specified in the request was reached. SAFETY: The token generation was stopped as the response was flagged for safety reasons. NOTE: When streaming the Candidate.content will be empty if content filters blocked the output. RECITATION: The token generation was stopped as the response was flagged for unauthorized citations. OTHER: All other reasons that stopped the token generation
FunctionDeclarationSchemaType
Contains the list of OpenAPI data types as defined by https://swagger.io/docs/specification/data-models/data-types/
HarmBlockThreshold
{string} Probability based thresholds levels for blocking. Values: HARM_BLOCK_THRESHOLD_UNSPECIFIED: Unspecified harm block threshold. BLOCK_LOW_AND_ABOVE: Block low threshold and above (i.e. block more). BLOCK_MEDIUM_AND_ABOVE: Block medium threshold and above. BLOCK_ONLY_HIGH: Block only high threshold (i.e. block less). BLOCK_NONE: Block none.
HarmCategory
{string} Harm categories that will block the content. Values: HARM_CATEGORY_UNSPECIFIED: The harm category is unspecified. HARM_CATEGORY_HATE_SPEECH: The harm category is hate speech. HARM_CATEGORY_DANGEROUS_CONTENT: The harm category is dangerous content. HARM_CATEGORY_HARASSMENT: The harm category is harassment. HARM_CATEGORY_SEXUALLY_EXPLICIT: The harm category is sexually explicit content.
HarmProbability
{string} Harm probability levels in the content. Values: HARM_PROBABILITY_UNSPECIFIED: Harm probability unspecified. NEGLIGIBLE: Negligible level of harm. LOW: Low level of harm. MEDIUM: Medium level of harm. HIGH: High level of harm.
Type Aliases
Part
export declare type Part = TextPart | InlineDataPart | FileDataPart | FunctionResponsePart | FunctionCallPart;A datatype containing media that is part of a multi-part Content message. A Part is a union type of TextPart, InlineDataPart, FileDataPart, and FunctionResponsePart. A Part has one of the following mutually exclusive fields: 1. text 2. inline_data 3. file_data 4. functionResponse
Tool
export declare type Tool = FunctionDeclarationsTool | RetrievalTool | GoogleSearchRetrievalTool;