LlmModelSettings

Settings for LLM models.

JSON representation 
{
  "model": string,
  "promptText": string,
  "parameters": {
    object (Parameters)
  }
}
Fields
model

string

The selected LLM model.
promptText

string

The custom prompt to use.
parameters

object (Parameters)

Generative model parameters.

Parameters

Generative model parameters to control the model behavior.

JSON representation 
{
  "temperature": number,
  "inputTokenLimit": enum (InputTokenLimit),
  "outputTokenLimit": enum (OutputTokenLimit)
}
Fields
temperature

number

The temperature used for sampling during response generation. Value ranges from 0 to 1. Temperature controls the degree of randomness in token selection. Lower temperature means less randomness, while higher temperature means more randomness. Valid range: [0.0, 1.0]
inputTokenLimit

enum (InputTokenLimit)

The input token limit. This setting is currently only supported by playbooks.
outputTokenLimit

enum (OutputTokenLimit)

The output token limit. This setting is currently only supported by playbooks. Only one of outputTokenLimit and max_output_tokens is allowed to be set.

InputTokenLimit

The input token limits for 1 LLM call. For the limit of each model, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models for more information.

Enums
INPUT_TOKEN_LIMIT_UNSPECIFIED Limit not specified. Treated as 'INPUT_TOKEN_LIMIT_SHORT'.
INPUT_TOKEN_LIMIT_SHORT Input token limit up to 8k.
INPUT_TOKEN_LIMIT_MEDIUM Input token limit up to 32k.
INPUT_TOKEN_LIMIT_LONG Input token limit up to 100k.

OutputTokenLimit

The output token limits for 1 LLM call. The limits are subject to change. For the limit of each model, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models for more information.

Enums
OUTPUT_TOKEN_LIMIT_UNSPECIFIED Limit not specified.
OUTPUT_TOKEN_LIMIT_SHORT Input token limit up to 512 tokens.
OUTPUT_TOKEN_LIMIT_MEDIUM Input token limit up to 1k.
OUTPUT_TOKEN_LIMIT_LONG Input token limit up to 2k.