Package cloud.ai.nl.llm.proto.service

Index

Candidate

Fields
index

int32

Index of the candidate.

content

Content

Content of the candidate.

finish_reason

FinishReason

The reason why the model stopped generating tokens.

safety_ratings[]

SafetyRating

Safety ratings of the generated content.

finish_message

string

A string that describes the filtering behavior in more detail. Only filled when reason is set.

citation_metadata

CitationMetadata

Source attribution of the generated content.

grounding_metadata

GroundingMetadata

Grounding metadata. Combine with the facts list from response to generate grounding citations for this choice.

FinishReason

The reason why the model stopped generating tokens. If empty, the model has not stopped generating the tokens.

Enums
FINISH_REASON_UNSPECIFIED The finish reason is unspecified.
FINISH_REASON_STOP Natural stop point of the model or provided stop sequence.
FINISH_REASON_MAX_TOKENS The maximum number of tokens as specified in the request was reached.
FINISH_REASON_SAFETY The token generation was stopped as the response was flagged for safety reasons. NOTE: When streaming the Candidate.content will be empty if content filters blocked the output.
FINISH_REASON_RECITATION The token generation was stopped as the response was flagged for unauthorized citations.
FINISH_REASON_OTHER All other reasons that stopped the token generation.
FINISH_REASON_BLOCKLIST The token generation was stopped as the response was flagged for the terms which are included from the terminology blocklist.
FINISH_REASON_PROHIBITED_CONTENT The token generation was stopped as the response was flagged for the prohibited contents.
FINISH_REASON_SPII The token generation was stopped as the response was flagged for Sensitive Personally Identifiable Information (SPII) contents.

Citation

Source attributions for content.

Fields
start_index

int32

Start index into the content.

end_index

int32

End index into the content.

uri

string

Url reference of the attribution.

title

string

Title of the attribution.

license

string

License of the attribution.

publication_date

Date

Publication date of the attribution.

CitationMetadata

A collection of source attributions for a piece of content.

Fields
citations[]

Citation

List of citations.

Content

The content of a single message from a participant.

Fields
role

string

The role of the current conversation participant.

parts[]

Part

The parts of the message.

Fact

A condense version of WorldFact (assistant/boq/lamda/factuality/proto/factuality.proto) to propagate the essential information about the fact used in factuality to the upstream caller.

Fields
query

string

Query that is used to retrieve this fact.

title

string

If present, it refers to the title of this fact.

url

string

If present, this URL links to the webpage of the fact.

summary

string

If present, the summary/snippet of the fact.

FunctionCall

Function call details.

Fields
name

string

Required. The name of the function to call.

args

Struct

The function parameters and values in JSON format.

FunctionResponse

Function response details.

Fields
name

string

Required. The name of the function to call.

response

Struct

Required. The function response in JSON object format.

GenerateMultiModalResponse

Fields
candidates[]

Candidate

Possible candidate responses to the conversation up until this point.

reporting_metrics

ReportingMetrics

Billable prediction metrics.

prompt_feedback

PromptFeedback

Content filter results for a prompt sent in the request. Note: Sent only in the first stream chunk. Only happens when no candidates were generated due to content violations.

usage_metadata

UsageMetadata

Usage metadata about the response(s).

facts[]

Fact

External facts retrieved for factuality/grounding.

HarmCategory

Harm categories that will block the content.

Enums
HARM_CATEGORY_UNSPECIFIED The harm category is unspecified.
HARM_CATEGORY_HATE_SPEECH The harm category is hate speech.
HARM_CATEGORY_DANGEROUS_CONTENT The harm category is dengerous content.
HARM_CATEGORY_HARASSMENT The harm category is harassment.
HARM_CATEGORY_SEXUALLY_EXPLICIT The harm category is sexually explicit.

Part

A single part of a message.

Fields
lm_root_metadata

LMRootMetadata

Metadata provides extra info for building the LM Root request. Note: High enough tag number for internal only fields.

Union field part_data.

part_data can be only one of the following:

text

string

Text input.

inline_data

Blob

Inline bytes data

file_data

FileData

URI-based data.

function_call

FunctionCall

Function call data.

function_response

FunctionResponse

Function response data.

Union field metadata.

metadata can be only one of the following:

video_metadata

VideoMetadata

Video metadata. The metadata should only be specified while the video data is presented in inline_data or file_data.

document_metadata

DocumentMetadata

Document metadata. The metadata should only be used by the Cloud LLM when supporting document mime types. It will only be populated when this image input part is converted from a document input part.

Blob

Represents arbitrary blob data input.

Fields
mime_type

string

The mime type corresponding to this input.

data

bytes

Inline data.

original_file_data

FileData

Original file data where the blob comes from.

DocumentMetadata

Metadata describes the original input document content.

Fields
page_number

int32

The (1-indexed) page number of the image in the original document. The first page carries the original document content and mime type.

original_document_blob

Blob

The original document blob.

FileData

Represents file data.

Fields
mime_type

string

The mime type corresponding to this input.

file_uri

string

Inline data.

LMRootMetadata

Metadata provides extra info for building the LM Root request.

Fields
chunk_id

string

Chunk id that will be used when mapping the part to the LM Root's chunk.

VideoMetadata

Metadata describes the input video content.

Fields
start_offset

Duration

The start offset of the video.

end_offset

Duration

The end offset of the video.

PromptFeedback

Content filter results for a prompt sent in the request.

Fields
block_reason

BlockedReason

Blocked reason.

safety_ratings[]

SafetyRating

Safety ratings.

block_reason_message

string

A readable block reason message.

BlockedReason

Blocked reason enumeration.

Enums
BLOCKED_REASON_UNSPECIFIED Unspecified blocked reason.
SAFETY Candidates blocked due to safety.
OTHER Candidates blocked due to other reason.
BLOCKLIST Candidates blocked due to the terms which are included from the terminology blocklist.
PROHIBITED_CONTENT Candidates blocked due to prohibited content.

SafetyRating

Safety rating corresponding to the generated content.

Fields
category

HarmCategory

Harm category.

probability

HarmProbability

Harm probability levels in the content.

probability_score

float

Harm probability score.

severity

HarmSeverity

Harm severity levels in the content.

severity_score

float

Harm severity score.

blocked

bool

Indicates whether the content was filtered out because of this rating.

influential_terms[]

InfluentialTerm

The influential terms that could potentially block the response.

HarmProbability

Harm probability levels in the content.

Enums
HARM_PROBABILITY_UNSPECIFIED Harm probability unspecified.
NEGLIGIBLE Negligible level of harm.
LOW Low level of harm.
MEDIUM Medium level of harm.
HIGH High level of harm.

HarmSeverity

Harm severity levels.

Enums
HARM_SEVERITY_UNSPECIFIED Harm severity unspecified.
HARM_SEVERITY_NEGLIGIBLE Negligible level of harm severity.
HARM_SEVERITY_LOW Low level of harm severity.
HARM_SEVERITY_MEDIUM Medium level of harm severity.
HARM_SEVERITY_HIGH High level of harm severity.

InfluentialTerm

The influential term that could potentially block the response.

Fields
term

string

The influential term.

begin_offset

int32

The beginning offset of the influential term.

source

Source

The source of the influential term, prompt or response.

confidence

float

The confidence score of the influential term.

Source

Source of the influential term.

Enums
SOURCE_UNSPECIFIED Unspecified source.
PROMPT The influential term comes from the prompt.
RESPONSE The influential term comes from the response.

UsageMetadata

Usage metadata about response(s).

Fields
prompt_token_count

int32

Number of tokens in the request.

candidates_token_count

int32

Number of tokens in the response(s).

total_token_count

int32