Content

The structured data content of a message.

A Content message contains a role field, which indicates the producer of the content, and a parts field, which contains the multi-part data of the message.

Fields

role string

Optional. The producer of the content. Must be either 'user' or 'model'.

If not set, the service will default to 'user'.

parts[] object (Part)

Required. A list of Part objects that make up a single message. Parts of a message can have different MIME types.

A Content message must have at least one Part.

JSON representation
{ "role": string, "parts": [ { object (`Part`) } ] }

Part

A datatype containing media that is part of a multi-part Content message.

A Part consists of data which has an associated datatype. A Part can only contain one of the accepted types in Part.data.

For media types that are not text, Part must have a fixed IANA MIME type identifying the type and subtype of the media if inlineData or fileData field is filled with raw bytes.

Fields

thought boolean

Optional. Indicates whether the part represents the model's thought process or reasoning.

thoughtSignature string (bytes format)

Optional. An opaque signature for the thought so it can be reused in subsequent requests.

A base64-encoded string.

data Union type

data can be only one of the following:

text string

Optional. The text content of the part.

inlineData object (Blob)

Optional. The inline data content of the part. This can be used to include images, audio, or video in a request.

fileData object (FileData)

Optional. The URI-based data of the part. This can be used to include files from Google Cloud Storage.

functionCall object (FunctionCall)

Optional. A predicted function call returned from the model. This contains the name of the function to call and the arguments to pass to the function.

functionResponse object (FunctionResponse)

Optional. The result of a function call. This is used to provide the model with the result of a function call that it predicted.

executableCode object (ExecutableCode)

Optional. code generated by the model that is intended to be executed.

codeExecutionResult object (CodeExecutionResult)

Optional. The result of executing the ExecutableCode.

metadata Union type

metadata can be only one of the following:

videoMetadata object (VideoMetadata)

Optional. Video metadata. The metadata should only be specified while the video data is presented in inlineData or fileData.

JSON representation

JSON representation
{ "thought": boolean, "thoughtSignature": string, // data "text": string, "inlineData": { object (`Blob`) }, "fileData": { object (`FileData`) }, "functionCall": { object (`FunctionCall`) }, "functionResponse": { object (`FunctionResponse`) }, "executableCode": { object (`ExecutableCode`) }, "codeExecutionResult": { object (`CodeExecutionResult`) } // Union type // metadata "videoMetadata": { object (`VideoMetadata`) } // Union type }

{
  "thought": boolean,
  "thoughtSignature": string,

  // data
  "text": string,
  "inlineData": {
    object (Blob)
  },
  "fileData": {
    object (FileData)
  },
  "functionCall": {
    object (FunctionCall)
  },
  "functionResponse": {
    object (FunctionResponse)
  },
  "executableCode": {
    object (ExecutableCode)
  },
  "codeExecutionResult": {
    object (CodeExecutionResult)
  }
  // Union type

  // metadata
  "videoMetadata": {
    object (VideoMetadata)
  }
  // Union type
}

Blob

A content blob.

A blob contains data of a specific media type. It is used to represent images, audio, and video.

Fields

mimeType string

Required. The IANA standard MIME type of the source data.

data string (bytes format)

Required. The raw bytes of the data.

A base64-encoded string.

displayName string

Optional. The display name of the blob. Used to provide a label or filename to distinguish blobs.

This field is only returned in PromptMessage for prompt management. It is used in the Gemini calls only when server-side tools (codeExecution, googleSearch, and urlContext) are enabled.

JSON representation
{ "mimeType": string, "data": string, "displayName": string }

FileData

URI-based data.

A FileData message contains a URI pointing to data of a specific media type. It is used to represent images, audio, and video stored in Google Cloud Storage.

Fields

mimeType string

Required. The IANA standard MIME type of the source data.

fileUri string

Required. The URI of the file in Google Cloud Storage.

displayName string

Optional. The display name of the file. Used to provide a label or filename to distinguish files.

This field is only returned in PromptMessage for prompt management. It is used in the Gemini calls only when server side tools (codeExecution, googleSearch, and urlContext) are enabled.

            
                  JSON representation
                
{
  "mimeType": string,
  "fileUri": string,
  "displayName": string
}

FunctionCall

A predicted [FunctionCall] returned from the model that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing the parameters and their values.

Fields

name string

Optional. The name of the function to call. Matches [FunctionDeclaration.name].

args object (Struct format)

Optional. The function parameters and values in JSON object format. See [FunctionDeclaration.parameters] for parameter details.

JSON representation
{ "name": string, "args": { object } }

FunctionResponse

The result output from a [FunctionCall] that contains a string representing the [FunctionDeclaration.name] and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a [FunctionCall] made based on model prediction.

Fields

name string

Required. The name of the function to call. Matches [FunctionDeclaration.name] and [FunctionCall.name].

response object (Struct format)

Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output.

parts[] object (FunctionResponsePart)

Optional. Ordered Parts that constitute a function response. Parts may have different IANA MIME types.

JSON representation
{ "name": string, "response": { object }, "parts": [ { object (`FunctionResponsePart`) } ] }

FunctionResponsePart

A datatype containing media that is part of a FunctionResponse message.

A FunctionResponsePart consists of data which has an associated datatype. A FunctionResponsePart can only contain one of the accepted types in FunctionResponsePart.data.

A FunctionResponsePart must have a fixed IANA MIME type identifying the type and subtype of the media if the inlineData field is filled with raw bytes.

Fields

data Union type

The data of the function response part. data can be only one of the following:

inlineData object (FunctionResponseBlob)

Inline media bytes.

fileData object (FunctionResponseFileData)

URI based data.

JSON representation
{ // data "inlineData": { object (`FunctionResponseBlob`) }, "fileData": { object (`FunctionResponseFileData`) } // Union type }

FunctionResponseBlob

Raw media bytes for function response.

Text should not be sent as raw bytes, use the 'text' field.

Fields

mimeType string

Required. The IANA standard MIME type of the source data.

data string (bytes format)

Required. Raw bytes.

A base64-encoded string.

displayName string

Optional. Display name of the blob.

Used to provide a label or filename to distinguish blobs.

This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (codeExecution, googleSearch, and urlContext) are enabled.

JSON representation
{ "mimeType": string, "data": string, "displayName": string }

FunctionResponseFileData

URI based data for function response.

Fields

mimeType string

Required. The IANA standard MIME type of the source data.

fileUri string

Required. URI.

displayName string

Optional. Display name of the file data.

Used to provide a label or filename to distinguish file datas.

JSON representation
{ "mimeType": string, "fileUri": string, "displayName": string }

ExecutableCode

code generated by the model that is meant to be executed, and the result returned to the model.

Generated when using the [CodeExecution] tool, in which the code will be automatically executed, and a corresponding [CodeExecutionResult] will also be generated.

Fields

language enum (Language)

Required. Programming language of the code.

code string

Required. The code to be executed.

JSON representation
{ "language": enum (`Language`), "code": string }

Language

Supported programming languages for the generated code.

Enums
`LANGUAGE_UNSPECIFIED`	Unspecified language. This value should not be used.
`PYTHON`	Python >= 3.10, with numpy and simpy available.

CodeExecutionResult

result of executing the [ExecutableCode].

Only generated when using the [CodeExecution] tool, and always follows a part containing the [ExecutableCode].

Fields

outcome enum (Outcome)

Required. Outcome of the code execution.

output string

Optional. Contains stdout when code execution is successful, stderr or other description otherwise.

JSON representation
{ "outcome": enum (`Outcome`), "output": string }

Outcome

Enumeration of possible outcomes of the code execution.

Enums
`OUTCOME_UNSPECIFIED`	Unspecified status. This value should not be used.
`OUTCOME_OK`	code execution completed successfully.
`OUTCOME_FAILED`	code execution finished but with a failure. `stderr` should contain the reason.
`OUTCOME_DEADLINE_EXCEEDED`	code execution ran for too long, and was cancelled. There may or may not be a partial output present.

VideoMetadata

Provides metadata for a video, including the start and end offsets for clipping and the frame rate.

Fields

startOffset string (Duration format)

Optional. The start offset of the video.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

endOffset string (Duration format)

Optional. The end offset of the video.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

fps number

Optional. The frame rate of the video sent to the model. If not specified, the default value is 1.0. The valid range is (0.0, 24.0].

JSON representation
{ "startOffset": string, "endOffset": string, "fps": number }