函数调用

概览

您可以使用函数调用来定义自定义函数，并将这些函数提供给生成式 AI 模型。处理查询时，该模型可以选择将某些数据处理任务委托给这些函数。该模型不会调用函数。不过，它会提供结构化数据输出，其中包括所选函数的名称，以及模型建议调用该函数时所采用的参数。您可以使用此输出来调用外部 API。然后，您可以将 API 输出返回给模型，使模型能够完成对查询的回答。以这种方式使用时，函数调用可让 LLM 访问实时信息并与各种服务（例如 SQL 数据库、客户关系管理系统和文档库）进行交互。

下图说明了函数调用的工作原理：  

如需了解函数调用的应用场景，请参阅函数调用的应用场景。如需了解如何创建函数调用应用，请参阅创建函数调用应用。 如需了解最佳实践，请参阅最佳实践。

函数调用是 gemini-1.0-pro-001 模型的一项功能，也是 gemini-1.0-pro-002 模型的预览版功能。

函数调用是 Gemini 1.5 Pro（预览版）模型的一项功能。

函数调用的应用场景

您可以使用函数调用执行以下任务：

• 从自然语言故事中提取实体：从故事中提取角色、关系、事物和地点的列表。
  Python 版 Vertex AI SDK 笔记本 - 使用函数调用提取结构化数据
• 使用自然语言查询和理解 SQL 数据库：让模型将像 What percentage of orders are returned? 这样的问题转换为 SQL 查询，并创建函数以将这些查询提交到 BigQuery。
  博文 - 使用 Gemini 中的函数调用构建 AI 赋能的 BigQuery 数据探索应用
• 帮助客户与商家互动：创建与商家 API 相连的函数，使模型能够为像 Do you have the Pixel 8 Pro in stock? 或 Is there a store in Mountain View, CA that I can visit to try it out? 这样的查询提供准确的答案
  Python 版 Vertex AI SDK 笔记本 - 使用 Vertex AI Gemini API 和 Python SDK 进行函数调用
  REST 示例 | Python 版 Vertex AI SDK 聊天示例
• 通过连接到公共 API 构建生成式 AI 应用，例如：
  • 货币换算：创建一个连接到货币兑换应用的函数，使模型能够为像 What's the exchange rate for euros to dollars today? 这样的查询提供准确的答案
    Codelab - 如何在 Gemini 中使用函数调用与 API 进行交互
  • 获取给定位置的天气：创建一个连接到气象服务的 API 的函数，使模型能够为像 What's the weather like in Paris? 这样的查询提供准确的答案
    Python 版 Vertex AI SDK 笔记本 - 使用 Vertex AI Gemini API 和 Python SDK 进行函数调用
    博文 - 函数调用：用于将 Gemini 连接到外部系统、数据和 API 的原生框架
    Python 版和 Node.js 版 Vertex AI SDK 文本示例 | Node.js、Java 和 Go 聊天示例
  • 将地址转换为纬度和经度坐标：创建一个将结构化位置数据转换为纬度和经度坐标的函数。要求模型在像 I want to get the lat/lon coordinates for the following address: 1600 Amphitheatre Pkwy, Mountain View, CA 94043, US. 这样的查询中识别街道地址、城市、州和邮政编码
    Python 版 Vertex AI SDK 笔记本 - 使用 Vertex AI Gemini API 和 Python SDK 进行函数调用
• 解读语音指令：创建与车载任务相对应的函数。例如，您可以创建用于开启电台或启动空调的函数。将用户语音指令的音频文件发送到模型，要求模型将音频转换为文字，并识别用户想要调用的功能。
• 根据环境触发器自动执行工作流：创建用于表示可自动化的流程的函数。向模型提供来自环境传感器的数据，并要求模型解析和处理数据，以确定是否应激活一个或多个工作流。例如，一个模型可以处理仓库中的温度数据，并选择启用洒水器功能。
• 自动分配支持服务工单：为模型提供支持服务工单、日志和内容感知规则。让模型处理所有这些信息，以确定应将工单分配给谁。调用一个函数，将工单分配给模型建议的人员。
• 从知识库中检索信息：创建用于检索并汇总给定主题方面的学术文章的函数。使模型能够回答有关学术主题的问题，并为其答案提供引用。

创建函数调用应用

为了让用户能够与模型交互并使用函数调用，您必须创建执行以下任务的代码：

• 使用函数声明定义和描述一组可用的函数。
• 向模型提交用户的查询和函数声明。
• 使用模型输出的结构化数据调用函数。
• 向模型提供函数输出。

您可以创建一个应用来管理所有这些任务。此应用可以是文本聊天机器人、语音客服、自动化工作流或任何其他程序。

定义和描述一组可用函数

应用必须声明一组函数，供模型用于处理查询。每个函数声明必须包含函数名称和函数参数。此外，我们强烈建议您在每个函数声明中添加函数说明。

函数名称

应用和模型使用函数名称来标识函数。

如需了解与函数名称相关的最佳实践，请参阅最佳实践 - 函数名称。

函数参数

提供的函数参数必须采用与 OpenAPI 架构兼容的格式。

Vertex AI 对 OpenAPI 架构提供有限支持。支持以下属性：type、nullable、required、format、description、properties、items、enum。不支持以下特性：default、optional、maximum、oneOf。

使用 curl 时，请使用 JSON 指定架构。使用 Python 版 Vertex AI SDK 时，请使用 Python 字典指定架构。

如需了解与函数参数相关的最佳实践，请参阅最佳实践 - 函数参数。

函数说明

模型使用函数说明来了解函数的用途，并确定这些函数对于处理用户查询是否有用。

如需了解与函数说明相关的最佳实践，请参阅最佳实践 - 函数说明。

函数声明示例

以下是 Python 中的简单函数声明示例：

get_current_weather_func = FunctionDeclaration(
    name="get_current_weather",
    description="Get the current weather in a given location",
    parameters={
        "type": "object",
        "properties": {"location": {"type": "string", "description": "The city name of the location for which to get the weather."}},
    },
)

下面是包含项数组的函数声明示例：

extract_sale_records_func = FunctionDeclaration(
   name="extract_sale_records",
   description="Extract sale records from a document.",
   parameters={
       "type": "object",
       "properties": {
           "records": {
               "type": "array",
               "description": "A list of sale records",
               "items": {
                   "description": "Data for a sale record",
                   "type": "object",
                   "properties": {
                "id": {"type": "integer", "description": "The unique id of the sale."},
                       "date": {"type": "string", "description": "Date of the sale, in the format of MMDDYY, e.g., 031023"},
                       "total_amount": {"type": "number", "description": "The total amount of the sale."},
                       "customer_name": {"type": "string", "description": "The name of the customer, including first name and last name."},
                       "customer_contact": {"type": "string", "description": "The phone number of the customer, e.g., 650-123-4567."},
                   },
                   "required": ["id", "date", "total_amount"],
               },
           },
       },
       "required": ["records"],
   },
)

将查询和函数声明提交给模型

当用户提供提示时，应用必须为模型提供用户提示和函数声明。如需配置模型生成结果的方式，应用可以为模型提供生成配置。如需配置模型使用函数声明的方式，应用可以为模型提供工具配置。

用户提示

以下是用户提示的示例：“波士顿的天气如何？”

如需了解与用户提示相关的最佳实践，请参阅最佳实践 - 用户提示。

生成配置

对于不同的参数值，模型会生成不同的结果。温度参数用于控制此生成中的随机程度。 较低的温度有利于需要确定性参数值的函数，而较高的温度有利于参数接受更多样化或创造性参数值的函数。温度为 0 表示确定性，即始终选择概率最高的回答。如需了解详情，请参阅 Gemini API。

如需设置此参数，请提交生成配置 (generation_config) 以及提示和函数声明。

如需了解与生成配置相关的最佳做法，请参阅最佳做法 - 生成配置。

工具配置

您可以对模型应如何使用您为模型提供的函数声明施加一些限制。例如，您可以强制模型仅预测函数调用，而不是允许模型在自然语言回答和函数调用之间进行选择。您还可以选择为模型提供一整套函数声明，但应将其回答限制为其中一部分函数。

如需放置这些限制条件，请提交工具配置 (tool_config) 以及提示和函数声明。在配置中，您可以指定以下模式之一：

模式	说明
自动	默认模型行为。模型决定是预测函数调用还是自然语言回答。
任意	模型必须仅预测函数调用。如需将模型限制为部分函数，请在 allowed_function_names 中定义允许的函数名称。
无	模型不得预测函数调用。此行为等效于没有任何关联函数声明的模型请求。

工具配置的 ANY 模式是一项预览版功能。只有 Gemini 1.5 Pro 模型支持此功能。

如需了解详情，请参阅函数调用 API。

如何提交提示和函数声明

以下示例展示了如何向模型提交查询和函数声明，并将模型限制为仅预测 get_current_weather 函数调用。

# Initialize Vertex AI
vertexai.init(project=project_id, location=location)

# Initialize Gemini model
model = GenerativeModel("gemini-1.0-pro-002")

# Define a tool that includes the function declaration get_current_weather_func
weather_tool = Tool(
    function_declarations=[get_current_weather_func],
)

# Define the user's prompt in a Content object that we can reuse in model calls
user_prompt_content = Content(
    role="user",
    parts=[
        Part.from_text(prompt),
    ],
)

# Send the prompt and instruct the model to generate content using the Tool object that you just created
response = model.generate_content(
    user_prompt_content,
    generation_config={"temperature": 0},
    tools=[weather_tool],
    tool_config=ToolConfig(
        function_calling_config=ToolConfig.FunctionCallingConfig(
            # ANY mode forces the model to predict a function call
            mode=ToolConfig.FunctionCallingConfig.Mode.ANY,
            # Allowed functions to call when the mode is ANY. If empty, any one of
            # the provided functions are called.
            allowed_function_names=["get_current_weather"],
    ))
)
response_function_call_content = response.candidates[0].content

如果模型确定它需要特定函数的输出，则应用从该模型收到的响应将包含函数名称以及调用该函数应使用的参数值。

以下是模型响应的示例。模型建议以参数 Boston, MA 调用 get_current_weather 函数。

candidates {
    content {
      role: "model"
      parts {
        function_call {
          name: "get_current_weather"
          args {
            fields {
              key: "location"
              value {
                string_value: "Boston, MA"
              }
            }
          }
        }
      }
    }
    ...
}

调用外部 API

如果应用从模型收到函数名称和参数值，则必须连接到外部 API 并调用该函数。

以下示例使用合成数据来模拟来自外部 API 的响应载荷：

# Check the function name that the model responded with, and make an API call to an external system
if (
    response.candidates[0].content.parts[0].function_call.name
    == "get_current_weather"
):
    # Extract the arguments to use in your API call
    location = (
        response.candidates[0].content.parts[0].function_call.args["location"]
    )

    # Here you can use your preferred method to make an API request to fetch the current weather, for example:
    # api_response = requests.post(weather_api_url, data={"location": location})

    # In this example, we'll use synthetic data to simulate a response payload from an external API
    api_response = """{ "location": "Boston, MA", "temperature": 38, "description": "Partly Cloudy",
                    "icon": "partly-cloudy", "humidity": 65, "wind": { "speed": 10, "direction": "NW" } }"""

如需了解与 API 调用相关的最佳实践，请参阅最佳实践 - API 调用。

向模型提供函数输出

应用从外部 API 收到响应后，必须将此响应提供给模型。以下示例展示了如何使用 Python 执行此操作：

response = model.generate_content(
    [
        user_prompt_content,  # User prompt
        response_function_call_content,  # Function call response
        Content(
            parts=[
                Part.from_function_response(
                    name="get_current_weather",
                    response={
                        "content": api_response,  # Return the API response to Gemini
                    },
                )
            ],
        ),
    ],
    tools=[weather_tool],
)
# Get the model summary response
summary = response.candidates[0].content.parts[0].text

如果模型确定 API 响应足以回答用户的查询，则会创建自然语言回答并将其返回给应用。在这种情况下，应用必须将回答传递回用户。以下是查询回答的示例： It is currently 38 degrees Fahrenheit in Boston, MA with partly cloudy skies. The humidity is 65% and the wind is blowing at 10 mph from the northwest.

如果模型确定需要另一个函数的输出来回答查询，则应用从模型收到的响应将包含另一个函数名称和另一组参数值。

最佳实践

函数名称

请勿在函数名称中使用英文句点 (.)、短划线 (-) 或空格字符。 请改用下划线 (_) 字符或任何其他字符。

函数参数

编写清晰而详细的参数说明，包括首选格式或值等详细信息。例如，对于 book_flight_ticket 函数：

• 以下是 departure 参数说明的一个良好示例：Use the 3 char airport code to represent the airport. For example, SJC or SFO. Don't use the city name.
• 以下是 departure 参数说明的一个不良示例：the departure airport

尽可能使用强类型参数来减少模型幻觉。例如，如果参数值来自有限集，请添加 enum 字段，而不是将值集放入说明中。如果参数值始终是整数，请将类型设置为 integer，而不是 number。

函数说明

清晰而详细地撰写函数说明。例如，对于 book_flight_ticket 函数：

• 下面是一个良好的函数说明示例：book flight tickets after confirming users' specific requirements, such as time, departure, destination, party size and preferred airline
• 下面是一个不良的函数说明示例：book flight ticket

用户提示

为获得最佳结果，请在用户查询前面加上以下详细信息：

• 模型的其他上下文，例如 You are a flight API assistant to help with searching flights based on user preferences.
• 有关如何及何时使用函数的详细信息或说明，例如 Don't make assumptions on the departure or destination airports. Always use a future date for the departure or destination time.
• 在用户查询不明确时询问澄清性问题的说明，例如 Ask clarifying questions if not enough information is available.

生成配置

对于温度参数，请使用 0 或其他较低值。这会指示模型生成置信度更高的结果并减少幻觉。

API 调用

如果模型建议调用一个会发送订单、更新数据库或以其他方式产生重大后果的函数，请在执行之前先向用户验证该函数调用。

价格

函数调用的价格基于文本输入和输出中的字符数。如需了解详情，请参阅 Vertex AI 价格。

此处，文本输入（提示）指的是针对当前对话回合的用户查询、针对当前对话回合的函数声明以及对话历史记录。对话的历史记录包括查询、函数调用以及上一轮对话的函数回答。Vertex AI 会将对话的历史记录截断为 32,000 个字符。

文本输出（回答）是指当前对话轮次的函数调用和文本回答。

函数调用示例

您可以使用函数调用生成单个文本回答或支持聊天会话。临时文本回答对于特定的业务任务（包括代码生成）非常有用。聊天会话在形式自由的对话场景中非常有用，这些场景中用户可能会询问后续问题。

如果您使用函数调用生成单个回答，则必须为模型提供互动的完整上下文。另一方面，如果您在聊天会话的上下文中使用函数调用，则会话会为您存储上下文，并将其包含在每个模型请求中。在这两种情况下，Vertex AI 都会将互动的历史记录存储在客户端。

如需了解如何使用函数调用生成单个文本回答，请参阅文本示例。如需了解如何使用函数调用来支持聊天会话，请参阅聊天示例。

文本示例

Python

此示例演示的是文本场景，其中包含一个函数和一个提示。它使用 GenerativeModel 类及其方法。如需详细了解如何将 Python 版 Vertex AI SDK 与 Gemini 多模态模型搭配使用，请参阅 Python 版 Vertex AI SDK 中的多模态类简介。

Python

如需了解如何安装或更新 Python，请参阅安装 Python 版 Vertex AI SDK。如需了解详情，请参阅 Python API 参考文档。

import vertexai
from vertexai.generative_models import (
    Content,
    FunctionDeclaration,
    GenerationConfig,
    GenerativeModel,
    Part,
    Tool,
)

def generate_function_call(prompt: str, project_id: str, location: str) -> tuple:
    # Initialize Vertex AI
    vertexai.init(project=project_id, location=location)

    # Initialize Gemini model
    model = GenerativeModel("gemini-1.0-pro")

    # Specify a function declaration and parameters for an API request
    get_current_weather_func = FunctionDeclaration(
        name="get_current_weather",
        description="Get the current weather in a given location",
        # Function parameters are specified in OpenAPI JSON schema format
        parameters={
            "type": "object",
            "properties": {"location": {"type": "string", "description": "Location"}},
        },
    )

    # Define a tool that includes the above get_current_weather_func
    weather_tool = Tool(
        function_declarations=[get_current_weather_func],
    )

    # Define the user's prompt in a Content object that we can reuse in model calls
    user_prompt_content = Content(
        role="user",
        parts=[
            Part.from_text(prompt),
        ],
    )

    # Send the prompt and instruct the model to generate content using the Tool that you just created
    response = model.generate_content(
        user_prompt_content,
        generation_config=GenerationConfig(temperature=0),
        tools=[weather_tool],
    )
    response_function_call_content = response.candidates[0].content

    # Check the function name that the model responded with, and make an API call to an external system
    if (
        response.candidates[0].content.parts[0].function_call.name
        == "get_current_weather"
    ):
        # Extract the arguments to use in your API call
        location = (
            response.candidates[0].content.parts[0].function_call.args["location"]
        )

        # Here you can use your preferred method to make an API request to fetch the current weather, for example:
        # api_response = requests.post(weather_api_url, data={"location": location})

        # In this example, we'll use synthetic data to simulate a response payload from an external API
        api_response = """{ "location": "Boston, MA", "temperature": 38, "description": "Partly Cloudy",
                        "icon": "partly-cloudy", "humidity": 65, "wind": { "speed": 10, "direction": "NW" } }"""

    # Return the API response to Gemini so it can generate a model response or request another function call
    response = model.generate_content(
        [
            user_prompt_content,  # User prompt
            response_function_call_content,  # Function call response
            Content(
                parts=[
                    Part.from_function_response(
                        name="get_current_weather",
                        response={
                            "content": api_response,  # Return the API response to Gemini
                        },
                    )
                ],
            ),
        ],
        tools=[weather_tool],
    )
    # Get the model summary response
    summary = response.candidates[0].content.parts[0].text

    return summary, response

Node.js

此示例演示的是文本场景，其中包含一个函数和一个提示。

Node.js

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Node.js 设置说明执行操作。如需了解详情，请参阅 Vertex AI Node.js API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

const {
  VertexAI,
  FunctionDeclarationSchemaType,
} = require('@google-cloud/vertexai');

const functionDeclarations = [
  {
    function_declarations: [
      {
        name: 'get_current_weather',
        description: 'get weather in a given location',
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            location: {type: FunctionDeclarationSchemaType.STRING},
            unit: {
              type: FunctionDeclarationSchemaType.STRING,
              enum: ['celsius', 'fahrenheit'],
            },
          },
          required: ['location'],
        },
      },
    ],
  },
];

const functionResponseParts = [
  {
    functionResponse: {
      name: 'get_current_weather',
      response: {name: 'get_current_weather', content: {weather: 'super nice'}},
    },
  },
];

/**
 * TODO(developer): Update these variables before running the sample.
 */
async function functionCallingStreamChat(
  projectId = 'PROJECT_ID',
  location = 'us-central1',
  model = 'gemini-1.0-pro'
) {
  // Initialize Vertex with your Cloud project and location
  const vertexAI = new VertexAI({project: projectId, location: location});

  // Instantiate the model
  const generativeModel = vertexAI.preview.getGenerativeModel({
    model: model,
  });

  const request = {
    contents: [
      {role: 'user', parts: [{text: 'What is the weather in Boston?'}]},
      {
        role: 'model',
        parts: [
          {
            functionCall: {
              name: 'get_current_weather',
              args: {location: 'Boston'},
            },
          },
        ],
      },
      {role: 'function', parts: functionResponseParts},
    ],
    tools: functionDeclarations,
  };
  const streamingResp = await generativeModel.generateContentStream(request);
  for await (const item of streamingResp.stream) {
    console.log(item.candidates[0].content.parts[0].text);
  }
}

REST

此示例演示的是文本场景，其中包含三个函数和一个提示。

在此示例中，您调用了两次生成式 AI 模型。

• 在第一次调用中，您为模型提供了提示和函数声明。
• 在第二次调用中，您为模型提供了 API 响应。

第一个模型请求

该请求必须在 text 参数中定义查询。本示例定义了以下查询：“Which theaters in Mountain View show Barbie movie?”（山景城的哪些影院在播放电影《芭比》？）。

该请求还必须定义一个工具 (tools)，其中包含一组函数声明 (functionDeclarations)。这些函数声明必须采用与 OpenAPI 架构兼容的格式指定。本示例定义了以下函数：

• find_movies 查找在影院上播放的电影。
• find_theatres 根据位置查找影院。
• get_showtimes 查找在特定影院播放的电影的开场时间。

如需详细了解模型请求的参数，请参阅 Gemini API。

将 my-project 替换为您的 Google Cloud 项目的名称。

第一个模型请求

PROJECT_ID=my-project
MODEL_ID=gemini-1.0-pro
API=streamGenerateContent
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json"  https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:${API} -d '{
"contents": {
  "role": "user",
  "parts": {
    "text": "Which theaters in Mountain View show the Barbie movie?"
  }
},
"tools": [
  {
    "function_declarations": [
      {
        "name": "find_movies",
        "description": "find movie titles currently playing in theaters based on any description, genre, title words, etc.",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616"
            },
            "description": {
              "type": "string",
              "description": "Any kind of description including category or genre, title words, attributes, etc."
            }
          },
          "required": [
            "description"
          ]
        }
      },
      {
        "name": "find_theaters",
        "description": "find theaters based on location and optionally movie title which are is currently playing in theaters",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616"
            },
            "movie": {
              "type": "string",
              "description": "Any movie title"
            }
          },
          "required": [
            "location"
          ]
        }
      },
      {
        "name": "get_showtimes",
        "description": "Find the start times for movies playing in a specific theater",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616"
            },
            "movie": {
              "type": "string",
              "description": "Any movie title"
            },
            "theater": {
              "type": "string",
              "description": "Name of the theater"
            },
            "date": {
              "type": "string",
              "description": "Date for requested showtime"
            }
          },
          "required": [
            "location",
            "movie",
            "theater",
            "date"
          ]
        }
      }
    ]
  }
]
}'
  

对于“Which theaters in Mountain View show Barbie movie?”（山景城的哪些影院在播放电影《芭比》？）提示，模型可能会返回带有参数 Barbie 和 Mountain View, CA 的函数 find_theatres。

对第一个模型请求的响应

[{
"candidates": [
  {
    "content": {
      "parts": [
        {
          "functionCall": {
            "name": "find_theaters",
            "args": {
              "movie": "Barbie",
              "location": "Mountain View, CA"
            }
          }
        }
      ]
    },
    "finishReason": "STOP",
    "safetyRatings": [
      {
        "category": "HARM_CATEGORY_HARASSMENT",
        "probability": "NEGLIGIBLE"
      },
      {
        "category": "HARM_CATEGORY_HATE_SPEECH",
        "probability": "NEGLIGIBLE"
      },
      {
        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
        "probability": "NEGLIGIBLE"
      },
      {
        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
        "probability": "NEGLIGIBLE"
      }
    ]
  }
],
"usageMetadata": {
  "promptTokenCount": 9,
  "totalTokenCount": 9
}
}]
  

第二个模型请求

此示例使用合成数据，而不是调用外部 API。具有两项结果，每个结果都包含两个参数（name 和 address）：

1. name: AMC Mountain View 16, address: 2000 W El Camino Real, Mountain View, CA 94040
2. name: Regal Edwards 14, address: 245 Castro St, Mountain View, CA 94040

将 my-project 替换为您的 Google Cloud 项目的名称。

第二个模型请求

PROJECT_ID=my-project
MODEL_ID=gemini-1.0-pro
API=streamGenerateContent
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json"  https://us-central1-autopush-aiplatform.sandbox.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:${API} -d '{
"contents": [{
  "role": "user",
  "parts": [{
    "text": "Which theaters in Mountain View show the Barbie movie?"
  }]
}, {
  "role": "model",
  "parts": [{
    "functionCall": {
      "name": "find_theaters",
      "args": {
        "location": "Mountain View, CA",
        "movie": "Barbie"
      }
    }
  }]
}, {
  "parts": [{
    "functionResponse": {
      "name": "find_theaters",
      "response": {
        "name": "find_theaters",
        "content": {
          "movie": "Barbie",
          "theaters": [{
            "name": "AMC Mountain View 16",
            "address": "2000 W El Camino Real, Mountain View, CA 94040"
          }, {
            "name": "Regal Edwards 14",
            "address": "245 Castro St, Mountain View, CA 94040"
          }]
        }
      }
    }
  }]
}],
"tools": [{
  "functionDeclarations": [{
    "name": "find_movies",
    "description": "find movie titles currently playing in theaters based on any description, genre, title words, etc.",
    "parameters": {
      "type": "OBJECT",
      "properties": {
        "location": {
          "type": "STRING",
          "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616"
        },
        "description": {
          "type": "STRING",
          "description": "Any kind of description including category or genre, title words, attributes, etc."
        }
      },
      "required": ["description"]
    }
  }, {
    "name": "find_theaters",
    "description": "find theaters based on location and optionally movie title which are is currently playing in theaters",
    "parameters": {
      "type": "OBJECT",
      "properties": {
        "location": {
          "type": "STRING",
          "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616"
        },
        "movie": {
          "type": "STRING",
          "description": "Any movie title"
        }
      },
      "required": ["location"]
    }
  }, {
    "name": "get_showtimes",
    "description": "Find the start times for movies playing in a specific theater",
    "parameters": {
      "type": "OBJECT",
      "properties": {
        "location": {
          "type": "STRING",
          "description": "The city and state, e.g. San Francisco, CA or a zip code e.g. 95616"
        },
        "movie": {
          "type": "STRING",
          "description": "Any movie title"
        },
        "theater": {
          "type": "STRING",
          "description": "Name of the theater"
        },
        "date": {
          "type": "STRING",
          "description": "Date for requested showtime"
        }
      },
      "required": ["location", "movie", "theater", "date"]
    }
  }]
}]
}'
  

模型的响应可能类似于以下内容：

对第二个模型请求的响应

{
"candidates": [
  {
    "content": {
      "parts": [
        {
          "text": " OK. Barbie is showing in two theaters in Mountain View, CA: AMC Mountain View 16 and Regal Edwards 14."
        }
      ]
    }
  }
],
"usageMetadata": {
  "promptTokenCount": 9,
  "candidatesTokenCount": 27,
  "totalTokenCount": 36
}
}
  

聊天示例

Python

此示例演示的是包含两个函数和两个连续提示的聊天场景。它使用 GenerativeModel 类及其方法。如需详细了解如何将 Python 版 Vertex AI SDK 与多模态模型搭配使用，请参阅 Python 版 Vertex AI SDK 中的多模态类简介。

如需了解如何安装或更新 Python，请参阅安装 Python 版 Vertex AI SDK。如需了解详情，请参阅 Python API 参考文档。

import vertexai
from vertexai.generative_models import (
    FunctionDeclaration,
    GenerativeModel,
    Part,
    Tool,
)

def generate_function_call_chat(project_id: str, location: str) -> tuple:
    prompts = []
    summaries = []

    # Initialize Vertex AI
    vertexai.init(project=project_id, location=location)

    # Specify a function declaration and parameters for an API request
    get_product_info_func = FunctionDeclaration(
        name="get_product_sku",
        description="Get the SKU for a product",
        # Function parameters are specified in OpenAPI JSON schema format
        parameters={
            "type": "object",
            "properties": {
                "product_name": {"type": "string", "description": "Product name"}
            },
        },
    )

    # Specify another function declaration and parameters for an API request
    get_store_location_func = FunctionDeclaration(
        name="get_store_location",
        description="Get the location of the closest store",
        # Function parameters are specified in OpenAPI JSON schema format
        parameters={
            "type": "object",
            "properties": {"location": {"type": "string", "description": "Location"}},
        },
    )

    # Define a tool that includes the above functions
    retail_tool = Tool(
        function_declarations=[
            get_product_info_func,
            get_store_location_func,
        ],
    )

    # Initialize Gemini model
    model = GenerativeModel(
        "gemini-1.0-pro", generation_config={"temperature": 0}, tools=[retail_tool]
    )

    # Start a chat session
    chat = model.start_chat()

    # Send a prompt for the first conversation turn that should invoke the get_product_sku function
    prompt = "Do you have the Pixel 8 Pro in stock?"
    response = chat.send_message(prompt)
    prompts.append(prompt)

    # Check the function name that the model responded with, and make an API call to an external system
    if response.candidates[0].content.parts[0].function_call.name == "get_product_sku":
        # Extract the arguments to use in your API call
        product_name = (
            response.candidates[0].content.parts[0].function_call.args["product_name"]
        )
        product_name

        # Here you can use your preferred method to make an API request to retrieve the product SKU, as in:
        # api_response = requests.post(product_api_url, data={"product_name": product_name})

        # In this example, we'll use synthetic data to simulate a response payload from an external API
        api_response = {"sku": "GA04834-US", "in_stock": "yes"}

    # Return the API response to Gemini so it can generate a model response or request another function call
    response = chat.send_message(
        Part.from_function_response(
            name="get_product_sku",
            response={
                "content": api_response,
            },
        ),
    )

    # Extract the text from the summary response
    summary = response.candidates[0].content.parts[0].text
    summaries.append(summary)

    # Send a prompt for the second conversation turn that should invoke the get_store_location function
    prompt = "Is there a store in Mountain View, CA that I can visit to try it out?"
    response = chat.send_message(prompt)
    prompts.append(prompt)

    # Check the function name that the model responded with, and make an API call to an external system
    if (
        response.candidates[0].content.parts[0].function_call.name
        == "get_store_location"
    ):
        # Extract the arguments to use in your API call
        location = (
            response.candidates[0].content.parts[0].function_call.args["location"]
        )
        location

        # Here you can use your preferred method to make an API request to retrieve store location closest to the user, as in:
        # api_response = requests.post(store_api_url, data={"location": location})

        # In this example, we'll use synthetic data to simulate a response payload from an external API
        api_response = {"store": "2000 N Shoreline Blvd, Mountain View, CA 94043, US"}

    # Return the API response to Gemini so it can generate a model response or request another function call
    response = chat.send_message(
        Part.from_function_response(
            name="get_store_location",
            response={
                "content": api_response,
            },
        ),
    )

    # Extract the text from the summary response
    summary = response.candidates[0].content.parts[0].text
    summaries.append(summary)

    return prompts, summaries

Node.js

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Node.js 设置说明执行操作。如需了解详情，请参阅 Vertex AI Node.js API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

const {
  VertexAI,
  FunctionDeclarationSchemaType,
} = require('@google-cloud/vertexai');

const functionDeclarations = [
  {
    function_declarations: [
      {
        name: 'get_current_weather',
        description: 'get weather in a given location',
        parameters: {
          type: FunctionDeclarationSchemaType.OBJECT,
          properties: {
            location: {type: FunctionDeclarationSchemaType.STRING},
            unit: {
              type: FunctionDeclarationSchemaType.STRING,
              enum: ['celsius', 'fahrenheit'],
            },
          },
          required: ['location'],
        },
      },
    ],
  },
];

const functionResponseParts = [
  {
    functionResponse: {
      name: 'get_current_weather',
      response: {name: 'get_current_weather', content: {weather: 'super nice'}},
    },
  },
];

/**
 * TODO(developer): Update these variables before running the sample.
 */
async function functionCallingStreamChat(
  projectId = 'PROJECT_ID',
  location = 'us-central1',
  model = 'gemini-1.0-pro'
) {
  // Initialize Vertex with your Cloud project and location
  const vertexAI = new VertexAI({project: projectId, location: location});

  // Instantiate the model
  const generativeModel = vertexAI.preview.getGenerativeModel({
    model: model,
  });

  // Create a chat session and pass your function declarations
  const chat = generativeModel.startChat({
    tools: functionDeclarations,
  });

  const chatInput1 = 'What is the weather in Boston?';

  // This should include a functionCall response from the model
  const result1 = await chat.sendMessageStream(chatInput1);
  for await (const item of result1.stream) {
    console.log(item.candidates[0]);
  }
  await result1.response;

  // Send a follow up message with a FunctionResponse
  const result2 = await chat.sendMessageStream(functionResponseParts);
  for await (const item of result2.stream) {
    console.log(item.candidates[0]);
  }

  // This should include a text response from the model using the response content
  // provided above
  const response2 = await result2.response;
  console.log(response2.candidates[0].content.parts[0].text);
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.Content;
import com.google.cloud.vertexai.api.FunctionDeclaration;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.api.Schema;
import com.google.cloud.vertexai.api.Tool;
import com.google.cloud.vertexai.api.Type;
import com.google.cloud.vertexai.generativeai.ChatSession;
import com.google.cloud.vertexai.generativeai.ContentMaker;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.PartMaker;
import com.google.cloud.vertexai.generativeai.ResponseHandler;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;

public class FunctionCalling {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-google-cloud-project-id";
    String location = "us-central1";
    String modelName = "gemini-1.0-pro";

    String promptText = "What's the weather like in Paris?";

    whatsTheWeatherLike(projectId, location, modelName, promptText);
  }

  public static String whatsTheWeatherLike(String projectId, String location,
                                           String modelName, String promptText)
      throws IOException {

    try (VertexAI vertexAI = new VertexAI(projectId, location)) {

      FunctionDeclaration functionDeclaration = FunctionDeclaration.newBuilder()
          .setName("getCurrentWeather")
          .setDescription("Get the current weather in a given location")
          .setParameters(
              Schema.newBuilder()
                  .setType(Type.OBJECT)
                  .putProperties("location", Schema.newBuilder()
                      .setType(Type.STRING)
                      .setDescription("location")
                      .build()
                  )
                  .addRequired("location")
                  .build()
          )
          .build();

      System.out.println("Function declaration:");
      System.out.println(functionDeclaration);

      // Add the function to a "tool"
      Tool tool = Tool.newBuilder()
          .addFunctionDeclarations(functionDeclaration)
          .build();

      // Start a chat session from a model, with the use of the declared function.
      GenerativeModel model =
          GenerativeModel.newBuilder()
              .setModelName(modelName)
              .setVertexAi(vertexAI)
              .setTools(Arrays.asList(tool))
              .build();
      ChatSession chat = model.startChat();

      System.out.println(String.format("Ask the question: %s", promptText));
      GenerateContentResponse response = chat.sendMessage(promptText);

      // The model will most likely return a function call to the declared
      // function `getCurrentWeather` with "Paris" as the value for the
      // argument `location`.
      System.out.println("\nPrint response: ");
      System.out.println(ResponseHandler.getContent(response));

      // Provide an answer to the model so that it knows what the result
      // of a "function call" is.
      Content content =
          ContentMaker.fromMultiModalData(
              PartMaker.fromFunctionResponse(
                  "getCurrentWeather",
                  Collections.singletonMap("currentWeather", "sunny")));
      System.out.println("Provide the function response: ");
      System.out.println(content);
      response = chat.sendMessage(content);

      // See what the model replies now
      System.out.println("Print response: ");
      String finalAnswer = ResponseHandler.getText(response);
      System.out.println(finalAnswer);

      return finalAnswer;
    }
  }
}

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// functionCalls opens a chat session and sends 2 messages to the model:
// - first, to convert a text into a structured function call request
// - second, to convert a structured function call response into natural language
func functionCalls(w io.Writer, prompt, projectID, location, modelName string) error {
	// prompt := "What's the weather like in Paris?"
	// location := "us-central1"
	// modelName := "gemini-1.0-pro"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %v", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	// Build an OpenAPI schema, in memory
	params := &genai.Schema{
		Type: genai.TypeObject,
		Properties: map[string]*genai.Schema{
			"location": {
				Type:        genai.TypeString,
				Description: "location",
			},
		},
	}
	fundecl := &genai.FunctionDeclaration{
		Name:        "getCurrentWeather",
		Description: "Get the current weather in a given location",
		Parameters:  params,
	}
	model.Tools = []*genai.Tool{
		{FunctionDeclarations: []*genai.FunctionDeclaration{fundecl}},
	}

	chat := model.StartChat()

	fmt.Fprintf(w, "Question: %s\n", prompt)
	resp, err := chat.SendMessage(ctx, genai.Text(prompt))
	if err != nil {
		return err
	}
	if len(resp.Candidates) == 0 ||
		len(resp.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	// The model has returned a function call to the declared function `getCurrentWeather`
	// with a value for the argument `location`.
	jsondata, err := json.MarshalIndent(resp.Candidates[0].Content.Parts[0], "", "  ")
	if err != nil {
		return fmt.Errorf("json.Marshal: %w", err)
	}
	fmt.Fprintf(w, "function call generated by the model:\n%s\n\n", string(jsondata))

	// Create a function call response, to simulate the result of a call to a
	// real service
	funresp := &genai.FunctionResponse{
		Name: "getCurrentWeather",
		Response: map[string]any{
			"currentWeather": "sunny",
		},
	}
	jsondata, err = json.MarshalIndent(funresp, "", "  ")
	if err != nil {
		return fmt.Errorf("json.Marshal: %w", err)
	}
	fmt.Fprintf(w, "function call response sent to the model:\n%s\n\n", string(jsondata))

	// And provide the function call response to the model
	resp, err = chat.SendMessage(ctx, funresp)
	if err != nil {
		return err
	}
	if len(resp.Candidates) == 0 ||
		len(resp.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	// The model has taken the function call response as input, and has
	// reformulated the response to the user.
	jsondata, err = json.MarshalIndent(resp.Candidates[0].Content.Parts[0], "", "  ")
	if err != nil {
		return fmt.Errorf("json.Marshal: %w", err)
	}
	fmt.Fprintf(w, "Answer generated by the model:\n%s\n", string(jsondata))

	return nil
}