此页面由 Cloud Translation API 翻译。

受控生成功能

您可以保证模型生成的输出始终遵循特定架构，以便您收到格式一致的响应。例如，您可能已经建立了用于其他任务的数据架构。如果模型遵循相同的架构，您可以直接从模型的输出中提取数据，而无需进行任何后期处理。

如需指定模型的输出结构，请定义响应架构，该架构类似于模型响应的蓝图。如果您提交提示并添加响应架构，模型的响应始终遵循您定义的架构。

您可以控制使用以下模型时生成的输出：

Gemini 2.0 Flash
Gemini 1.5 Pro
Gemini 1.5 Flash

应用场景示例

应用响应架构的一个用例是确保模型的响应生成有效的 JSON 并符合您的架构。生成模型输出可能会有一定程度的变化，添加响应架构可确保您始终收到有效的 JSON。因此，您的下游任务预计可以可靠地得到来自所生成响应的有效 JSON 输入。

另一个示例是限制模型的响应方式。例如，您可以让模型使用用户定义的标签而不是模型生成的标签来为文本添加注释。如果您期望获得一组特定标签（例如 positive 或 negative）而不希望同时收到模型可能会生成的各种其他标签（例如 good、positive、negative 或 bad），这种限制就非常有用。

注意事项

以下注意事项讨论了当您计划使用响应架构时可能会遇到的限制：

您必须使用 API 来定义和使用响应架构。目前不提供控制台支持。
响应架构的大小会占用输入词元限额。
仅支持某些输出格式，例如 application/json 或 text/x.enum。如需了解详情，请参阅 Gemini API 参考文档中的 responseMimeType 参数。
受控生成支持 Vertex AI 架构参考文档中的一部分。如需了解详情，请参阅支持的架构字段。
复杂的架构可能会导致 InvalidArgument: 400 错误。复杂性可能来自属性名称过长、数组长度限制过长、枚举值过多、对象具有许多可选属性，或者是这些因素的组合。

如果您在使用有效架构时收到此错误，请进行以下一项或多项更改以解决此错误：
- 缩短属性名称或枚举名称。
- 展平嵌套数组。
- 减少具有约束条件的属性数量，例如具有最小值和最大值限制的数字。
- 减少具有复杂约束条件的属性数量，例如采用 date-time 等复杂格式的属性。
- 减少可选属性的数量。
- 减少枚举的有效值数量。

支持的架构字段

受控生成支持以下 Vertex AI 架构中的字段。如果您使用不受支持的字段，Vertex AI 仍可以处理您的请求，但会忽略该字段。

anyOf
enum
format
items
maximum
maxItems
minimum
minItems
nullable
properties
propertyOrdering^*
required

^* propertyOrdering 专门用于受控生成，不属于 Vertex AI 架构。此字段定义了生成属性的顺序。列出的属性必须是唯一的，并且必须是 properties 字典中的有效键。

对于 format 字段，Vertex AI 支持以下值：date、date-time、duration 和 time。每个值的说明和格式在 OpenAPI 计划注册表中都有说明

准备工作

通过定义响应架构指定模型输出的结构、字段名称以及每个字段的预期数据类型。仅使用注意事项部分中列出的受支持字段。系统会忽略所有其他字段。

仅在 responseSchema 字段中添加响应架构。请勿在输入提示中重复使用架构。如果您这样做，生成的输出可能会降低质量。

如需查看示例架构，请参阅示例架构和模型响应部分。

模型行为和响应架构

模型生成响应时，会使用提示中的字段名称和上下文。因此，我们建议您使用清晰的结构和明确的字段名称，让您的意图清晰明了。

默认情况下，字段是可选的，这意味着模型可以填充或跳过字段。您可以将字段设置为必填字段，以强制模型提供值。如果关联的输入提示中上下文不足，模型会主要基于其训练所依据的数据生成回答。

如果您未看到预期结果，请在输入提示中添加更多上下文或修改响应架构。例如，查看非受控生成情况下模型的响应，以了解模型的响应方式。然后，您可以更新响应架构，使其更贴合模型的输出。

发送包含响应架构的提示

默认情况下，所有字段均为可选字段，这意味着模型可能会生成针对某个字段的响应。如需强制模型始终生成针对某个字段的响应，请将该字段设置为必填字段。

Gen AI SDK for Python

了解如何安装或更新 Google Gen AI SDK for Python。
如需了解详情，请参阅 Gen AI SDK for Python API 参考文档或 python-genai GitHub 代码库。
设置环境变量以将 Gen AI SDK 与 Vertex AI 搭配使用：

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=us-central1
export GOOGLE_GENAI_USE_VERTEXAI=True

from google import genai
from google.genai.types import HttpOptions

response_schema = {
    "type": "ARRAY",
    "items": {
        "type": "OBJECT",
        "properties": {
            "recipe_name": {"type": "STRING"},
            "ingredients": {"type": "ARRAY", "items": {"type": "STRING"}},
        },
        "required": ["recipe_name", "ingredients"],
    },
}

prompt = """
    List a few popular cookie recipes.
"""

client = genai.Client(http_options=HttpOptions(api_version="v1"))
response = client.models.generate_content(
    model="gemini-2.0-flash-001",
    contents=prompt,
    config={
        "response_mime_type": "application/json",
        "response_schema": response_schema,
    },
)

print(response.text)
# Example output:
# [
#     {
#         "ingredients": [
#             "2 1/4 cups all-purpose flour",
#             "1 teaspoon baking soda",
#             "1 teaspoon salt",
#             "1 cup (2 sticks) unsalted butter, softened",
#             "3/4 cup granulated sugar",
#             "3/4 cup packed brown sugar",
#             "1 teaspoon vanilla extract",
#             "2 large eggs",
#             "2 cups chocolate chips",
#         ],
#         "recipe_name": "Chocolate Chip Cookies",
#     }
# ]

Python 版 Vertex AI SDK

如需了解如何安装或更新 Vertex AI SDK for Python，请参阅安装 Vertex AI SDK for Python。如需了解详情，请参阅 Python 版 Vertex AI SDK API 参考文档。

import vertexai

from vertexai.generative_models import GenerationConfig, GenerativeModel

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

response_schema = {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "recipe_name": {
                "type": "string",
            },
        },
        "required": ["recipe_name"],
    },
}

model = GenerativeModel("gemini-1.5-pro-002")

response = model.generate_content(
    "List a few popular cookie recipes",
    generation_config=GenerationConfig(
        response_mime_type="application/json", response_schema=response_schema
    ),
)

print(response.text)
# Example response:
# [
#     {"recipe_name": "Chocolate Chip Cookies"},
#     {"recipe_name": "Peanut Butter Cookies"},
#     {"recipe_name": "Snickerdoodles"},
#     {"recipe_name": "Oatmeal Raisin Cookies"},
# ]

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// controlledGenerationResponseSchema shows how to make sure the generated output
// will always be valid JSON and adhere to a specific schema.
func controlledGenerationResponseSchema(w io.Writer, projectID, location, modelName string) error {
	// location := "us-central1"
	// modelName := "gemini-1.5-pro-001"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	model.GenerationConfig.ResponseMIMEType = "application/json"

	// Build an OpenAPI schema, in memory
	model.GenerationConfig.ResponseSchema = &genai.Schema{
		Type: genai.TypeArray,
		Items: &genai.Schema{
			Type: genai.TypeObject,
			Properties: map[string]*genai.Schema{
				"recipe_name": {
					Type: genai.TypeString,
				},
			},
			Required: []string{
				"recipe_name",
			},
		},
	}

	prompt := "List a few popular cookie recipes"

	res, err := model.GenerateContent(ctx, genai.Text(prompt))
	if err != nil {
		return fmt.Errorf("unable to generate contents: %v", err)
	}

	if len(res.Candidates) == 0 ||
		len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	fmt.Fprint(w, res.Candidates[0].Content.Parts[0])
	return nil
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.api.GenerationConfig;
import com.google.cloud.vertexai.api.Schema;
import com.google.cloud.vertexai.api.Type;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.ResponseHandler;
import java.io.IOException;
import java.util.Collections;

public class ControlledGenerationSchema {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "genai-java-demos";
    String location = "us-central1";
    String modelName = "gemini-1.5-pro-001";

    controlGenerationWithJsonSchema(projectId, location, modelName);
  }

  // Generate responses that are always valid JSON and comply with a JSON schema
  public static String controlGenerationWithJsonSchema(
      String projectId, String location, String modelName)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs
    // to be created once, and can be reused for multiple requests.
    try (VertexAI vertexAI = new VertexAI(projectId, location)) {
      GenerationConfig generationConfig = GenerationConfig.newBuilder()
          .setResponseMimeType("application/json")
          .setResponseSchema(Schema.newBuilder()
              .setType(Type.ARRAY)
              .setItems(Schema.newBuilder()
                  .setType(Type.OBJECT)
                  .putProperties("recipe_name", Schema.newBuilder().setType(Type.STRING).build())
                  .addAllRequired(Collections.singletonList("recipe_name"))
                  .build())
              .build())
          .build();

      GenerativeModel model = new GenerativeModel(modelName, vertexAI)
          .withGenerationConfig(generationConfig);

      GenerateContentResponse response = model.generateContent(
          "List a few popular cookie recipes."
      );

      String output = ResponseHandler.getText(response);
      System.out.println(output);
      return output;
    }
  }
}

REST

在使用任何请求数据之前，请先进行以下替换：

GENERATE_RESPONSE_METHOD：您希望模型生成的回答类型。选择一种方法来生成您希望返回模型回答的方式：
- streamGenerateContent：在生成回答时进行流式传输，以降低真人受众群体对于延迟的感知度。
- generateContent：回答在完全生成后返回。
LOCATION：处理请求的区域。
PROJECT_ID：您的项目 ID。
MODEL_ID：您要使用的多模态模型 ID。选项包括：
- gemini-1.5-flash
- gemini-1.5-pro
ROLE：与内容关联的对话中的角色。即使在单轮应用场景中，也需要指定角色。可接受的值包括：
- USER：指定由您发送的内容。
TEXT：要包含在提示中的文本说明。
RESPONSE_MIME_TYPE：生成的候选文本的格式类型。如需查看受支持的值的列表，请参阅 Gemini API 中的 responseMimeType 参数。
RESPONSE_SCHEMA：生成响应时模型应遵循的架构。如需了解详情，请参阅架构参考文档。

HTTP 方法和网址：

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:GENERATE_RESPONSE_METHOD

请求 JSON 正文：

{
  "contents": {
    "role": "ROLE",
    "parts": {
      "text": "TEXT"
    }
  },
  "generation_config": {
    "responseMimeType": "RESPONSE_MIME_TYPE",
    "responseSchema": RESPONSE_SCHEMA,
  }
}

如需发送请求，请选择以下方式之一：

curl

注意：以下命令假定您已使用您的用户账号通过运行 gcloud init 或 gcloud auth login 登录 gcloud CLI，或者使用了 Cloud Shell，这会使您自动登录 gcloud CLI。您可以运行 gcloud auth list 来检查当前活跃的账号。

将请求正文保存在名为 request.json 的文件中，然后执行以下命令：

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:GENERATE_RESPONSE_METHOD"

PowerShell

注意：以下命令假定您已使用您的用户账号通过运行 gcloud init 或 gcloud auth login 登录 gcloud CLI。您可以运行 gcloud auth list 来检查当前活跃的账号。

将请求正文保存在名为 request.json 的文件中，然后执行以下命令：

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:GENERATE_RESPONSE_METHOD" | Select-Object -Expand Content

您应该收到类似以下内容的 JSON 响应。

响应

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": "[{\"recipe_name\": \"Chocolate Chip Cookies\"}, {\"recipe_name\": \"Peanut Butter Cookies\"}, {\"recipe_name\": \"Oatmeal Raisin Cookies\"}, {\"recipe_name\": \"Sugar Cookies\"}, {\"recipe_name\": \"Snickerdoodles\"}]"
          }
        ]
      },
      "finishReason": "STOP",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.08021325,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.0921962
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.14730969,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.08866235
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.13432105,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.07172113
        },
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE",
          "probabilityScore": 0.12787028,
          "severity": "HARM_SEVERITY_NEGLIGIBLE",
          "severityScore": 0.10017223
        }
      ]
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 7,
    "candidatesTokenCount": 55,
    "totalTokenCount": 62
  }
}

示例 curl 命令

LOCATION="us-central1"
MODEL_ID="gemini-1.5-pro"
PROJECT_ID="test-project"
GENERATE_RESPONSE_METHOD="generateContent"

cat << EOF > request.json
{
  "contents": {
    "role": "user",
    "parts": {
      "text": "List a few popular cookie recipes."
    }
  },
  "generation_config": {
    "maxOutputTokens": 2048,
    "responseMimeType": "application/json",
    "responseSchema": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "recipe_name": {
            "type": "string",
          },
        },
        "required": ["recipe_name"],
      },
    }
  }
}
EOF

curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:${GENERATE_RESPONSE_METHOD} -d \
-d `@request.json`

JSON 输出的示例架构

以下部分演示了各种示例提示和响应架构。每个代码示例后还包含模型响应示例。

以嵌套列表形式汇总评价评分
以数组形式预测一周内每一天的天气
使用定义明确的枚举对商品进行分类
识别图片中的对象

汇总评价评分

以下示例输出一个对象数组，其中每个对象都有两个属性：评分和冰淇淋口味的名称。

Python 版 Vertex AI SDK

如需了解如何安装或更新 Vertex AI SDK for Python，请参阅安装 Vertex AI SDK for Python。如需了解详情，请参阅 Python 版 Vertex AI SDK API 参考文档。

import vertexai

from vertexai.generative_models import GenerationConfig, GenerativeModel

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

response_schema = {
    "type": "ARRAY",
    "items": {
        "type": "ARRAY",
        "items": {
            "type": "OBJECT",
            "properties": {
                "rating": {"type": "INTEGER"},
                "flavor": {"type": "STRING"},
            },
        },
    },
}

prompt = """
    Reviews from our social media:
    - "Absolutely loved it! Best ice cream I've ever had." Rating: 4, Flavor: Strawberry Cheesecake
    - "Quite good, but a bit too sweet for my taste." Rating: 1, Flavor: Mango Tango
"""

model = GenerativeModel("gemini-1.5-pro-002")

response = model.generate_content(
    prompt,
    generation_config=GenerationConfig(
        response_mime_type="application/json", response_schema=response_schema
    ),
)

print(response.text)
# Example response:
# [
#     [
#         {"flavor": "Strawberry Cheesecake", "rating": 4},
#         {"flavor": "Mango Tango", "rating": 1},
#     ]
# ]

模型响应示例

candidates {
  content {
    role: "model"
    parts {
      text: "[\n    [\n        {\n            \"rating\": 4\n        },\n        {\n            \"flavor\": \"Strawberry Cheesecake\"\n        },\n        {\n            \"rating\": 1\n        },\n        {\n            \"flavor\": \"Mango Tango\"\n        }\n    ]\n] "
    }
  }
  finish_reason: STOP
  safety_ratings {
    category: HARM_CATEGORY_HATE_SPEECH
    probability: NEGLIGIBLE
    probability_score: 0.1139734759926796
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.10070161521434784
  }
  safety_ratings {
    category: HARM_CATEGORY_DANGEROUS_CONTENT
    probability: NEGLIGIBLE
    probability_score: 0.13695430755615234
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.12241825461387634
  }
  safety_ratings {
    category: HARM_CATEGORY_HARASSMENT
    probability: NEGLIGIBLE
    probability_score: 0.11676400154829025
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.05310790613293648
  }
  safety_ratings {
    category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    probability: NEGLIGIBLE
    probability_score: 0.10521054267883301
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.08299414813518524
  }
}
usage_metadata {
  prompt_token_count: 61
  candidates_token_count: 66
  total_token_count: 127
}

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// controlledGenerationResponseSchema2 shows how to make sure the generated output
// will always be valid JSON and adhere to a specific schema.
func controlledGenerationResponseSchema2(w io.Writer, projectID, location, modelName string) error {
	// location := "us-central1"
	// modelName := "gemini-1.5-pro-001"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	model.GenerationConfig.ResponseMIMEType = "application/json"

	// Build an OpenAPI schema, in memory
	model.GenerationConfig.ResponseSchema = &genai.Schema{
		Type: genai.TypeArray,
		Items: &genai.Schema{
			Type: genai.TypeArray,
			Items: &genai.Schema{
				Type: genai.TypeObject,
				Properties: map[string]*genai.Schema{
					"rating": {
						Type: genai.TypeInteger,
					},
					"flavor": {
						Type: genai.TypeString,
					},
				},
			},
		},
	}

	prompt := `
		Reviews from our social media:

		- "Absolutely loved it! Best ice cream I've ever had." Rating: 4, Flavor: Strawberry Cheesecake
		- "Quite good, but a bit too sweet for my taste." Rating: 1, Flavor: Mango Tango
	`

	res, err := model.GenerateContent(ctx, genai.Text(prompt))
	if err != nil {
		return fmt.Errorf("unable to generate contents: %v", err)
	}

	if len(res.Candidates) == 0 ||
		len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	fmt.Fprint(w, res.Candidates[0].Content.Parts[0])
	return nil
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.api.GenerationConfig;
import com.google.cloud.vertexai.api.Schema;
import com.google.cloud.vertexai.api.Type;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.ResponseHandler;
import java.io.IOException;
import java.util.Arrays;

public class ControlledGenerationSchema2 {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "genai-java-demos";
    String location = "us-central1";
    String modelName = "gemini-1.5-pro-001";

    controlGenerationWithJsonSchema2(projectId, location, modelName);
  }

  // Generate responses that are always valid JSON and comply with a JSON schema
  public static String controlGenerationWithJsonSchema2(
      String projectId, String location, String modelName)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs
    // to be created once, and can be reused for multiple requests.
    try (VertexAI vertexAI = new VertexAI(projectId, location)) {
      GenerationConfig generationConfig = GenerationConfig.newBuilder()
          .setResponseMimeType("application/json")
          .setResponseSchema(Schema.newBuilder()
              .setType(Type.ARRAY)
              .setItems(Schema.newBuilder()
                  .setType(Type.OBJECT)
                  .putProperties("rating", Schema.newBuilder().setType(Type.INTEGER).build())
                  .putProperties("flavor", Schema.newBuilder().setType(Type.STRING).build())
                  .addAllRequired(Arrays.asList("rating", "flavor"))
                  .build())
              .build())
          .build();

      GenerativeModel model = new GenerativeModel(modelName, vertexAI)
          .withGenerationConfig(generationConfig);

      GenerateContentResponse response = model.generateContent(
          "Reviews from our social media:\n"
              + "\"Absolutely loved it! Best ice cream I've ever had.\" "
              + "Rating: 4, Flavor: Strawberry Cheesecake\n"
              + "\"Quite good, but a bit too sweet for my taste.\" "
              + "Rating: 1, Flavor: Mango Tango"
      );

      String output = ResponseHandler.getText(response);
      System.out.println(output);
      return output;
    }
  }
}

预测一周内每一天的天气

以下示例针对一周内的每一天输出一个 forecast 对象，其中包含一组属性（例如当天的预期温度和湿度）。某些属性会设为可为 null，以便模型在上下文不足以生成有意义的响应时返回 null。此策略有助于减少模型幻觉。

Gen AI SDK for Python

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=us-central1
export GOOGLE_GENAI_USE_VERTEXAI=True

from google import genai
from google.genai.types import GenerateContentConfig, HttpOptions

response_schema = {
    "type": "OBJECT",
    "properties": {
        "forecast": {
            "type": "ARRAY",
            "items": {
                "type": "OBJECT",
                "properties": {
                    "Day": {"type": "STRING", "nullable": True},
                    "Forecast": {"type": "STRING", "nullable": True},
                    "Temperature": {"type": "INTEGER", "nullable": True},
                    "Humidity": {"type": "STRING", "nullable": True},
                    "Wind Speed": {"type": "INTEGER", "nullable": True},
                },
                "required": ["Day", "Temperature", "Forecast", "Wind Speed"],
            },
        }
    },
}

prompt = """
    The week ahead brings a mix of weather conditions.
    Sunday is expected to be sunny with a temperature of 77°F and a humidity level of 50%. Winds will be light at around 10 km/h.
    Monday will see partly cloudy skies with a slightly cooler temperature of 72°F and the winds will pick up slightly to around 15 km/h.
    Tuesday brings rain showers, with temperatures dropping to 64°F and humidity rising to 70%.
    Wednesday may see thunderstorms, with a temperature of 68°F.
    Thursday will be cloudy with a temperature of 66°F and moderate humidity at 60%.
    Friday returns to partly cloudy conditions, with a temperature of 73°F and the Winds will be light at 12 km/h.
    Finally, Saturday rounds off the week with sunny skies, a temperature of 80°F, and a humidity level of 40%. Winds will be gentle at 8 km/h.
"""

client = genai.Client(http_options=HttpOptions(api_version="v1"))
response = client.models.generate_content(
    model="gemini-2.0-flash-001",
    contents=prompt,
    config=GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=response_schema,
    ),
)

print(response.text)
# Example output:
# {"forecast": [{"Day": "Sunday", "Forecast": "sunny", "Temperature": 77, "Wind Speed": 10, "Humidity": "50%"},
#   {"Day": "Monday", "Forecast": "partly cloudy", "Temperature": 72, "Wind Speed": 15},
#   {"Day": "Tuesday", "Forecast": "rain showers", "Temperature": 64, "Wind Speed": null, "Humidity": "70%"},
#   {"Day": "Wednesday", "Forecast": "thunderstorms", "Temperature": 68, "Wind Speed": null},
#   {"Day": "Thursday", "Forecast": "cloudy", "Temperature": 66, "Wind Speed": null, "Humidity": "60%"},
#   {"Day": "Friday", "Forecast": "partly cloudy", "Temperature": 73, "Wind Speed": 12},
#   {"Day": "Saturday", "Forecast": "sunny", "Temperature": 80, "Wind Speed": 8, "Humidity": "40%"}]}

Python 版 Vertex AI SDK

如需了解如何安装或更新 Vertex AI SDK for Python，请参阅安装 Vertex AI SDK for Python。如需了解详情，请参阅 Python 版 Vertex AI SDK API 参考文档。

import vertexai

from vertexai.generative_models import GenerationConfig, GenerativeModel

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"

vertexai.init(project=PROJECT_ID, location="us-central1")

response_schema = {
    "type": "OBJECT",
    "properties": {
        "forecast": {
            "type": "ARRAY",
            "items": {
                "type": "OBJECT",
                "properties": {
                    "Day": {"type": "STRING", "nullable": True},
                    "Forecast": {"type": "STRING", "nullable": True},
                    "Temperature": {"type": "INTEGER", "nullable": True},
                    "Humidity": {"type": "STRING", "nullable": True},
                    "Wind Speed": {"type": "INTEGER", "nullable": True},
                },
                "required": ["Day", "Temperature", "Forecast", "Wind Speed"],
            },
        }
    },
}

prompt = """
    The week ahead brings a mix of weather conditions.
    Sunday is expected to be sunny with a temperature of 77°F and a humidity level of 50%. Winds will be light at around 10 km/h.
    Monday will see partly cloudy skies with a slightly cooler temperature of 72°F and the winds will pick up slightly to around 15 km/h.
    Tuesday brings rain showers, with temperatures dropping to 64°F and humidity rising to 70%.
    Wednesday may see thunderstorms, with a temperature of 68°F.
    Thursday will be cloudy with a temperature of 66°F and moderate humidity at 60%.
    Friday returns to partly cloudy conditions, with a temperature of 73°F and the Winds will be light at 12 km/h.
    Finally, Saturday rounds off the week with sunny skies, a temperature of 80°F, and a humidity level of 40%. Winds will be gentle at 8 km/h.
"""

model = GenerativeModel("gemini-1.5-pro-002")

response = model.generate_content(
    prompt,
    generation_config=GenerationConfig(
        response_mime_type="application/json", response_schema=response_schema
    ),
)

print(response.text)
# Example response:
#  {"forecast": [{"Day": "Sunday", "Forecast": "Sunny", "Temperature": 77, "Humidity": "50%", "Wind Speed": 10},
#     {"Day": "Monday", "Forecast": "Partly Cloudy", "Temperature": 72, "Wind Speed": 15},
#     {"Day": "Tuesday", "Forecast": "Rain Showers", "Temperature": 64, "Humidity": "70%"},
#     {"Day": "Wednesday", "Forecast": "Thunderstorms", "Temperature": 68},
#     {"Day": "Thursday", "Forecast": "Cloudy", "Temperature": 66, "Humidity": "60%"},
#     {"Day": "Friday", "Forecast": "Partly Cloudy", "Temperature": 73, "Wind Speed": 12},
#     {"Day": "Saturday", "Forecast": "Sunny", "Temperature": 80, "Humidity": "40%", "Wind Speed": 8}]}

模型响应示例

candidates {
  content {
    role: "model"
    parts {
      text: "{\"forecast\": [{\"Day\": \"Sunday\", \"Forecast\": \"sunny\", \"Humidity\": \"50%\", \"Temperature\": 77, \"Wind Speed\": 10}, {\"Day\": \"Monday\", \"Forecast\": \"partly cloudy\", \"Humidity\": null, \"Temperature\": 72, \"Wind Speed\": 15}, {\"Day\": \"Tuesday\", \"Forecast\": \"rain showers\", \"Humidity\": \"70%\", \"Temperature\": 64, \"Wind Speed\": null}, {\"Day\": \"Wednesday\", \"Forecast\": \"thunderstorms\", \"Humidity\": null, \"Temperature\": 68, \"Wind Speed\": null}, {\"Day\": \"Thursday\", \"Forecast\": \"cloudy\", \"Humidity\": \"60%\", \"Temperature\": 66, \"Wind Speed\": null}, {\"Day\": \"Friday\", \"Forecast\": \"partly cloudy\", \"Humidity\": null, \"Temperature\": 73, \"Wind Speed\": 12}, {\"Day\": \"Saturday\", \"Forecast\": \"sunny\", \"Humidity\": \"40%\", \"Temperature\": 80, \"Wind Speed\": 8}]}"
    }
  }
  finish_reason: STOP
  safety_ratings {
    category: HARM_CATEGORY_HATE_SPEECH
    probability: NEGLIGIBLE
    probability_score: 0.1037486344575882
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.09670579433441162
  }
  safety_ratings {
    category: HARM_CATEGORY_DANGEROUS_CONTENT
    probability: NEGLIGIBLE
    probability_score: 0.18126320838928223
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.10052486509084702
  }
  safety_ratings {
    category: HARM_CATEGORY_HARASSMENT
    probability: NEGLIGIBLE
    probability_score: 0.15960998833179474
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.09518112242221832
  }
  safety_ratings {
    category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    probability: NEGLIGIBLE
    probability_score: 0.1388116478919983
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.10539454221725464
  }
}
usage_metadata {
  prompt_token_count: 280
  candidates_token_count: 249
  total_token_count: 529
}

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// controlledGenerationResponseSchema3 shows how to make sure the generated output
// will always be valid JSON and adhere to a specific schema.
func controlledGenerationResponseSchema3(w io.Writer, projectID, location, modelName string) error {
	// location := "us-central1"
	// modelName := "gemini-1.5-pro-001"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	model.GenerationConfig.ResponseMIMEType = "application/json"

	// Build an OpenAPI schema, in memory
	model.GenerationConfig.ResponseSchema = &genai.Schema{
		Type: genai.TypeObject,
		Properties: map[string]*genai.Schema{
			"forecast": {
				Type: genai.TypeArray,
				Items: &genai.Schema{
					Type: genai.TypeObject,
					Properties: map[string]*genai.Schema{
						"Day": {
							Type: genai.TypeString,
						},
						"Forecast": {
							Type: genai.TypeString,
						},
						"Humidity": {
							Type: genai.TypeString,
						},
						"Temperature": {
							Type: genai.TypeInteger,
						},
						"Wind Speed": {
							Type: genai.TypeInteger,
						},
					},
					Required: []string{"Day", "Temperature", "Forecast"},
				},
			},
		},
	}

	prompt := `
		The week ahead brings a mix of weather conditions.
		Sunday is expected to be sunny with a temperature of 77°F and a humidity level of 50%. Winds will be light at around 10 km/h.
		Monday will see partly cloudy skies with a slightly cooler temperature of 72°F and humidity increasing to 55%. Winds will pick up slightly to around 15 km/h.
		Tuesday brings rain showers, with temperatures dropping to 64°F and humidity rising to 70%. Expect stronger winds at 20 km/h.
		Wednesday may see thunderstorms, with a temperature of 68°F and high humidity of 75%. Winds will be gusty at 25 km/h.
		Thursday will be cloudy with a temperature of 66°F and moderate humidity at 60%. Winds will ease slightly to 18 km/h.
		Friday returns to partly cloudy conditions, with a temperature of 73°F and lower humidity at 45%. Winds will be light at 12 km/h.
		Finally, Saturday rounds off the week with sunny skies, a temperature of 80°F, and a humidity level of 40%. Winds will be gentle at 8 km/h.
	`

	res, err := model.GenerateContent(ctx, genai.Text(prompt))
	if err != nil {
		return fmt.Errorf("unable to generate contents: %v", err)
	}

	if len(res.Candidates) == 0 ||
		len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	fmt.Fprint(w, res.Candidates[0].Content.Parts[0])
	return nil
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.api.GenerationConfig;
import com.google.cloud.vertexai.api.Schema;
import com.google.cloud.vertexai.api.Type;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.ResponseHandler;
import java.io.IOException;
import java.util.Arrays;

public class ControlledGenerationSchema3 {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "genai-java-demos";
    String location = "us-central1";
    String modelName = "gemini-1.5-pro-001";

    controlGenerationWithJsonSchema3(projectId, location, modelName);
  }

  // Generate responses that are always valid JSON and comply with a JSON schema
  public static String controlGenerationWithJsonSchema3(
      String projectId, String location, String modelName)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs
    // to be created once, and can be reused for multiple requests.
    try (VertexAI vertexAI = new VertexAI(projectId, location)) {
      GenerationConfig generationConfig = GenerationConfig.newBuilder()
          .setResponseMimeType("application/json")
          .setResponseSchema(Schema.newBuilder()
              .setType(Type.OBJECT)
              .putProperties("forecast", Schema.newBuilder()
                      .setType(Type.ARRAY)
                      .setItems(Schema.newBuilder()
                          .setType(Type.OBJECT)
                          .putProperties("Day", Schema.newBuilder()
                              .setType(Type.STRING)
                              .build())
                          .putProperties("Forecast", Schema.newBuilder()
                              .setType(Type.STRING)
                              .build())
                          .putProperties("Humidity", Schema.newBuilder()
                              .setType(Type.STRING)
                              .build())
                          .putProperties("Temperature", Schema.newBuilder()
                              .setType(Type.INTEGER)
                              .build())
                          .putProperties("Wind Speed", Schema.newBuilder()
                              .setType(Type.INTEGER)
                              .build())
                          .addAllRequired(Arrays.asList("Day", "Temperature", "Forecast"))
                          .build())
                      .build())
              )
          .build();

      GenerativeModel model = new GenerativeModel(modelName, vertexAI)
          .withGenerationConfig(generationConfig);

      GenerateContentResponse response = model.generateContent(
          "The week ahead brings a mix of weather conditions.\n"
              + "Sunday is expected to be sunny with a temperature of 77°F and a humidity level "
              + "of 50%. Winds will be light at around 10 km/h.\n"
              + "Monday will see partly cloudy skies with a slightly cooler temperature of 72°F "
              + "and humidity increasing to 55%. Winds will pick up slightly to around 15 km/h.\n"
              + "Tuesday brings rain showers, with temperatures dropping to 64°F and humidity"
              + "rising to 70%. Expect stronger winds at 20 km/h.\n"
              + "Wednesday may see thunderstorms, with a temperature of 68°F and high humidity "
              + "of 75%. Winds will be gusty at 25 km/h.\n"
              + "Thursday will be cloudy with a temperature of 66°F and moderate humidity at 60%. "
              + "Winds will ease slightly to 18 km/h.\n"
              + "Friday returns to partly cloudy conditions, with a temperature of 73°F and lower "
              + "humidity at 45%. Winds will be light at 12 km/h.\n"
              + "Finally, Saturday rounds off the week with sunny skies, a temperature of 80°F, "
              + "and a humidity level of 40%. Winds will be gentle at 8 km/h."
      );

      String output = ResponseHandler.getText(response);
      System.out.println(output);
      return output;
    }
  }
}

对产品进行分类

以下示例包含枚举，其中模型必须从给定值的列表中对对象的类型和条件进行分类。

Python

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Python 设置说明执行操作。如需了解详情，请参阅 Vertex AI Python API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import vertexai

from vertexai.generative_models import GenerationConfig, GenerativeModel

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

response_schema = {
    "type": "ARRAY",
    "items": {
        "type": "OBJECT",
        "properties": {
            "to_discard": {"type": "INTEGER"},
            "subcategory": {"type": "STRING"},
            "safe_handling": {"type": "INTEGER"},
            "item_category": {
                "type": "STRING",
                "enum": [
                    "clothing",
                    "winter apparel",
                    "specialized apparel",
                    "furniture",
                    "decor",
                    "tableware",
                    "cookware",
                    "toys",
                ],
            },
            "for_resale": {"type": "INTEGER"},
            "condition": {
                "type": "STRING",
                "enum": [
                    "new in package",
                    "like new",
                    "gently used",
                    "used",
                    "damaged",
                    "soiled",
                ],
            },
        },
    },
}

prompt = """
    Item description:
    The item is a long winter coat that has many tears all around the seams and is falling apart.
    It has large questionable stains on it.
"""

model = GenerativeModel("gemini-1.5-pro-002")

response = model.generate_content(
    prompt,
    generation_config=GenerationConfig(
        response_mime_type="application/json", response_schema=response_schema
    ),
)

print(response.text)
# Example response:
# [
#     {
#         "condition": "damaged",
#         "item_category": "clothing",
#         "subcategory": "winter apparel",
#         "to_discard": 123,
#     }
# ]

模型响应示例

candidates {
  content {
    role: "model"
    parts {
      text: " [{\n    \"item_category\": \"winter apparel\",\n    \"subcategory\": \"coat\",\n    \"to_discard\":  1\n  }] "
    }
  }
  finish_reason: STOP
  safety_ratings {
    category: HARM_CATEGORY_HATE_SPEECH
    probability: NEGLIGIBLE
    probability_score: 0.08945459872484207
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.13753245770931244
  }
  safety_ratings {
    category: HARM_CATEGORY_DANGEROUS_CONTENT
    probability: NEGLIGIBLE
    probability_score: 0.19208428263664246
    severity: HARM_SEVERITY_LOW
    severity_score: 0.23810701072216034
  }
  safety_ratings {
    category: HARM_CATEGORY_HARASSMENT
    probability: NEGLIGIBLE
    probability_score: 0.07585817575454712
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.04336579889059067
  }
  safety_ratings {
    category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    probability: NEGLIGIBLE
    probability_score: 0.12667709589004517
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.07396338135004044
  }
}
usage_metadata {
  prompt_token_count: 38
  candidates_token_count: 33
  total_token_count: 71
}

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// controlledGenerationResponseSchema4 shows how to make sure the generated output
// will always be valid JSON and adhere to a specific schema.
func controlledGenerationResponseSchema4(w io.Writer, projectID, location, modelName string) error {
	// location := "us-central1"
	// modelName := "gemini-1.5-pro-001"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	model.GenerationConfig.ResponseMIMEType = "application/json"

	// Build an OpenAPI schema, in memory
	model.GenerationConfig.ResponseSchema = &genai.Schema{
		Type: genai.TypeArray,
		Items: &genai.Schema{
			Type: genai.TypeObject,
			Properties: map[string]*genai.Schema{
				"to_discard":    {Type: genai.TypeInteger},
				"subcategory":   {Type: genai.TypeString},
				"safe_handling": {Type: genai.TypeString},
				"item_category": {
					Type: genai.TypeString,
					Enum: []string{
						"clothing",
						"winter apparel",
						"specialized apparel",
						"furniture",
						"decor",
						"tableware",
						"cookware",
						"toys",
					},
				},
				"for_resale": {Type: genai.TypeInteger},
				"condition": {
					Type: genai.TypeString,
					Enum: []string{
						"new in package",
						"like new",
						"gently used",
						"used",
						"damaged",
						"soiled",
					},
				},
			},
		},
	}

	prompt := `
		Item description:
		The item is a long winter coat that has many tears all around the seams and is falling apart.
		It has large questionable stains on it.
	`

	res, err := model.GenerateContent(ctx, genai.Text(prompt))
	if err != nil {
		return fmt.Errorf("unable to generate contents: %v", err)
	}

	if len(res.Candidates) == 0 ||
		len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	fmt.Fprint(w, res.Candidates[0].Content.Parts[0])
	return nil
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.api.GenerationConfig;
import com.google.cloud.vertexai.api.Schema;
import com.google.cloud.vertexai.api.Type;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.ResponseHandler;
import java.io.IOException;
import java.util.Arrays;

public class ControlledGenerationSchema4 {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "genai-java-demos";
    String location = "us-central1";
    String modelName = "gemini-1.5-pro-001";

    controlGenerationWithJsonSchema4(projectId, location, modelName);
  }

  // Generate responses that are always valid JSON and comply with a JSON schema
  public static String controlGenerationWithJsonSchema4(
      String projectId, String location, String modelName)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs
    // to be created once, and can be reused for multiple requests.
    try (VertexAI vertexAI = new VertexAI(projectId, location)) {
      Schema itemSchema = Schema.newBuilder()
          .setType(Type.OBJECT)
          .putProperties("to_discard", Schema.newBuilder().setType(Type.INTEGER).build())
          .putProperties("subcategory", Schema.newBuilder().setType(Type.STRING).build())
          .putProperties("safe_handling", Schema.newBuilder().setType(Type.INTEGER).build())
          .putProperties("item_category", Schema.newBuilder()
              .setType(Type.STRING)
              .addAllEnum(Arrays.asList(
                  "clothing", "winter apparel", "specialized apparel", "furniture",
                  "decor", "tableware", "cookware", "toys"))
              .build())
          .putProperties("for_resale", Schema.newBuilder().setType(Type.INTEGER).build())
          .putProperties("condition", Schema.newBuilder()
              .setType(Type.STRING)
              .addAllEnum(Arrays.asList(
                  "new in package", "like new", "gently used", "used", "damaged", "soiled"))
              .build())
          .build();

      GenerationConfig generationConfig = GenerationConfig.newBuilder()
          .setResponseMimeType("application/json")
          .setResponseSchema(Schema.newBuilder()
              .setType(Type.ARRAY)
              .setItems(itemSchema)
              .build())
          .build();

      GenerativeModel model = new GenerativeModel(modelName, vertexAI)
          .withGenerationConfig(generationConfig);

      GenerateContentResponse response = model.generateContent(
          "Item description:\n"
              + "The item is a long winter coat that has many tears all around the seams "
              + "and is falling apart.\n"
              + "It has large questionable stains on it."
      );

      String output = ResponseHandler.getText(response);
      System.out.println(output);
      return output;
    }
  }
}

识别图片中的对象

以下示例会识别存储在 Cloud Storage 中的两张图片的对象。

Python

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Python 设置说明执行操作。如需了解详情，请参阅 Vertex AI Python API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import vertexai

from vertexai.generative_models import GenerationConfig, GenerativeModel, Part

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

response_schema = {
    "type": "ARRAY",
    "items": {
        "type": "ARRAY",
        "items": {
            "type": "OBJECT",
            "properties": {
                "object": {"type": "STRING"},
            },
        },
    },
}

model = GenerativeModel("gemini-1.5-pro-002")

response = model.generate_content(
    [
        # Text prompt
        "Generate a list of objects in the images.",
        # Http Image
        Part.from_uri(
            "https://storage.googleapis.com/cloud-samples-data/generative-ai/image/office-desk.jpeg",
            "image/jpeg",
        ),
        # Cloud storage object
        Part.from_uri(
            "gs://cloud-samples-data/generative-ai/image/gardening-tools.jpeg",
            "image/jpeg",
        ),
    ],
    generation_config=GenerationConfig(
        response_mime_type="application/json", response_schema=response_schema
    ),
)

print(response.text)
# Example response:
# [
#     [
#         {"object": "globe"}, {"object": "tablet"}, {"object": "toy car"},
#         {"object": "airplane"}, {"object": "keyboard"}, {"object": "mouse"},
#         {"object": "passport"}, {"object": "sunglasses"}, {"object": "money"},
#         {"object": "notebook"}, {"object": "pen"}, {"object": "coffee cup"},
#     ],
#     [
#         {"object": "watering can"}, {"object": "plant"}, {"object": "flower pot"},
#         {"object": "gloves"}, {"object": "garden tool"},
#     ],
# ]

模型响应示例

candidates {
  content {
    role: "model"
    parts {
      text: "[\n    [\n        {\n            \"object\": \"globe model\"\n        },\n        {\n            \"object\": \"tablet computer\"\n        },\n        {\n            \"object\": \"shopping cart\"\n        },\n        {\n            \"object\": \"Eiffel Tower model\"\n        },\n        {\n            \"object\": \"airplane model\"\n        },\n        {\n            \"object\": \"coffee cup\"\n        },\n        {\n            \"object\": \"computer keyboard\"\n        },\n        {\n            \"object\": \"computer mouse\"\n        },\n        {\n            \"object\": \"passport\"\n        },\n        {\n            \"object\": \"sunglasses\"\n        },\n        {\n            \"object\": \"US Dollar bills\"\n        },\n        {\n            \"object\": \"notepad\"\n        },\n        {\n            \"object\": \"pen\"\n        }\n    ],\n    [\n        {\n            \"object\": \"watering can\"\n        },\n        {\n            \"object\": \"oregano\"\n        },\n        {\n            \"object\": \"flower pot\"\n        },\n        {\n            \"object\": \"flower pot\"\n        },\n        {\n            \"object\": \"gardening gloves\"\n        },\n        {\n            \"object\": \"hand rake\"\n        },\n        {\n            \"object\": \"hand trowel\"\n        },\n        {\n            \"object\": \"grass\"\n        }\n    ]\n] "
    }
  }
  finish_reason: STOP
  safety_ratings {
    category: HARM_CATEGORY_HATE_SPEECH
    probability: NEGLIGIBLE
    probability_score: 0.1872812658548355
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.16357900202274323
  }
  safety_ratings {
    category: HARM_CATEGORY_DANGEROUS_CONTENT
    probability: LOW
    probability_score: 0.37920594215393066
    severity: HARM_SEVERITY_LOW
    severity_score: 0.29320207238197327
  }
  safety_ratings {
    category: HARM_CATEGORY_HARASSMENT
    probability: NEGLIGIBLE
    probability_score: 0.14175598323345184
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.12074951827526093
  }
  safety_ratings {
    category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    probability: NEGLIGIBLE
    probability_score: 0.12241825461387634
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.0955180674791336
  }
}
usage_metadata {
  prompt_token_count: 525
  candidates_token_count: 333
  total_token_count: 858
}

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// controlledGenerationResponseSchema6 shows how to make sure the generated output
// will always be valid JSON and adhere to a specific schema.
func controlledGenerationResponseSchema6(w io.Writer, projectID, location, modelName string) error {
	// location := "us-central1"
	// modelName := "gemini-1.5-pro-001"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	model.GenerationConfig.ResponseMIMEType = "application/json"

	// Build an OpenAPI schema, in memory
	model.GenerationConfig.ResponseSchema = &genai.Schema{
		Type: genai.TypeArray,
		Items: &genai.Schema{
			Type: genai.TypeArray,
			Items: &genai.Schema{
				Type: genai.TypeObject,
				Properties: map[string]*genai.Schema{
					"object": {
						Type: genai.TypeString,
					},
				},
			},
		},
	}

	// These images in Cloud Storage are viewable at
	// https://storage.googleapis.com/cloud-samples-data/generative-ai/image/office-desk.jpeg
	// https://storage.googleapis.com/cloud-samples-data/generative-ai/image/gardening-tools.jpeg

	img1 := genai.FileData{
		MIMEType: "image/jpeg",
		FileURI:  "gs://cloud-samples-data/generative-ai/image/office-desk.jpeg",
	}

	img2 := genai.FileData{
		MIMEType: "image/jpeg",
		FileURI:  "gs://cloud-samples-data/generative-ai/image/gardening-tools.jpeg",
	}

	prompt := "Generate a list of objects in the images."

	res, err := model.GenerateContent(ctx, img1, img2, genai.Text(prompt))
	if err != nil {
		return fmt.Errorf("unable to generate contents: %v", err)
	}

	if len(res.Candidates) == 0 ||
		len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("empty response from model")
	}

	fmt.Fprint(w, res.Candidates[0].Content.Parts[0])
	return nil
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.api.GenerationConfig;
import com.google.cloud.vertexai.api.Schema;
import com.google.cloud.vertexai.api.Type;
import com.google.cloud.vertexai.generativeai.ContentMaker;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.PartMaker;
import com.google.cloud.vertexai.generativeai.ResponseHandler;
import java.io.IOException;

public class ControlledGenerationSchema6 {
  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "genai-java-demos";
    String location = "us-central1";
    String modelName = "gemini-1.5-pro-001";

    controlGenerationWithJsonSchema6(projectId, location, modelName);
  }

  // Generate responses that are always valid JSON and comply with a JSON schema
  public static String controlGenerationWithJsonSchema6(
      String projectId, String location, String modelName)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs
    // to be created once, and can be reused for multiple requests.
    try (VertexAI vertexAI = new VertexAI(projectId, location)) {
      GenerationConfig generationConfig = GenerationConfig.newBuilder()
          .setResponseMimeType("application/json")
          .setResponseSchema(Schema.newBuilder()
              .setType(Type.ARRAY)
              .setItems(Schema.newBuilder()
                  .setType(Type.OBJECT)
                  .putProperties("object", Schema.newBuilder().setType(Type.STRING).build())
                  .build())
              .build())
          .build();

      GenerativeModel model = new GenerativeModel(modelName, vertexAI)
          .withGenerationConfig(generationConfig);

      // These images in Cloud Storage are viewable at
      // https://storage.googleapis.com/cloud-samples-data/generative-ai/image/office-desk.jpeg
      // https://storage.googleapis.com/cloud-samples-data/generative-ai/image/gardening-tools.jpeg

      GenerateContentResponse response = model.generateContent(
          ContentMaker.fromMultiModalData(
              PartMaker.fromMimeTypeAndData("image/jpeg",
                  "gs://cloud-samples-data/generative-ai/image/office-desk.jpeg"),
              PartMaker.fromMimeTypeAndData("image/jpeg",
                  "gs://cloud-samples-data/generative-ai/image/gardening-tools.jpeg"),
              "Generate a list of objects in the images."
          )
      );

      String output = ResponseHandler.getText(response);
      System.out.println(output);
      return output;
    }
  }
}

枚举输出的示例架构

以下示例根据电影的说明来识别其类型。输出是模型从在响应架构中定义的列表值中选择的一个纯文本枚举值。

Gen AI SDK for Python

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values
# with appropriate values for your project.
export GOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_LOCATION=us-central1
export GOOGLE_GENAI_USE_VERTEXAI=True

from google import genai
from google.genai.types import GenerateContentConfig, HttpOptions

client = genai.Client(http_options=HttpOptions(api_version="v1"))
response = client.models.generate_content(
    model="gemini-2.0-flash-001",
    contents="What type of instrument is an oboe?",
    config=GenerateContentConfig(
        response_mime_type="text/x.enum",
        response_schema={
            "type": "STRING",
            "enum": ["Percussion", "String", "Woodwind", "Brass", "Keyboard"],
        },
    ),
)

print(response.text)
# Example output:
# Woodwind

Python 版 Vertex AI SDK

如需了解如何安装或更新 Vertex AI SDK for Python，请参阅安装 Vertex AI SDK for Python。如需了解详情，请参阅 Python 版 Vertex AI SDK API 参考文档。

import vertexai

from vertexai.generative_models import GenerationConfig, GenerativeModel

# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"

vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-pro")

response_schema = {"type": "STRING", "enum": ["drama", "comedy", "documentary"]}

prompt = (
    "The film aims to educate and inform viewers about real-life subjects, events, or people."
    "It offers a factual record of a particular topic by combining interviews, historical footage, "
    "and narration. The primary purpose of a film is to present information and provide insights "
    "into various aspects of reality."
)

response = model.generate_content(
    prompt,
    generation_config=GenerationConfig(
        response_mime_type="text/x.enum", response_schema=response_schema
    ),
)

print(response.text)
# Example response:
#     'documentary'

模型响应示例

candidates {
  content {
    role: "model"
    parts {
      text: "documentary"
    }
  }
  finish_reason: STOP
  safety_ratings {
    category: HARM_CATEGORY_HATE_SPEECH
    probability: NEGLIGIBLE
    probability_score: 0.051025390625
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.08056640625
  }
  safety_ratings {
    category: HARM_CATEGORY_DANGEROUS_CONTENT
    probability: NEGLIGIBLE
    probability_score: 0.1416015625
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.068359375
  }
  safety_ratings {
    category: HARM_CATEGORY_HARASSMENT
    probability: NEGLIGIBLE
    probability_score: 0.11572265625
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.0439453125
  }
  safety_ratings {
    category: HARM_CATEGORY_SEXUALLY_EXPLICIT
    probability: NEGLIGIBLE
    probability_score: 0.099609375
    severity: HARM_SEVERITY_NEGLIGIBLE
    severity_score: 0.146484375
  }
  avg_logprobs: -8.783838711678982e-05
}
usage_metadata {
  prompt_token_count: 33
  candidates_token_count: 2
  total_token_count: 35
}

Go

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Go 设置说明执行操作。如需了解详情，请参阅 Vertex AI Go API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。如需了解详情，请参阅为本地开发环境设置身份验证。

import (
	"context"
	"errors"
	"fmt"
	"io"

	"cloud.google.com/go/vertexai/genai"
)

// controlledGenerationResponseSchemaEnum demonstrates how to constrain model responses
// to a predefined set of enum values for genre classification.
func controlledGenerationResponseSchemaEnum(w io.Writer, projectID, location, modelName string) error {
	// location = "us-central1"
	// modelName = "gemini-1.5-pro-001"
	ctx := context.Background()
	client, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		return fmt.Errorf("failed to create GenAI client: %w", err)
	}
	defer client.Close()

	model := client.GenerativeModel(modelName)

	model.GenerationConfig.ResponseMIMEType = "text/x.enum"
	model.GenerationConfig.ResponseSchema = &genai.Schema{
		Type: genai.TypeString,
		Enum: []string{"drama", "comedy", "documentary"},
	}

	prompt := `
The film aims to educate and inform viewers about real-life subjects, events, or people.
It offers a factual record of a particular topic by combining interviews, historical footage,
and narration. The primary purpose of a film is to present information and provide insights
into various aspects of reality.
`

	res, err := model.GenerateContent(ctx, genai.Text(prompt))
	if err != nil {
		return fmt.Errorf("failed to generate content: %w", err)
	}

	if len(res.Candidates) == 0 || len(res.Candidates[0].Content.Parts) == 0 {
		return errors.New("got empty response from model")
	}

	fmt.Fprintf(w, "Candidate label: %q", res.Candidates[0].Content.Parts[0])
	// Example response:
	// Candidate label: "documentary"

	return nil
}