从图片生成文本

此示例演示了如何使用 Gemini 模型从图片生成文本。

深入探索

如需查看包含此代码示例的详细文档，请参阅以下内容：

• 使用 Gemini API 生成内容
• 图片推理
• 向 Vertex AI API for Gemini 发送请求
• Vertex AI 客户端库

代码示例

C++

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 C++ 设置说明执行操作。 如需了解详情，请参阅 Vertex AI C++ API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

namespace vertex_ai = ::google::cloud::aiplatform_v1;
namespace vertex_ai_proto = ::google::cloud::aiplatform::v1;
[](std::string const& project_id, std::string const& location_id,
   std::string const& model, std::string const& prompt,
   std::string const& mime_type, std::string const& file_uri) {
  google::cloud::Location location(project_id, location_id);
  auto client = vertex_ai::PredictionServiceClient(
      vertex_ai::MakePredictionServiceConnection(location.location_id()));

  vertex_ai_proto::GenerateContentRequest request;
  request.set_model(location.FullName() + "/publishers/google/models/" +
                    model);
  auto generation_config = request.mutable_generation_config();
  generation_config->set_temperature(0.4f);
  generation_config->set_top_k(32);
  generation_config->set_top_p(1);
  generation_config->set_max_output_tokens(2048);

  auto contents = request.add_contents();
  contents->set_role("user");
  contents->add_parts()->set_text(prompt);
  auto image_part = contents->add_parts();
  image_part->mutable_file_data()->set_file_uri(file_uri);
  image_part->mutable_file_data()->set_mime_type(mime_type);

  auto response = client.GenerateContent(request);
  if (!response) throw std::move(response).status();

  for (auto const& candidate : response->candidates()) {
    for (auto const& p : candidate.content().parts()) {
      std::cout << p.text() << "\n";
    }
  }
}

Java

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Java 设置说明执行操作。 如需了解详情，请参阅 Vertex AI Java API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

import com.google.cloud.vertexai.VertexAI;
import com.google.cloud.vertexai.api.GenerateContentResponse;
import com.google.cloud.vertexai.generativeai.ContentMaker;
import com.google.cloud.vertexai.generativeai.GenerativeModel;
import com.google.cloud.vertexai.generativeai.PartMaker;
import java.io.IOException;

public class Quickstart {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-google-cloud-project-id";
    String location = "us-central1";
    String modelName = "gemini-1.5-flash-001";

    String output = quickstart(projectId, location, modelName);
    System.out.println(output);
  }

  // Analyzes the provided Multimodal input.
  public static String quickstart(String projectId, String location, String modelName)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs
    // to be created once, and can be reused for multiple requests.
    try (VertexAI vertexAI = new VertexAI(projectId, location)) {
      String imageUri = "gs://generativeai-downloads/images/scones.jpg";

      GenerativeModel model = new GenerativeModel(modelName, vertexAI);
      GenerateContentResponse response = model.generateContent(ContentMaker.fromMultiModalData(
          PartMaker.fromMimeTypeAndData("image/png", imageUri),
          "What's in this photo"
      ));

      return response.toString();
    }
  }
}

Node.js

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Node.js 设置说明执行操作。 如需了解详情，请参阅 Vertex AI Node.js API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

const {VertexAI} = require('@google-cloud/vertexai');

/**
 * TODO(developer): Update these variables before running the sample.
 */
async function createNonStreamingMultipartContent(
  projectId = 'PROJECT_ID',
  location = 'us-central1',
  model = 'gemini-1.5-flash-001',
  image = 'gs://generativeai-downloads/images/scones.jpg',
  mimeType = 'image/jpeg'
) {
  // Initialize Vertex with your Cloud project and location
  const vertexAI = new VertexAI({project: projectId, location: location});

  // Instantiate the model
  const generativeVisionModel = vertexAI.getGenerativeModel({
    model: model,
  });

  // For images, the SDK supports both Google Cloud Storage URI and base64 strings
  const filePart = {
    fileData: {
      fileUri: image,
      mimeType: mimeType,
    },
  };

  const textPart = {
    text: 'what is shown in this image?',
  };

  const request = {
    contents: [{role: 'user', parts: [filePart, textPart]}],
  };

  console.log('Prompt Text:');
  console.log(request.contents[0].parts[1].text);

  console.log('Non-Streaming Response Text:');

  // Generate a response
  const response = await generativeVisionModel.generateContent(request);

  // Select the text from the response
  const fullTextResponse =
    response.response.candidates[0].content.parts[0].text;

  console.log(fullTextResponse);
}

Python

在尝试此示例之前，请按照《Vertex AI 快速入门：使用客户端库》中的 Python 设置说明执行操作。 如需了解详情，请参阅 Vertex AI Python API 参考文档。

如需向 Vertex AI 进行身份验证，请设置应用默认凭据。 如需了解详情，请参阅为本地开发环境设置身份验证。

import vertexai

from vertexai.generative_models import GenerativeModel, Part

# TODO(developer): Update & uncomment line below
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")

model = GenerativeModel("gemini-1.5-flash-002")

response = model.generate_content(
    [
        Part.from_uri(
            "gs://cloud-samples-data/generative-ai/image/scones.jpg",
            mime_type="image/jpeg",
        ),
        "What is shown in this image?",
    ]
)

print(response.text)

后续步骤

如需搜索和过滤其他 Google Cloud 产品的代码示例，请参阅 Google Cloud 示例浏览器。