检测多个对象

Vision API 可以使用对象本地化功能检测并提取图片中的多个对象。

对象本地化功能可识别图片中的多个对象,并为图片中的每个对象提供一个 LocalizedObjectAnnotation。每个 LocalizedObjectAnnotation 标识了对象相关信息、对象位置以及对象所属图片区域的矩形边界。

对象本地化功能可识别图片中显眼和不太显眼的对象。

对象信息仅以英文形式返回。Cloud Translation 可以将英文标签翻译成多种其他语言

具有边界框的图片
图片来源:Unsplash 用户 Bogdan Dada(添加了注释)。

例如,API 可能会返回以上图片中对象的以下信息和边界位置数据:

名称 mid 得分 边界
Bicycle wheel /m/01bqk0 0.89648587 (0.32076266, 0.78941387)、(0.43812272, 0.78941387)、(0.43812272, 0.97331065)、(0.32076266, 0.97331065)
Bicycle /m/0199g 0.886761 (0.312, 0.6616471)、(0.638353, 0.6616471)、(0.638353, 0.9705882)、(0.312, 0.9705882)
Bicycle wheel /m/01bqk0 0.6345275 (0.5125398, 0.760708)、(0.6256646, 0.760708)、(0.6256646, 0.94601655)、(0.5125398, 0.94601655)
Picture frame /m/06z37_ 0.6207608 (0.79177403, 0.16160682)、(0.97047985, 0.16160682)、(0.97047985, 0.31348917)、(0.79177403, 0.31348917)
Tire /m/0h9mv 0.55886006 (0.32076266, 0.78941387)、(0.43812272, 0.78941387)、(0.43812272, 0.97331065)、(0.32076266, 0.97331065)
Door /m/02dgv 0.5160098 (0.77569866, 0.37104446)、(0.9412425, 0.37104446)、(0.9412425, 0.81507325)、(0.77569866, 0.81507325)

mid 包含与标签的 Google 知识图谱条目相对应并由机器生成的标识符 (MID)。如需了解如何检查 mid 值,请参阅 Google Knowledge Graph Search API 文档。

亲自尝试

如果您是 Google Cloud 新手,请创建一个帐号来评估 Cloud Vision API 在实际场景中的表现。新客户还可获享 $300 赠金,用于运行、测试和部署工作负载。

免费试用 Cloud Vision API

对象本地化请求

设置您的 GCP 项目和身份验证

检测本地图片中的对象

Vision API 可以将本地图片文件的内容作为 base64 编码的字符串在请求正文中发送,从而对此图片文件执行特征检测。

REST 和命令行

在使用任何请求数据之前,请先进行以下替换:

  • base64-encoded-image:二进制图片数据的 base64 表示(ASCII 字符串)。此字符串应类似于以下字符串:
    • /9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
    如需了解详情,请参阅 base64 编码主题。

HTTP 方法和网址:

POST https://vision.googleapis.com/v1/images:annotate

请求 JSON 正文:

{
  "requests": [
    {
      "image": {
        "content": "base64-encoded-image"
      },
      "features": [
        {
          "maxResults": 10,
          "type": "OBJECT_LOCALIZATION"
        },
      ]
    }
  ]
}

如需发送请求,请选择以下方式之一:

curl

将请求正文保存在名为 request.json 的文件中,然后执行以下命令:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"

PowerShell

将请求正文保存在名为 request.json 的文件中,然后执行以下命令:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

如果请求成功,服务器将返回一个 200 OK HTTP 状态代码以及 JSON 格式的响应。

响应

Go

试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Go 设置说明进行操作。如需了解详情,请参阅 Vision Go API 参考文档


// localizeObjects gets objects and bounding boxes from the Vision API for an image at the given file path.
func localizeObjects(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	f, err := os.Open(file)
	if err != nil {
		return err
	}
	defer f.Close()

	image, err := vision.NewImageFromReader(f)
	if err != nil {
		return err
	}
	annotations, err := client.LocalizeObjects(ctx, image, nil)
	if err != nil {
		return err
	}

	if len(annotations) == 0 {
		fmt.Fprintln(w, "No objects found.")
		return nil
	}

	fmt.Fprintln(w, "Objects:")
	for _, annotation := range annotations {
		fmt.Fprintln(w, annotation.Name)
		fmt.Fprintln(w, annotation.Score)

		for _, v := range annotation.BoundingPoly.NormalizedVertices {
			fmt.Fprintf(w, "(%f,%f)\n", v.X, v.Y)
		}
	}

	return nil
}

Java

在试用此示例之前,请按照Vision API 快速入门:使用客户端库中的 Java 设置说明进行操作。如需了解详情,请参阅 Vision API Java API 参考文档

/**
 * Detects localized objects in the specified local image.
 *
 * @param filePath The path to the file to perform localized object detection on.
 * @throws Exception on errors while closing the client.
 * @throws IOException on Input/Output errors.
 */
public static void detectLocalizedObjects(String filePath) throws IOException {
  List<AnnotateImageRequest> requests = new ArrayList<>();

  ByteString imgBytes = ByteString.readFrom(new FileInputStream(filePath));

  Image img = Image.newBuilder().setContent(imgBytes).build();
  AnnotateImageRequest request =
      AnnotateImageRequest.newBuilder()
          .addFeatures(Feature.newBuilder().setType(Type.OBJECT_LOCALIZATION))
          .setImage(img)
          .build();
  requests.add(request);

  // Initialize client that will be used to send requests. This client only needs to be created
  // once, and can be reused for multiple requests. After completing all of your requests, call
  // the "close" method on the client to safely clean up any remaining background resources.
  try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
    // Perform the request
    BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
    List<AnnotateImageResponse> responses = response.getResponsesList();

    // Display the results
    for (AnnotateImageResponse res : responses) {
      for (LocalizedObjectAnnotation entity : res.getLocalizedObjectAnnotationsList()) {
        System.out.format("Object name: %s%n", entity.getName());
        System.out.format("Confidence: %s%n", entity.getScore());
        System.out.format("Normalized Vertices:%n");
        entity
            .getBoundingPoly()
            .getNormalizedVerticesList()
            .forEach(vertex -> System.out.format("- (%s, %s)%n", vertex.getX(), vertex.getY()));
      }
    }
  }
}

Node.js

试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Node.js 设置说明进行操作。如需了解详情,请参阅 Vision Node.js API 参考文档

// Imports the Google Cloud client libraries
const vision = require('@google-cloud/vision');
const fs = require('fs');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const fileName = `/path/to/localImage.png`;
const request = {
  image: {content: fs.readFileSync(fileName)},
};

const [result] = await client.objectLocalization(request);
const objects = result.localizedObjectAnnotations;
objects.forEach(object => {
  console.log(`Name: ${object.name}`);
  console.log(`Confidence: ${object.score}`);
  const vertices = object.boundingPoly.normalizedVertices;
  vertices.forEach(v => console.log(`x: ${v.x}, y:${v.y}`));
});

Python

试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Python 设置说明进行操作。如需了解详情,请参阅 Vision Python API 参考文档

def localize_objects(path):
    """Localize objects in the local image.

    Args:
    path: The path to the local file.
    """
    from google.cloud import vision
    client = vision.ImageAnnotatorClient()

    with open(path, 'rb') as image_file:
        content = image_file.read()
    image = vision.Image(content=content)

    objects = client.object_localization(
        image=image).localized_object_annotations

    print('Number of objects found: {}'.format(len(objects)))
    for object_ in objects:
        print('\n{} (confidence: {})'.format(object_.name, object_.score))
        print('Normalized bounding polygon vertices: ')
        for vertex in object_.bounding_poly.normalized_vertices:
            print(' - ({}, {})'.format(vertex.x, vertex.y))

其他语言

C#: 请按照客户端库页面上的 C# 设置说明操作,然后访问 .NET 版 Vision 参考文档。

PHP: 请按照客户端库页面上的 PHP 设置说明操作,然后访问 PHP 版 Vision 参考文档。

Ruby 版: 请按照客户端库页面上的 Ruby 设置说明操作,然后访问 Ruby 版 Vision 参考文档。

检测远程图片中的对象

为方便起见,Vision API 可以直接对位于 Google Cloud Storage 或网络中的图片文件执行特征检测,无需在请求正文中发送图片文件的内容。

REST 和命令行

在使用任何请求数据之前,请先进行以下替换:

  • cloud-storage-image-uri:Cloud Storage 存储分区中有效图片文件的路径。您必须至少拥有该文件的读取权限。 示例:
    • https://cloud.google.com/vision/docs/images/bicycle_example.png

HTTP 方法和网址:

POST https://vision.googleapis.com/v1/images:annotate

请求 JSON 正文:

{
  "requests": [
    {
      "image": {
        "source": {
          "imageUri": "cloud-storage-image-uri"
        }
      },
      "features": [
        {
          "maxResults": 10,
          "type": "OBJECT_LOCALIZATION"
        },
      ]
    }
  ]
}

如需发送请求,请选择以下方式之一:

curl

将请求正文保存在名为 request.json 的文件中,然后执行以下命令:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"

PowerShell

将请求正文保存在名为 request.json 的文件中,然后执行以下命令:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

如果请求成功,服务器将返回一个 200 OK HTTP 状态代码以及 JSON 格式的响应。

响应

Go

试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Go 设置说明进行操作。如需了解详情,请参阅 Vision Go API 参考文档


// localizeObjects gets objects and bounding boxes from the Vision API for an image at the given file path.
func localizeObjectsURI(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	image := vision.NewImageFromURI(file)
	annotations, err := client.LocalizeObjects(ctx, image, nil)
	if err != nil {
		return err
	}

	if len(annotations) == 0 {
		fmt.Fprintln(w, "No objects found.")
		return nil
	}

	fmt.Fprintln(w, "Objects:")
	for _, annotation := range annotations {
		fmt.Fprintln(w, annotation.Name)
		fmt.Fprintln(w, annotation.Score)

		for _, v := range annotation.BoundingPoly.NormalizedVertices {
			fmt.Fprintf(w, "(%f,%f)\n", v.X, v.Y)
		}
	}

	return nil
}

Java

在试用此示例之前,请按照Vision API 快速入门:使用客户端库中的 Java 设置说明进行操作。如需了解详情,请参阅 Vision API Java API 参考文档

/**
 * Detects localized objects in a remote image on Google Cloud Storage.
 *
 * @param gcsPath The path to the remote file on Google Cloud Storage to detect localized objects
 *     on.
 * @throws Exception on errors while closing the client.
 * @throws IOException on Input/Output errors.
 */
public static void detectLocalizedObjectsGcs(String gcsPath) throws IOException {
  List<AnnotateImageRequest> requests = new ArrayList<>();

  ImageSource imgSource = ImageSource.newBuilder().setGcsImageUri(gcsPath).build();
  Image img = Image.newBuilder().setSource(imgSource).build();

  AnnotateImageRequest request =
      AnnotateImageRequest.newBuilder()
          .addFeatures(Feature.newBuilder().setType(Type.OBJECT_LOCALIZATION))
          .setImage(img)
          .build();
  requests.add(request);

  // Initialize client that will be used to send requests. This client only needs to be created
  // once, and can be reused for multiple requests. After completing all of your requests, call
  // the "close" method on the client to safely clean up any remaining background resources.
  try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
    // Perform the request
    BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
    List<AnnotateImageResponse> responses = response.getResponsesList();
    client.close();
    // Display the results
    for (AnnotateImageResponse res : responses) {
      for (LocalizedObjectAnnotation entity : res.getLocalizedObjectAnnotationsList()) {
        System.out.format("Object name: %s%n", entity.getName());
        System.out.format("Confidence: %s%n", entity.getScore());
        System.out.format("Normalized Vertices:%n");
        entity
            .getBoundingPoly()
            .getNormalizedVerticesList()
            .forEach(vertex -> System.out.format("- (%s, %s)%n", vertex.getX(), vertex.getY()));
      }
    }
  }
}

Node.js

试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Node.js 设置说明进行操作。如需了解详情,请参阅 Vision Node.js API 参考文档

// Imports the Google Cloud client libraries
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const gcsUri = `gs://bucket/bucketImage.png`;

const [result] = await client.objectLocalization(gcsUri);
const objects = result.localizedObjectAnnotations;
objects.forEach(object => {
  console.log(`Name: ${object.name}`);
  console.log(`Confidence: ${object.score}`);
  const veritices = object.boundingPoly.normalizedVertices;
  veritices.forEach(v => console.log(`x: ${v.x}, y:${v.y}`));
});

Python

试用此示例之前,请按照《Vision 快速入门:使用客户端库》中的 Python 设置说明进行操作。如需了解详情,请参阅 Vision Python API 参考文档

def localize_objects_uri(uri):
    """Localize objects in the image on Google Cloud Storage

    Args:
    uri: The path to the file in Google Cloud Storage (gs://...)
    """
    from google.cloud import vision
    client = vision.ImageAnnotatorClient()

    image = vision.Image()
    image.source.image_uri = uri

    objects = client.object_localization(
        image=image).localized_object_annotations

    print('Number of objects found: {}'.format(len(objects)))
    for object_ in objects:
        print('\n{} (confidence: {})'.format(object_.name, object_.score))
        print('Normalized bounding polygon vertices: ')
        for vertex in object_.bounding_poly.normalized_vertices:
            print(' - ({}, {})'.format(vertex.x, vertex.y))

gcloud

如需检测图片中的标签,请使用 gcloud ml vision detect-objects 命令,如以下示例所示:

gcloud ml vision detect-objects https://cloud.google.com/vision/docs/images/bicycle_example.png

其他语言

C#: 请按照客户端库页面上的 C# 设置说明操作,然后访问 .NET 版 Vision 参考文档。

PHP: 请按照客户端库页面上的 PHP 设置说明操作,然后访问 PHP 版 Vision 参考文档。

Ruby 版: 请按照客户端库页面上的 Ruby 设置说明操作,然后访问 Ruby 版 Vision 参考文档。

试用

接下来,请尝试执行对象检测和本地化您可以使用已指定的图片 (https://cloud.google.com/vision/docs/images/bicycle_example.png) 或指定您自己的图片。选择执行即可发送请求。

无边界框的图片
图片来源Unsplash 用户 Bogdan Dada

请求正文:

{
  "requests": [
    {
      "features": [
        {
          "maxResults": 10,
          "type": "OBJECT_LOCALIZATION"
        }
      ],
      "image": {
        "source": {
          "imageUri": "https://cloud.google.com/vision/docs/images/bicycle_example.png"
        }
      }
    }
  ]
}