了解搜索响应和多对象检测

Vision API Product Search 可以在一张搜索图片中识别出单件商品或多件商品，并列出与之匹配的商品。

单件商品图片和响应

此搜索图片包含单件商品。

gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg

使用 Vision API Product Search 搜索相似商品时将返回一条响应，该响应识别图片中的两个边界框，一个是商品的边界框，另一个是整张图片的边界框。

展开下面的部分可以查看完整响应。

完整响应

Note: Zero coordinate values omitted. When the API detects a coordinate ("x" or "y") value of 0, that coordinate is omitted in the JSON response. Thus, a response with a bounding poly around the entire image would be
[{},{"x": 1},{"x": 1,"y": 1},{"y": 1}]. For more information, see the API Reference documentation.

{
  "responses": [
    {
      "productSearchResults": {
        "indexTime": "2018-12-04T22:33:53.673600055Z",
        "results": [
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id16",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 1,
            "image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
          },
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id29",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.34263745,
            "image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
          },
          ...
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id89",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.20485385,
            "image": "projects/project-id/locations/location-id/products/product_id89/referenceImages/image89"
          }
        ],
        "productGroupedResults": [
          {
            "boundingPoly": {
              "normalizedVertices": [
                {
                  "x": 0.25610325,
                  "y": 0.1357359
                },
                {
                  "x": 0.77213204,
                  "y": 0.1357359
                },
                {
                  "x": 0.77213204,
                  "y": 0.9287346
                },
                {
                  "x": 0.25610325,
                  "y": 0.9287346
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id16",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 1,
                "image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
              },
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id29",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.3345438,
                "image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
              },
              ...
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id76",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "shoe"
                    }
                  ]
                },
                "score": 0.20218614,
                "image": "projects/project-id/locations/location-id/products/product_id76/referenceImages/image76"
              }
            ]
          },
          {
            "boundingPoly": {
              "normalizedVertices": [
                {},
                {
                  "x": 1
                },
                {
                  "x": 1,
                  "y": 1
                },
                {
                  "y": 1
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id16",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 1,
                "image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
              },
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id29",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.3507311,
                "image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
              },
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id8",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.26709077,
                "image": "projects/project-id/locations/location-id/products/product_id8/referenceImages/image8"
              },
              ...
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id51",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.21597815,
                "image": "projects/project-id/locations/location-id/products/product_id51/referenceImages/image51"
              }
            ]
          }
        ]
      }
    }
  ]
}

边界框和匹配商品

图片中的以下两件商品（边界框）由请求返回的两组顶点确定。

系统会针对 Vision API Product Search 识别出的每个边界框列出匹配商品。

gs://cloud-samples-data/product_search/multiple_dresses.jpg

在 API 响应中，边界框和匹配商品结果在 productGroupedResults 字段中列出。这些结果按图片中检测到的商品进行分组。每个 boundingPoly 条目会列出由归一化顶点（数值介于 0 到 1 之间，不对应于图片的实际像素值）指定的区域。然后，系统会列出该特定区域所对应的匹配商品。

响应中的以下部分用于描述服装周围的边界框。

单件商品边界框响应部分

....
        "productGroupedResults": [
          {
            "boundingPoly": {
              "normalizedVertices": [
                {
                  "x": 0.25610325,
                  "y": 0.1357359
                },
                {
                  "x": 0.77213204,
                  "y": 0.1357359
                },
                {
                  "x": 0.77213204,
                  "y": 0.9287346
                },
                {
                  "x": 0.25610325,
                  "y": 0.9287346
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id16",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels":...
                "score": 1,
                ...

匹配商品会在 results 字段后面列出，并按匹配得分以降序排列。匹配得分介于 0（完全不匹配）到 1（完全匹配）之间。

Several matching products for the dress

响应中返回的另一个 boundingPoly 是指整张图片的边界框。 其归一化顶点如下所示：

• [{},{"x": 1},{"x": 1,"y": 1},{"y": 1}]

当坐标值为 0 时，会被省略。因此，上述边界框对应于：

• [{"x": 0,"y": 0},{"x": 1, "y": 0},{"x": 1,"y": 1},{"y": 1}]

然后，系统会根据整张图片的边界框，以同样的方式列出匹配商品。

多件商品图片（多件商品检测）和响应

以下搜索图片包含多个商品。

gs://cloud-samples-data/product_search/multiple_dresses.jpg

使用 Vision API Product Search 搜索相似商品时，系统会返回一条响应，该响应会识别出图片中的多个边界框，包括整张图片的边界框，以及三条连衣裙各自的边界框。

展开下面的部分可以查看完整响应。

完整响应

Note: Zero coordinate values omitted. When the API detects a coordinate ("x" or "y") value of 0, that coordinate is omitted in the JSON response. Thus, a response with a bounding poly around the entire image would be
[{},{"x": 1},{"x": 1,"y": 1},{"y": 1}]. For more information, see the API Reference documentation.

{
  "responses": [
    {
      "productSearchResults": {
        "indexTime": "2018-12-05T18:34:22.231193168Z",
        "results": [
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id89",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.40636745,
            "image": "projects/project-id/locations/location-id/products/product_id89/referenceImages/image89"
          },
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id56",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.35433072,
            "image": "projects/project-id/locations/location-id/products/product_id56/referenceImages/image56"
          },
          ...
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id92",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.30736163,
            "image": "projects/project-id/locations/location-id/products/product_id92/referenceImages/image92"
          }
        ],
        "productGroupedResults": [
          {
            "boundingPoly": {
              "normalizedVertices": [
                {
                  "x": 0.38727918,
                  "y": 0.10391616
                },
                {
                  "x": 0.57177967,
                  "y": 0.10391616
                },
                {
                  "x": 0.57177967,
                  "y": 0.94973093
                },
                {
                  "x": 0.38727918,
                  "y": 0.94973093
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id39",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.36689043,
                "image": "projects/project-id/locations/location-id/products/product_id39/referenceImages/image39"
              },
              {
                "boundingPoly": {
                  "normalizedVertices": [
                    {
                      "x": 0.63010275,
                      "y": 0.08650439
                    },
                    {
                      "x": 0.88989735,
                      "y": 0.08650439
                    },
                    {
                      "x": 0.88989735,
                      "y": 0.9285545
                    },
                    {
                      "x": 0.63010275,
                      "y": 0.9285545
                    }
                  ]
                },
                "results": [
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id89",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.4007347,
                    "image": "projects/project-id/locations/location-id/products/product_id89/referenceImages/image89"
                  },
                  ...
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id30",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.27045864,
                    "image": "projects/project-id/locations/location-id/products/product_id30/referenceImages/image30"
                  },
                  ...
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id63",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.23139484,
                    "image": "projects/project-id/locations/location-id/products/product_id63/referenceImages/image63"
                  }
                ]
              },
              ...
              {
                "boundingPoly": {
                  "normalizedVertices": [
                    {},
                    {
                      "x": 1
                    },
                    {
                      "x": 1,
                      "y": 1
                    },
                    {
                      "y": 1
                    }
                  ]
                },
                "results": [
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id30",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.39522344,
                    "image": "projects/project-id/locations/location-id/products/product_id30/referenceImages/image30"
                  },
                  ...
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id4",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.34226564,
                    "image": "projects/project-id/locations/location-id/products/product_id4/referenceImages/image4"
                  }
                ]
              },
              {
                "boundingPoly": {
                  "normalizedVertices": [
                    {
                      "x": 0.030102689,
                      "y": 0.067680866
                    },
                    {
                      "x": 0.28942674,
                      "y": 0.067680866
                    },
                    {
                      "x": 0.28942674,
                      "y": 0.94973093
                    },
                    {
                      "x": 0.030102689,
                      "y": 0.94973093
                    }
                  ]
                },
                "results": [
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id89",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.42439392,
                    "image": "projects/project-id/locations/location-id/products/product_id89/referenceImages/image89"
                  },
                  ...
                  {
                    "product": {
                      "name": "projects/project-id/locations/location-id/products/product_id51",
                      "displayName": " ",
                      "productCategory": "apparel",
                      "productLabels": [
                        {
                          "key": "style",
                          "value": "women"
                        },
                        {
                          "key": "category",
                          "value": "dress"
                        }
                      ]
                    },
                    "score": 0.26797917,
                    "image": "projects/project-id/locations/location-id/products/product_id51/referenceImages/image51"
                  }
                ]
              }
            ]
          }
        }
      ]
}

边界框和匹配商品

下图显示了 Vision API Product Search 在这张包含多件商品的图片中识别出的所有边界框。

gs://cloud-samples-data/product_search/multiple_dresses.jpg

与单商品图片类似，在多商品图片的响应中，也会在 productGroupedResults 字段中列出边界框和匹配商品结果。系统通过边界框识别每件商品，并且按照检测到的每件商品对匹配结果进行分组。

响应中的以下部分对应于中间的连衣裙周围的边界框。

产品边界框响应部分（中间的连衣裙）

"productGroupedResults": [
          {
            "boundingPoly": {
              "normalizedVertices": [
                {
                  "x": 0.38727918,
                  "y": 0.10391616
                },
                {
                  "x": 0.57177967,
                  "y": 0.10391616
                },
                {
                  "x": 0.57177967,
                  "y": 0.94973093
                },
                {
                  "x": 0.38727918,
                  "y": 0.94973093
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name":...
                "score": 0.36689043,
                ...

下图显示了 Vision API Product Search 返回的中间的连衣裙的部分匹配结果。

Several matching products for the middle dress

系统还在图片中识别出以下其他边界框，并为每个边界框列出了各自的匹配商品列表（此处未列出）：

• 左侧连衣裙：{"x":0.03, "y":0.07},{"x":0.29, "y":0.07},{"x":0.29, "y":0.95},{"x":0.03, "y":0.95}
• 右侧连衣裙：{"x":0.63, "y":0.09},{"x":0.89, "y":0.09},{"x":0.89, "y":0.93},{"x":0.63, "y":0.93}
• 整张图片：
  • {},{"x": 1},{"x": 1,"y": 1},{"y": 1}；也可以表示为：
  • {"x": 0, "y": 0},{"x": 1,"y:0"},{"x": 1,"y": 1},{"x": 0,"y": 1}