Quickstart

This quickstart demonstrates how to create a product set which contains a group of products with reference images for those products. The quickstart shows users how to create a product set, products, and their reference images in a single step by batch import. After the product set has been indexed, you can query the product set using Vision Product Search.

This quickstart steps you through the process of:

  • Using a CSV and bulk import to create a product set, products, and reference images.
  • Making a request to the Vision API Product Search with an image stored in a Google Cloud Storage bucket.

Before you begin

If you haven't done so already, set up your project as explained below.

Set up your project

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the GCP Console, go to the Manage resources page and select or create a project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your Google Cloud Platform project.

    Learn how to enable billing

  4. Enable the Cloud Vision Product Search API.

    Enable the API

  5. Set up authentication:
    1. In the GCP Console, go to the Create service account key page.

      Go to the Create Service Account Key page
    2. From the Service account list, select New service account.
    3. In the Service account name field, enter a name.
    4. From the Role list, select Project > Owner.

      Note: The Role field authorizes your service account to access resources. You can view and change this field later by using the GCP Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
    5. Click Create. A JSON file that contains your key downloads to your computer.
  6. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

Set environment variables

To make it more convenient to run the curl samples in this topic, set the following environment variables where:

  • project-id is the ID of your Google Cloud Platform (GCP) project.
  • region-name is the GCP location that will run your tutorial, for example, us-east1. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
export PROJECT_ID=project-id
export LOCATION_ID=region-name

Using a dataset

In this quickstart you use a dataset of ~100 apparel product category entries. This publicly available dataset is located in a public Google Cloud Storage bucket at gs://cloud-ai-vision-data/product-search-tutorial/product_catalog.csv.

The CSV format is as follows:

gs://cloud-ai-vision-data/product-search-tutorial/images/46a0cbcf70ba11e89399d20059124800.jpg,image0,product_set0,product_id0,apparel,,"style=women,category=shoe",
gs://cloud-ai-vision-data/product-search-tutorial/images/46a1aea370ba11e888d4d20059124800.jpg,image1,product_set0,product_id1,apparel,,"style=men,category=shoe",
gs://cloud-ai-vision-data/product-search-tutorial/images/46a1cdb370ba11e8b538d20059124800.jpg,image2,product_set0,product_id2,apparel,,"style=women,category=dress",

Use bulk import to create a product set, products, and reference images

Use the following curl command to create a new product set with products and reference images:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
"https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets:import" -d "{
  'inputConfig': {
    'gcsSource': {
      'csvFileUri': 'gs://cloud-ai-vision-data/product-search-tutorial/product_catalog.csv'
    }
  }
}"

A successful response contains a long-running operation object:

{
  "name": "locations/location-id/operations/0a0aec86192599fa"
}

The response also contains a relative operation ID (for example, 0a0aec86192599fa) that can be used to get the status of the operation.

Get import operation status

You can use the operation-id returned above to check the status of the bulk import operation:

curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/locations/$LOCATION_ID/operations/operation-id

A successful response looks like:

{
  "name": "locations/location-id/operations/0a0aec86192599fb",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.BatchOperationMetadata",
    "state": "SUCCESSFUL",
    "submitTime": "2018-11-30T03:11:04.808114024Z",
    "endTime": "2018-11-30T03:11:38.624444324Z"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.ImportProductSetsResponse",
    "referenceImages": [
      {
        "name": "projects/project-id/locations/location-id/products/product_id0/referenceImages/image0",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/46a0cbcf70ba11e89399d20059124800.jpg"
      },
      {
        "name": "projects/project-id/locations/location-id/products/product_id1/referenceImages/image1",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/46a1aea370ba11e888d4d20059124800.jpg"
      },
      ...
      {
        "name": "projects/project-id/locations/location-id/products/product_id93/referenceImages/image93",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/4697319970ba11e8a7bfd20059124800.jpg"
      },
      {
        "name": "projects/project-id/locations/location-id/products/product_id94/referenceImages/image94",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/4698596370ba11e8bf6ad20059124800.jpg"
      }
    ],
    "statuses": [
      {},
      {},
       ...
      {},
      {}
    ]
  }
}

List product sets and check indexing

You can list all your product sets and use the indexTime field to verify that indexing has completed successfully:

curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets

A successful response lists all your product sets, including a product set id (for example, product_set0) as well as the indexTime field indicating when indexing completed:

{
  "productSets": [
    {
      "name": "projects/project-id/locations/location-id/productSets/product_set0",
      "displayName": " ",
      "indexTime": "2018-11-30T18:33:40.093508652Z",
      "indexError": {}
    }
  ]
}

List products

You can use the product-set-id returned above to list all products in your product set:

curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets/$PRODUCT_SET_ID/products?pageSize=15

A successful response lists product details. In this request you use the optional query parameter pageSize to set the result list to 15 products. The nextPageToken in the response also indicates there are more products to list. You can use the token listed to retrieve further results.

{
  "products": [
    {
      "name": "projects/project-id/locations/location-id/products/product_id0",
      "displayName": " ",
      "productCategory": "apparel",
      "productLabels": [
        {
          "key": "style",
          "value": "women"
        },
        {
          "key": "category",
          "value": "shoe"
        }
      ]
    },
    {
      "name": "projects/project-id/locations/location-id/products/product_id1",
      "displayName": " ",
      "productCategory": "apparel",
      "productLabels": [
        {
          "key": "style",
          "value": "men"
        },
        {
          "key": "category",
          "value": "shoe"
        }
      ]
    },
    ...
    {
      "name": "projects/project-id/locations/location-id/products/product_id21",
      "displayName": " ",
      "productCategory": "apparel",
      "productLabels": [
        {
          "key": "style",
          "value": "women"
        },
        {
          "key": "category",
          "value": "dress"
        }
      ]
    }
  ],
  "nextPageToken": "1LqhSgZfM_uWKOxvog"
}

Search for matching products with Vision API Product Search

You can search using a base64 encoded local image, or use an image URI for an publicly accessible online image or an image stored in a Google Cloud Storage bucket such as the image below.

dress image in cloud storage bucket
gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg

Search using a remote image

Use the following request to search using the image stored in a public Google Cloud Storage bucket. Swap the three unique values (my-project-id, my-location-id, and my-product-set-id) to correspond to your project:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/images:annotate -d "{
  'requests': [
    {
      'image': {
        'source': {
          'gcsImageUri': 'gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg'
        }
      },
      'features': [
        {
          'type': 'PRODUCT_SEARCH'
        }
      ],
      'imageContext': {
        'productSearchParams': {
          'productSet': 'projects/my-project-id/locations/my-location-id/productSets/my-product-set-id',
          'productCategories': [
               'apparel'
             ],
        'filter': 'style=womens OR style=women'
        }
      }
    }
  ]
}"

A successful request returns a list of matching products, indicated by their product ID. These results are further broken down by individual products identified by bounding boxes if there are multiple products in a single image.

See the Understanding search responses & multi-detection topic for an example of single-product detection and multi-detection of products in an image.

A field score is returned as well. This field indicates the confidence at which the service feels the product matches the supplied image, on a scale of 0 (no confidence) to 1 (full confidence).

The indexTime field shows which version of the index is being searched. Image changes made after this time are not reflected in the results.

{
  "responses": [
    {
      "productSearchResults": {
        "indexTime": "2018-12-04T22:33:53.673600055Z",
        "results": [
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id16",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 1,
            "image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
          },
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id29",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.34263745,
            "image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
          },
          ...
          {
            "product": {
              "name": "projects/project-id/locations/location-id/products/product_id89",
              "displayName": " ",
              "productCategory": "apparel",
              "productLabels": [
                {
                  "key": "style",
                  "value": "women"
                },
                {
                  "key": "category",
                  "value": "dress"
                }
              ]
            },
            "score": 0.20485385,
            "image": "projects/project-id/locations/location-id/products/product_id89/referenceImages/image89"
          }
        ],
        "productGroupedResults": [
          {
            "boundingPoly": {
              "normalizedVertices": [
                {
                  "x": 0.25610325,
                  "y": 0.1357359
                },
                {
                  "x": 0.77213204,
                  "y": 0.1357359
                },
                {
                  "x": 0.77213204,
                  "y": 0.9287346
                },
                {
                  "x": 0.25610325,
                  "y": 0.9287346
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id16",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 1,
                "image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
              },
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id29",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.3345438,
                "image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
              },
              ...
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id76",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "shoe"
                    }
                  ]
                },
                "score": 0.20218614,
                "image": "projects/project-id/locations/location-id/products/product_id76/referenceImages/image76"
              }
            ]
          },
          {
            "boundingPoly": {
              "normalizedVertices": [
                {},
                {
                  "x": 1
                },
                {
                  "x": 1,
                  "y": 1
                },
                {
                  "y": 1
                }
              ]
            },
            "results": [
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id16",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 1,
                "image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
              },
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id29",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.3507311,
                "image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
              },
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id8",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.26709077,
                "image": "projects/project-id/locations/location-id/products/product_id8/referenceImages/image8"
              },
              ...
              {
                "product": {
                  "name": "projects/project-id/locations/location-id/products/product_id51",
                  "displayName": " ",
                  "productCategory": "apparel",
                  "productLabels": [
                    {
                      "key": "style",
                      "value": "women"
                    },
                    {
                      "key": "category",
                      "value": "dress"
                    }
                  ]
                },
                "score": 0.21597815,
                "image": "projects/project-id/locations/location-id/products/product_id51/referenceImages/image51"
              }
            ]
          }
        ]
      }
    }
  ]
}

Congratulations! You've made your first images.annotate request to the Vision API Product Search service.

Clean up

To avoid unnecessary Google Cloud Platform charges, use the GCP Console to delete your project if you do not need it.

What's next

Kunde den här sidan hjälpa dig? Berätta:

Skicka feedback om ...

Cloud Vision API Product Search
Behöver du hjälp? Besök vår supportsida.