Quickstart

This quickstart demonstrates how to create and use the three types of Vision API Product Search resources: a product set which contains a group of products, and reference images associated with those products.

In this quickstart you will create a product set, products, and their reference images in a single step by batch import.

After the product set has been indexed, you can query the product set using Vision API Product Search.

This quickstart steps you through the process of:

  • Using a CSV and bulk import to create a product set, products, and reference images.
  • Making a request to the Vision API Product Search with an image stored in a Cloud Storage bucket.

Before you begin

If you haven't done so already, set up your project as explained below.

Set up your project

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Enable the Vision API.

    Enable the API

  5. Set up authentication:

    1. In the Cloud Console, go to the Create service account key page.

      Go to the Create Service Account Key page
    2. From the Service account list, select New service account.
    3. In the Service account name field, enter a name.

    4. From the Role list, select Project > Owner.

    5. Click Create. A JSON file that contains your key downloads to your computer.

  6. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

Set environment variables

To make it more convenient to run the curl samples in this topic, set the following environment variables where:

  • project-id is the ID of your Google Cloud Platform (GCP) project.
  • region-name is the GCP location that will run your tutorial, for example, us-east1. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1 .
export PROJECT_ID=project-id
export LOCATION_ID=region-name

Using a dataset

In this quickstart you use a dataset of ~100 apparel-v2 product category entries. This publicly available dataset is located in a public Cloud Storage bucket at:

The CSV format is as follows:

gs://cloud-ai-vision-data/product-search-tutorial/images/filename1.jpg,image0,product_set0,product_id0,apparel-v2,,"style=women,category=shoe",
gs://cloud-ai-vision-data/product-search-tutorial/images/filename2.jpg,image1,product_set0,product_id1,apparel-v2,,"style=men,category=shoe",
gs://cloud-ai-vision-data/product-search-tutorial/images/filename3.jpg,image2,product_set0,product_id2,apparel-v2,,"style=women,category=dress",

Use bulk import to create a product set, products, and reference images

Use the following curl command to create a new product set with products and reference images. This set is named product_set0, a value declared in the import CSV.

First create a request JSON file called import_request.json and save it in you current working directory:

import_request.json

{
  "inputConfig": {
    "gcsSource": {
      "csvFileUri": "gs://cloud-samples-data/vision/product_search/product_catalog.csv"
    }
  }
}

After creating the request JSON file, send the request:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @import_request.json \
https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets:import

A successful response contains a long-running operation object:

{
  "name": "locations/location-id/operations/0a0aec86192599fa"
}

The response also contains a relative operation ID (for example, 0a0aec86192599fa) that can be used to get the status of the operation.

Get import operation status

You can use the operation-id returned from the import operation to check the status of the bulk import operation:

curl -X GET \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/locations/$LOCATION_ID/operations/operation-id

A successful response looks like:

{
  "name": "locations/location-id/operations/0a0aec86192599fb",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.BatchOperationMetadata",
    "state": "SUCCESSFUL",
    "submitTime": "2018-11-30T03:11:04.808114024Z",
    "endTime": "2018-11-30T03:11:38.624444324Z"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.ImportProductSetsResponse",
    "referenceImages": [
      {
        "name": "projects/project-id/locations/location-id/products/product_id0/referenceImages/image0",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/46a0cbcf70ba11e89399d20059124800.jpg"
      },
      {
        "name": "projects/project-id/locations/location-id/products/product_id1/referenceImages/image1",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/46a1aea370ba11e888d4d20059124800.jpg"
      },
      ...
      {
        "name": "projects/project-id/locations/location-id/products/product_id93/referenceImages/image93",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/4697319970ba11e8a7bfd20059124800.jpg"
      },
      {
        "name": "projects/project-id/locations/location-id/products/product_id94/referenceImages/image94",
        "uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/4698596370ba11e8bf6ad20059124800.jpg"
      }
    ],
    "statuses": [
      {},
      {},
       ...
      {},
      {}
    ]
  }
}

Indexing

The Product Search index of products is updated approximately every 30 minutes. When images are added or deleted, the change won't be reflected in your Product Search responses until the index is next updated.

To make sure that indexing has completed successfully, check the indexTime field of a product set.

List product sets and check indexing

You can list all your product sets and use the indexTime field to verify that indexing has completed successfully:

curl -X GET \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets

A successful response lists all your product sets, including a product set id (for example, product_set0) as well as the indexTime field indicating when indexing completed:

{
  "productSets": [
    {
      "name": "projects/project-id/locations/location-id/productSets/product_set0",
      "displayName": " ",
      "indexTime": "2019-11-30T18:33:40.093508652Z",
      "indexError": {}
    }
  ]
}

List products

You can use the product-set-id returned from the list of product sets to list all products in your product set:

curl -X GET \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets/product-set-id/products?pageSize=15

A successful response lists product details.

In this request you use the optional query parameter pageSize to set the result list to 15 products. The nextPageToken in the response also indicates there are more products to list. You can use the token listed to retrieve further results. See Getting and listing resources for more information on using a pageToken.

{
  "products": [
    {
      "name": "projects/project-id/locations/location-id/products/product_id0",
      "displayName": " ",
      "productCategory": "apparel",
      "productLabels": [
        {
          "key": "style",
          "value": "women"
        },
        {
          "key": "category",
          "value": "shoe"
        }
      ]
    },
    {
      "name": "projects/project-id/locations/location-id/products/product_id1",
      "displayName": " ",
      "productCategory": "apparel",
      "productLabels": [
        {
          "key": "style",
          "value": "men"
        },
        {
          "key": "category",
          "value": "shoe"
        }
      ]
    },
    ...
    {
      "name": "projects/project-id/locations/location-id/products/product_id21",
      "displayName": " ",
      "productCategory": "apparel",
      "productLabels": [
        {
          "key": "style",
          "value": "women"
        },
        {
          "key": "category",
          "value": "dress"
        }
      ]
    }
  ],
  "nextPageToken": "1LqhSgZfM_uWKOxvog"
}

Search for matching products with Vision API Product Search

After indexing is complete, you can search for products that match a sample image. In this quickstart you use an image stored in a Google Cloud Storage bucket such as the image below.

dress image in cloud storage bucket
gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg

Search using a remote image

Use the following request to search using the image stored in a public Cloud Storage bucket.

First create a request JSON file called search_request.json and save it in you current working directory. Change the following values in the request JSON to match your project's information:

  • my-project-id
  • my-location-id
  • my-product-set-id

search_request.json

{
  "requests": [
    {
      "image": {
        "source": {
          "gcsImageUri": "gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg"
        }
      },
      "features": [
        {
          "type": "PRODUCT_SEARCH"
        }
      ],
      "imageContext": {
        "productSearchParams": {
          "productSet": "projects/project-id/locations/location-id/productSets/product-set-id",
          "productCategories": [
               "apparel-v2"
             ],
        "filter": "style=womens OR style=women"
        }
      }
    }
  ]
}

After creating the request JSON file, send the request:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @search_request.json \
https://vision.googleapis.com/v1/images:annotate

A successful request returns a list of matching products, indicated by their product ID. These results are further broken down by individual products identified by bounding boxes if there are multiple products in a single image.

See the Understanding search responses & multi-detection topic for an example of single-product detection and multi-detection of products in an image.

A field score is returned as well. This field indicates the confidence at which the service feels the product matches the supplied image, on a scale of 0 (no confidence) to 1 (full confidence).

The indexTime field shows which version of the index is being searched. Image changes made after this time are not reflected in the results.

Congratulations! You've made your first images.annotate request to the Vision API Product Search service.

Clean up

To avoid unnecessary Google Cloud Platform charges, use the Cloud Console to delete your project if you do not need it.

What's next