This quickstart demonstrates how to create a product set which contains a group of products with reference images for those products. The quickstart shows users how to create a product set, products, and their reference images in a single step by batch import. After the product set has been indexed, you can query the product set using Vision Product Search.
This quickstart steps you through the process of:
- Using a CSV and bulk import to create a product set, products, and reference images.
- Making a request to the Vision API Product Search with an image stored in a Google Cloud Storage bucket.
Before you begin
If you haven't done so already, set up your project as explained below.
Set up your project
-
Sign in to your Google Account.
If you don't already have one, sign up for a new account.
-
In the GCP Console, go to the Manage resources page and select or create a project.
-
Make sure that billing is enabled for your Google Cloud Platform project.
- Enable the Cloud Vision Product Search API.
-
Set up authentication:
-
In the GCP Console, go to the Create service account key page.
Go to the Create Service Account Key page - From the Service account list, select New service account.
- In the Service account name field, enter a name.
From the Role list, select Project > Owner.
Note: The Role field authorizes your service account to access resources. You can view and change this field later by using the GCP Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.- Click Create. A JSON file that contains your key downloads to your computer.
-
-
Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
Set environment variables
To make it more convenient to run the curl samples in this topic, set the
following environment variables where:
- project-id is the ID of your Google Cloud Platform (GCP) project.
- region-name is the GCP location that will run your tutorial, for example,
us-east1. Valid location identifiers are:us-west1,us-east1,europe-west1, andasia-east1.
export PROJECT_ID=project-id export LOCATION_ID=region-name
Using a dataset
In this quickstart you use a dataset of ~100 apparel product category entries.
This publicly available dataset is located in a public Google Cloud Storage
bucket at gs://cloud-ai-vision-data/product-search-tutorial/product_catalog.csv.
The CSV format is as follows:
gs://cloud-ai-vision-data/product-search-tutorial/images/46a0cbcf70ba11e89399d20059124800.jpg,image0,product_set0,product_id0,apparel,,"style=women,category=shoe",
gs://cloud-ai-vision-data/product-search-tutorial/images/46a1aea370ba11e888d4d20059124800.jpg,image1,product_set0,product_id1,apparel,,"style=men,category=shoe",
gs://cloud-ai-vision-data/product-search-tutorial/images/46a1cdb370ba11e8b538d20059124800.jpg,image2,product_set0,product_id2,apparel,,"style=women,category=dress",
Use bulk import to create a product set, products, and reference images
Use the following curl command to create a new product set with products
and reference images:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
"https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets:import" -d "{
'inputConfig': {
'gcsSource': {
'csvFileUri': 'gs://cloud-ai-vision-data/product-search-tutorial/product_catalog.csv'
}
}
}"
A successful response contains a long-running operation object:
{
"name": "locations/location-id/operations/0a0aec86192599fa"
}
The response also contains a relative operation ID (for example, 0a0aec86192599fa)
that can be used to get the status of the operation.
Get import operation status
You can use the operation-id returned above to check the status of the bulk import operation:
curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \ -H "Content-Type: application/json" \ https://vision.googleapis.com/v1/locations/$LOCATION_ID/operations/operation-id
A successful response looks like:
{
"name": "locations/location-id/operations/0a0aec86192599fb",
"metadata": {
"@type": "type.googleapis.com/google.cloud.vision.v1.BatchOperationMetadata",
"state": "SUCCESSFUL",
"submitTime": "2018-11-30T03:11:04.808114024Z",
"endTime": "2018-11-30T03:11:38.624444324Z"
},
"done": true,
"response": {
"@type": "type.googleapis.com/google.cloud.vision.v1.ImportProductSetsResponse",
"referenceImages": [
{
"name": "projects/project-id/locations/location-id/products/product_id0/referenceImages/image0",
"uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/46a0cbcf70ba11e89399d20059124800.jpg"
},
{
"name": "projects/project-id/locations/location-id/products/product_id1/referenceImages/image1",
"uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/46a1aea370ba11e888d4d20059124800.jpg"
},
...
{
"name": "projects/project-id/locations/location-id/products/product_id93/referenceImages/image93",
"uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/4697319970ba11e8a7bfd20059124800.jpg"
},
{
"name": "projects/project-id/locations/location-id/products/product_id94/referenceImages/image94",
"uri": "gs://cloud-ai-vision-data/product-search-tutorial/images/4698596370ba11e8bf6ad20059124800.jpg"
}
],
"statuses": [
{},
{},
...
{},
{}
]
}
}
List product sets and check indexing
You can list all your product sets and use the indexTime field to verify that
indexing has completed successfully:
curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \ -H "Content-Type: application/json" \ https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets
A successful response lists all your product sets, including a product set id
(for example, product_set0) as well as the indexTime field indicating
when indexing completed:
{
"productSets": [
{
"name": "projects/project-id/locations/location-id/productSets/product_set0",
"displayName": " ",
"indexTime": "2018-11-30T18:33:40.093508652Z",
"indexError": {}
}
]
}
List products
You can use the product-set-id returned above to list all products in your product set:
curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \ -H "Content-Type: application/json" \ https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets/$PRODUCT_SET_ID/products?pageSize=15
A successful response lists product details. In this request you use
the optional query parameter
pageSize
to set the result list to 15 products. The nextPageToken
in the response also indicates there are more products to list. You can use
the token listed to retrieve further results.
{
"products": [
{
"name": "projects/project-id/locations/location-id/products/product_id0",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "shoe"
}
]
},
{
"name": "projects/project-id/locations/location-id/products/product_id1",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "men"
},
{
"key": "category",
"value": "shoe"
}
]
},
...
{
"name": "projects/project-id/locations/location-id/products/product_id21",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
}
],
"nextPageToken": "1LqhSgZfM_uWKOxvog"
}
Search for matching products with Vision API Product Search
You can search using a base64 encoded local image, or use an image URI for an publicly accessible online image or an image stored in a Google Cloud Storage bucket such as the image below.
gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpgSearch using a remote image
Use the following request to search using the image stored in a public Google Cloud Storage bucket. Swap the three unique values (my-project-id, my-location-id, and my-product-set-id) to correspond to your project:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/images:annotate -d "{
'requests': [
{
'image': {
'source': {
'gcsImageUri': 'gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg'
}
},
'features': [
{
'type': 'PRODUCT_SEARCH'
}
],
'imageContext': {
'productSearchParams': {
'productSet': 'projects/my-project-id/locations/my-location-id/productSets/my-product-set-id',
'productCategories': [
'apparel'
],
'filter': 'style=womens OR style=women'
}
}
}
]
}"
A successful request returns a list of matching products, indicated by their product ID. These results are further broken down by individual products identified by bounding boxes if there are multiple products in a single image.
See the Understanding search responses & multi-detection topic for an example of single-product detection and multi-detection of products in an image.
A field score is returned as well. This field indicates the
confidence at which the service feels the product matches the supplied image,
on a scale of 0 (no confidence) to 1 (full confidence).
The indexTime field shows which version of the index is being searched. Image
changes made after this time are not reflected in the results.
{
"responses": [
{
"productSearchResults": {
"indexTime": "2018-12-04T22:33:53.673600055Z",
"results": [
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id16",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 1,
"image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
},
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id29",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 0.34263745,
"image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
},
...
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id89",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 0.20485385,
"image": "projects/project-id/locations/location-id/products/product_id89/referenceImages/image89"
}
],
"productGroupedResults": [
{
"boundingPoly": {
"normalizedVertices": [
{
"x": 0.25610325,
"y": 0.1357359
},
{
"x": 0.77213204,
"y": 0.1357359
},
{
"x": 0.77213204,
"y": 0.9287346
},
{
"x": 0.25610325,
"y": 0.9287346
}
]
},
"results": [
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id16",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 1,
"image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
},
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id29",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 0.3345438,
"image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
},
...
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id76",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "shoe"
}
]
},
"score": 0.20218614,
"image": "projects/project-id/locations/location-id/products/product_id76/referenceImages/image76"
}
]
},
{
"boundingPoly": {
"normalizedVertices": [
{},
{
"x": 1
},
{
"x": 1,
"y": 1
},
{
"y": 1
}
]
},
"results": [
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id16",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 1,
"image": "projects/project-id/locations/location-id/products/product_id16/referenceImages/image16"
},
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id29",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 0.3507311,
"image": "projects/project-id/locations/location-id/products/product_id29/referenceImages/image29"
},
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id8",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 0.26709077,
"image": "projects/project-id/locations/location-id/products/product_id8/referenceImages/image8"
},
...
{
"product": {
"name": "projects/project-id/locations/location-id/products/product_id51",
"displayName": " ",
"productCategory": "apparel",
"productLabels": [
{
"key": "style",
"value": "women"
},
{
"key": "category",
"value": "dress"
}
]
},
"score": 0.21597815,
"image": "projects/project-id/locations/location-id/products/product_id51/referenceImages/image51"
}
]
}
]
}
}
]
}
Congratulations! You've made your first images.annotate request to the
Vision API Product Search service.
Clean up
To avoid unnecessary Google Cloud Platform charges, use the GCP Console to delete your project if you do not need it.
What's next
- Get started with the Vision API Product Search in your language of choice by using a Cloud Vision API Product Search Client Library.
- Work through the How-To Guides.
- Work through the Tutorial.
Need help? Visit our