Product Search Tutorial

This tutorial demonstrates how to create a product set which contains a group of products with reference images for those products. The tutorial shows users how to create a product set via online (individual) import. After the product set has been indexed, you can query the product set using Vision Product Search.

In this tutorial, you will learn how to:

  1. Create a product set through online (individual) import
  2. Create an individual product
  3. Add a product to a product set
  4. Update a product
  5. Create a reference image
  6. Search for similar products

Before you begin

Before you begin this tutorial make sure you have installed the appropriate client libraries, enabled billing and the API for your project, and have properly set up authentication.

Import Libraries

To use the Cloud Vision Product Search, import the following modules after downloading and installing the client library:

Java

import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.vision.v1.BatchOperationMetadata;
import com.google.cloud.vision.v1.ImportProductSetsGcsSource;
import com.google.cloud.vision.v1.ImportProductSetsGcsSource.Builder;
import com.google.cloud.vision.v1.ImportProductSetsInputConfig;
import com.google.cloud.vision.v1.ImportProductSetsResponse;
import com.google.cloud.vision.v1.ProductSearchClient;
import com.google.cloud.vision.v1.ReferenceImage;

Node.js

const vision = require('@google-cloud/vision');

PHP

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\ImportProductSetsGcsSource;
use Google\Cloud\Vision\V1\ImportProductSetsInputConfig;

Python

from google.cloud import vision

Running the Application

Step 1: Create a Product Catalog

Users have two options for creating a product catalog, either via batch import using a CSV file, which allows an entire product catalog to be imported in a single API call, or via online import, which offers you control over your product sets and allows for management of one resource or relationship at a time. This primarily means individual creation of product sets, products, and reference images. Online import also allows you to incrementally update a product catalog you have already created via batch import.

In this tutorial you will use online import. See the Quickstart for an example of batch import with a CSV.

Online (Individual) Import

1. Create a Product Set

Create an empty Product Set which is a simple container for a group of products.

Request

Create an empty Product Set and name it "PS_CLOTH-SHOE_070318" by executing the following request using the method create_product_set(). Pass the product set id and display name as arguments.

Command-line

To create a product set, send a POST to the https://vision.googleapis.com/v1/{parent=projects/*/locations/*}/productSets endpoint in the following format.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier.Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
  • Replace display-name with the display name for your new product set.
curl -X POST \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    -H "Content-Type: application/json" \ 
    "https://vision.googleapis.com/v1/projects/project-id/locations/location-id/productSets" -d '{
       'displayName' : 'display-name'
    }

Go

import (
	"context"
	"fmt"
	"io"

	vision "cloud.google.com/go/vision/apiv1"
	visionpb "google.golang.org/genproto/googleapis/cloud/vision/v1"
)

// createProductSet creates a product set.
func createProductSet(w io.Writer, projectID string, location string, productSetID string, productSetDisplayName string) error {
	ctx := context.Background()
	c, err := vision.NewProductSearchClient(ctx)
	if err != nil {
		return fmt.Errorf("NewProductSearchClient: %v", err)
	}

	req := &visionpb.CreateProductSetRequest{
		Parent:       fmt.Sprintf("projects/%s/locations/%s", projectID, location),
		ProductSetId: productSetID,
		ProductSet: &visionpb.ProductSet{
			DisplayName: productSetDisplayName,
		},
	}

	resp, err := c.CreateProductSet(ctx, req)
	if err != nil {
		return fmt.Errorf("CreateProductSet: %v", err)
	}

	fmt.Fprintf(w, "Product set name: %s\n", resp.Name)

	return nil
}

Java

/**
 * Create a product set
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productSetId - Id of the product set.
 * @param productSetDisplayName - Display name of the product set.
 * @throws IOException - on I/O errors.
 */
public static void createProductSet(
    String projectId, String computeRegion, String productSetId, String productSetDisplayName)
    throws IOException {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // A resource that represents Google Cloud Platform location.
    String formattedParent = ProductSearchClient.formatLocationName(projectId, computeRegion);

    // Create a product set with the product set specification in the region.
    ProductSet myProductSet =
        ProductSet.newBuilder().setDisplayName(productSetDisplayName).build();
    CreateProductSetRequest request =
        CreateProductSetRequest.newBuilder()
            .setParent(formattedParent)
            .setProductSet(myProductSet)
            .setProductSetId(productSetId)
            .build();
    ProductSet productSet = client.createProductSet(request);
    // Display the product set information
    System.out.println(String.format("Product set name: %s", productSet.getName()));
  }
}

Node.js

// Imports the Google Cloud client library
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productSetId = 'Id of the product set';
// const productSetDisplayName = 'Display name of the product set';

// Resource path that represents Google Cloud Platform location.
const locationPath = client.locationPath(projectId, location);

const productSet = {
  displayName: productSetDisplayName,
};

const request = {
  parent: locationPath,
  productSet: productSet,
  productSetId: productSetId,
};

const [createdProductSet] = await client.createProductSet(request);
console.log(`Product Set name: ${createdProductSet.name}`);

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\ProductSet;

/**
 * Create a product set
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productSetId ID of the product set
 * @param string $productSetDisplayName Display name of the product set
 */
function product_set_create($projectId, $location, $productSetId, $productSetDisplayName)
{
    $client = new ProductSearchClient();

    # a resource that represents Google Cloud Platform location.
    $locationPath = $client->locationName($projectId, $location);

    # create a product set with the product set specification in the region.
    $productSet = (new ProductSet())
        ->setDisplayName($productSetDisplayName);

    # the response is the product set with the `name` field populated.
    $response = $client->createProductSet($locationPath, $productSet, ['productSetId' => $productSetId]);

    # display the product information.
    printf('Product set name: %s' . PHP_EOL, $response->getName());

    $client->close();
}

Python

from google.cloud import vision

def create_product_set(
        project_id, location, product_set_id, product_set_display_name):
    """Create a product set.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_set_id: Id of the product set.
        product_set_display_name: Display name of the product set.
    """
    client = vision.ProductSearchClient()

    # A resource that represents Google Cloud Platform location.
    location_path = client.location_path(
        project=project_id, location=location)

    # Create a product set with the product set specification in the region.
    product_set = vision.types.ProductSet(
            display_name=product_set_display_name)

    # The response is the product set with `name` populated.
    response = client.create_product_set(
        parent=location_path,
        product_set=product_set,
        product_set_id=product_set_id)

    # Display the product set information.
    print('Product set name: {}'.format(response.name))

Response

Product set name: projects/prj-prod-search-tutorials/locations/us-east1/productSets/PS_CLOTH-SHOE_070318
Product set id: PS_CLOTH-SHOE_070318
Product set display name: CLOTH-SHOE

2. Create a Product

After a product set has been created, the next step is to create a product. Create a product by executing the following request.

Command-line

To get the information for a product set, send a request to the following URI.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/project-id/locations/location-id/products -d '{
  'displayName': 'sample-product-1234',
  'description': 'Athletic shorts',
  'productCategory': 'apparel',
  'productLabels': [
      { 'key': 'style', 'value': 'womens' },
      { 'key': 'color', 'value': 'blue' }
  ]
}'

Go

import (
	"context"
	"fmt"
	"io"

	vision "cloud.google.com/go/vision/apiv1"
	visionpb "google.golang.org/genproto/googleapis/cloud/vision/v1"
)

// createProduct creates a product.
func createProduct(w io.Writer, projectID string, location string, productID string, productDisplayName string, productCategory string) error {
	ctx := context.Background()
	c, err := vision.NewProductSearchClient(ctx)
	if err != nil {
		return fmt.Errorf("NewProductSearchClient: %v", err)
	}

	req := &visionpb.CreateProductRequest{
		Parent:    fmt.Sprintf("projects/%s/locations/%s", projectID, location),
		ProductId: productID,
		Product: &visionpb.Product{
			DisplayName:     productDisplayName,
			ProductCategory: productCategory,
		},
	}

	resp, err := c.CreateProduct(ctx, req)
	if err != nil {
		return fmt.Errorf("CreateProduct: %v", err)
	}

	fmt.Fprintf(w, "Product name: %s\n", resp.Name)

	return nil
}

Java

/**
 * Create one product.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productId - Id of the product.
 * @param productDisplayName - Display name of the product.
 * @param productCategory - Category of the product.
 * @throws IOException - on I/O errors.
 */
public static void createProduct(
    String projectId,
    String computeRegion,
    String productId,
    String productDisplayName,
    String productCategory)
    throws IOException {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // A resource that represents Google Cloud Platform location.
    String formattedParent = ProductSearchClient.formatLocationName(projectId, computeRegion);
    // Create a product with the product specification in the region.
    // Multiple labels are also supported.
    Product myProduct =
        Product.newBuilder()
            .setName(productId)
            .setDisplayName(productDisplayName)
            .setProductCategory(productCategory)
            .build();
    Product product = client.createProduct(formattedParent, myProduct, productId);
    // Display the product information
    System.out.println(String.format("Product name: %s", product.getName()));
  }
}

Node.js

// Imports the Google Cloud client library
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productId = 'Id of the product';
// const productDisplayName = 'Display name of the product';
// const productCategory = 'Catoegory of the product';

// Resource path that represents Google Cloud Platform location.
const locationPath = client.locationPath(projectId, location);

const product = {
  displayName: productDisplayName,
  productCategory: productCategory,
};

const request = {
  parent: locationPath,
  product: product,
  productId: productId,
};

const [createdProduct] = await client.createProduct(request);
console.log(`Product name: ${createdProduct.name}`);

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\Product;

/**
 * Create one product
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productId ID of the product
 * @param string $productDisplayName Display name of the product
 * @param string $productCategory Category of the product
 */
function product_create($projectId, $location, $productId, $productDisplayName, $productCategory)
{
    $client = new ProductSearchClient();

    # a resource that represents Google Cloud Platform location.
    $locationPath = $client->locationName($projectId, $location);

    # create a product with the product specification in the region.
    # set product name and product display name.
    $product = (new Product())
        ->setDisplayName($productDisplayName)
        ->setProductCategory($productCategory);

    # the response is the product with the `name` field populated.
    $response = $client->createProduct($locationPath, $product, ['productId' => $productId]);

    # display the product information.
    printf('Product name: %s' . PHP_EOL, $response->getName());

    $client->close();
}

Python

from google.cloud import vision

def create_product(
        project_id, location, product_id, product_display_name,
        product_category):
    """Create one product.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_id: Id of the product.
        product_display_name: Display name of the product.
        product_category: Category of the product.
    """
    client = vision.ProductSearchClient()

    # A resource that represents Google Cloud Platform location.
    location_path = client.location_path(project=project_id, location=location)

    # Create a product with the product specification in the region.
    # Set product name and product display name.
    product = vision.types.Product(
        display_name=product_display_name,
        product_category=product_category)

    # The response is the product with the `name` field populated.
    response = client.create_product(
        parent=location_path,
        product=product,
        product_id=product_id)

    # Display the product information.
    print('Product name: {}'.format(response.name))

Response

Product name: projects/prj-prod-search-tutorials/locations/us-east1/products/P_CLOTH-SHOE_46903668_070318
Product id: P_CLOTH-SHOE_46903668_070318
Product display name: Blue Dress
Product category: apparel
Product description: Short sleeved and 1950s style satin dress
Product labels:
  Product label 1:
        key: style
        value: women
  Product label 2:
        key: category
        value: dress
  Product label 3:
        key: color
        value: dark-blue

3. Add a Product to a Product Set

After a product set and a product have been created, you can add the product to the product set.

Command-line

To add a product to a product set, send a POST request to the following URI.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
  • Replace product-set-id with the id for the product set to associate with the product.
  • Replace product-id with the id for the product to add.
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/project-id/locations/location-id/productSets/product-set-id:addProduct -d '{
  'product': 'product-id'
}'

Go

import (
	"context"
	"fmt"
	"io"

	vision "cloud.google.com/go/vision/apiv1"
	visionpb "google.golang.org/genproto/googleapis/cloud/vision/v1"
)

// addProductToProductSet adds a product to a product set.
func addProductToProductSet(w io.Writer, projectID string, location string, productID string, productSetID string) error {
	ctx := context.Background()
	c, err := vision.NewProductSearchClient(ctx)
	if err != nil {
		return fmt.Errorf("NewProductSearchClient: %v", err)
	}

	req := &visionpb.AddProductToProductSetRequest{
		Name:    fmt.Sprintf("projects/%s/locations/%s/productSets/%s", projectID, location, productSetID),
		Product: fmt.Sprintf("projects/%s/locations/%s/products/%s", projectID, location, productID),
	}

	if err = c.AddProductToProductSet(ctx, req); err != nil {
		return fmt.Errorf("NewProductSearchClient: %v", err)
	}

	fmt.Fprintf(w, "Product added to product set.\n")

	return nil
}

Java

/**
 * Add a product to a product set.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productId - Id of the product.
 * @param productSetId - Id of the product set.
 * @throws IOException - on I/O errors.
 */
public static void addProductToProductSet(
    String projectId, String computeRegion, String productId, String productSetId)
    throws IOException {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // Get the full path of the product set.
    String formattedName =
        ProductSearchClient.formatProductSetName(projectId, computeRegion, productSetId);

    // Get the full path of the product.
    String productPath = ProductName.of(projectId, computeRegion, productId).toString();

    // Add the product to the product set.
    client.addProductToProductSet(formattedName, productPath);

    System.out.println(String.format("Product added to product set."));
  }
}

Node.js

const vision = require('@google-cloud/vision');

const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productId = 'Id of the product';
// const productSetId = 'Id of the product set';

const productPath = client.productPath(projectId, location, productId);
const productSetPath = client.productSetPath(
  projectId,
  location,
  productSetId
);

const request = {
  name: productSetPath,
  product: productPath,
};

await client.addProductToProductSet(request);
console.log(`Product added to product set.`);

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;

/**
 * Create a product set
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productId ID of the product
 * @param string $productSetId ID of the product set
 */
function product_set_add_product($projectId, $location, $productId, $productSetId)
{
    $client = new ProductSearchClient();

    # get the name of the product set
    $productSetPath = $client->productSetName($projectId, $location, $productSetId);

    # get the name of the product.
    $productPath = $client->productName($projectId, $location, $productId);

    # add product to product set
    $client->addProductToProductSet($productSetPath, $productPath);
    print('Product added to product set.' . PHP_EOL);

    $client->close();
}

Python

from google.cloud import vision

def add_product_to_product_set(
        project_id, location, product_id, product_set_id):
    """Add a product to a product set.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_id: Id of the product.
        product_set_id: Id of the product set.
    """
    client = vision.ProductSearchClient()

    # Get the full path of the product set.
    product_set_path = client.product_set_path(
        project=project_id, location=location,
        product_set=product_set_id)

    # Get the full path of the product.
    product_path = client.product_path(
        project=project_id, location=location, product=product_id)

    # Add the product to the product set.
    client.add_product_to_product_set(
        name=product_set_path, product=product_path)
    print('Product added to product set.')

Response

Product added to product set.

4. Update a product

If you need to update a product or product set after it has been created, you can use our update methods. This example shows a product update where the labels are changed:

Command-line

To update a product, send a PATCH request to the following URI.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
  • Replace product-id with the id for the product to update.
curl -X PATCH \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/project-id/locations/location-id/products/product-id -d '{
    'updateMask': { 'paths': ['display_name'] }
   'product': { 'display_name': 'new-display-name' },
}

Go

import (
	"context"
	"fmt"
	"io"

	vision "cloud.google.com/go/vision/apiv1"
	visionpb "google.golang.org/genproto/googleapis/cloud/vision/v1"
	field_mask "google.golang.org/genproto/protobuf/field_mask"
)

// updateProductLabels updates product labels of a product.
func updateProductLabels(w io.Writer, projectID string, location string, productID string, key string, value string) error {
	ctx := context.Background()
	c, err := vision.NewProductSearchClient(ctx)
	if err != nil {
		return fmt.Errorf("NewProductSearchClient: %v", err)
	}

	req := &visionpb.UpdateProductRequest{
		UpdateMask: &field_mask.FieldMask{
			Paths: []string{
				"product_labels",
			},
		},
		Product: &visionpb.Product{
			Name: fmt.Sprintf("projects/%s/locations/%s/products/%s", projectID, location, productID),
			ProductLabels: []*visionpb.Product_KeyValue{
				{
					Key:   key,
					Value: value,
				},
			},
		},
	}

	resp, err := c.UpdateProduct(ctx, req)
	if err != nil {
		return fmt.Errorf("UpdateProduct: %v", err)
	}

	fmt.Fprintf(w, "Product name: %s\n", resp.Name)
	fmt.Fprintf(w, "Updated product labels: %s\n", resp.ProductLabels)

	return nil
}

Java

/**
 * Update the product labels.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productId -Id of the product.
 * @param productLabels - Labels of the product.
 * @throws IOException - on I/O errors.
 */
public static void updateProductLabels(
    String projectId, String computeRegion, String productId, String productLabels)
    throws IOException {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // Get the full path of the product.
    String formattedName =
        ProductSearchClient.formatProductName(projectId, computeRegion, productId);

    // Set product name, product labels and product display name.
    // Multiple labels are also supported.
    Product product =
        Product.newBuilder()
            .setName(formattedName)
            .addProductLabels(
                KeyValue.newBuilder()
                    .setKey(productLabels.split(",")[0].split("=")[0])
                    .setValue(productLabels.split(",")[0].split("=")[1])
                    .build())
            .build();

    // Set product update field name.
    FieldMask updateMask = FieldMask.newBuilder().addPaths("product_labels").build();

    // Update the product.
    Product updatedProduct = client.updateProduct(product, updateMask);
    // Display the product information
    System.out.println(String.format("Product name: %s", updatedProduct.getName()));
    System.out.println(String.format("Updated product labels: "));
    for (Product.KeyValue element : updatedProduct.getProductLabelsList()) {
      System.out.println(String.format("%s: %s", element.getKey(), element.getValue()));
    }
  }
}

Node.js

// Imports the Google Cloud client library
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productId = 'Id of the product';
// const key = 'The key of the label';
// const value = 'The value of the label';

// Resource path that represents full path to the product.
const productPath = client.productPath(projectId, location, productId);

const product = {
  name: productPath,
  productLabels: [
    {
      key: key,
      value: value,
    },
  ],
};

const updateMask = {
  paths: ['product_labels'],
};

const request = {
  product: product,
  updateMask: updateMask,
};

const [updatedProduct] = await client.updateProduct(request);
console.log(`Product name: ${updatedProduct.name}`);
console.log(`Product display name: ${updatedProduct.displayName}`);
console.log(`Product description: ${updatedProduct.description}`);
console.log(`Product category: ${updatedProduct.productCategory}`);
console.log(
  `Product Labels: ${updatedProduct.productLabels[0].key}: ${
    updatedProduct.productLabels[0].value
  }`
);

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\Product\KeyValue;
use Google\Cloud\Vision\V1\Product;
use Google\Protobuf\FieldMask;

/**
 * Update product labels
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productId ID of the product
 * @param string $key Key of the label to update
 * @param string $value Value of label to update
 */
function product_update($projectId, $location, $productId, $key, $value)
{
    $client = new ProductSearchClient();

    # get the name of the product.
    $productPath = $client->productName($projectId, $location, $productId);

    # set product name, product label and product display name.
    # multiple labels are also supported.
    $keyValue = (new KeyValue())
        ->setKey($key)
        ->setValue($value);
    $product = (new Product())
        ->setName($productPath)
        ->setProductLabels([$keyValue]);

    # updating only the product labels field here.
    $updateMask = (new FieldMask())
        ->setPaths(['product_labels']);

    # this overwrites the product_labels.
    $updatedProduct = $client->updateProduct($product, ['updateMask' => $updateMask]);

    # display the product information.
    printf('Product name: %s' . PHP_EOL, $updatedProduct->getName());
    print('Product labels: ' . PHP_EOL);
    foreach ($product->getProductLabels() as $label) {
        printf('%s : %s' . PHP_EOL, $label->getKey(), $label->getValue());
    }

    $client->close();
}

Python

from google.cloud import vision

def update_product_labels(
        project_id, location, product_id, key, value):
    """Update the product labels.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_id: Id of the product.
        key: The key of the label.
        value: The value of the label.
    """
    client = vision.ProductSearchClient()

    # Get the name of the product.
    product_path = client.product_path(
        project=project_id, location=location, product=product_id)

    # Set product name, product label and product display name.
    # Multiple labels are also supported.
    key_value = vision.types.Product.KeyValue(key=key, value=value)
    product = vision.types.Product(
        name=product_path,
        product_labels=[key_value])

    # Updating only the product_labels field here.
    update_mask = vision.types.FieldMask(paths=['product_labels'])

    # This overwrites the product_labels.
    updated_product = client.update_product(
        product=product, update_mask=update_mask)

    # Display the updated product information.
    print('Product name: {}'.format(updated_product.name))
    print('Updated product labels: {}'.format(product.product_labels))

Response

Product name: projects/prj-prod-search-tutorials/locations/us-east1/products/P_CLOTH-SHOE_46903668_070318
Product id: P_CLOTH-SHOE_46903668_070318
Product display name: Blue Dress
Updated product labels:
  Product label 1:
        key: style
        value: women
  Product label 2:
        key: category
        value: dress
  Product label 3:
        key: color
        value: blue
Product description: Short sleeved and 1950s style satin dress

5. Create a Product's Reference Image

Creating a reference image for an individual product allows Vision Product Search to search for the product by this image after it is indexed. You can have multiple reference images in a product, particularly if you desire a better match quality.

You can add a new reference image to a product at any time.

When creating a reference image you have the option to include bounding poly coordinates. A bounding poly identifies an area of interest in the reference image. For example, if you create a reference image for a product that is a jacket, you can provide the coordinates for the jacket in the bounding poly argument, and the system will only consider the jacket when looking for product matches. Note: You can provide multiple bounding polys at index time, though at query time the API supports only a single bounding poly.

A convenient way to get the bounding poly coordinates for an image is to use Vision API object localization. For more information on object localization, see Detecting Multiple Objects.

Command-line

To create a reference image, send a POST request to the following URI.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
  • Replace product-id with the id of the product to associate the reference image with.
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/project-id/locations/location-id/products/product-id/referenceImages -d '{
  'uri': 'gs://bucket/path/to/red-shoe.jpg',
  'boundingPolys': [
    {
      'vertices': [
        {
          'x': 44,
          'y': 252
        },
        {
          'x': 44,
          'y': 1351
        },
        {
          'x': 1161,
          'y': 252
        },
        {
          'x': 1161,
          'y': 1351
        }
      ]
    }
  ]
}
The boundingPolys field is optional. If omitted, the service will infer the values of the bounds for the image.

Go

import (
	"context"
	"fmt"
	"io"

	vision "cloud.google.com/go/vision/apiv1"
	visionpb "google.golang.org/genproto/googleapis/cloud/vision/v1"
)

// createReferenceImage creates a reference image for a product.
func createReferenceImage(w io.Writer, projectID string, location string, productID string, referenceImageID string, gcsURI string) error {
	ctx := context.Background()
	c, err := vision.NewProductSearchClient(ctx)
	if err != nil {
		return fmt.Errorf("NewProductSearchClient: %v", err)
	}

	req := &visionpb.CreateReferenceImageRequest{
		Parent: fmt.Sprintf("projects/%s/locations/%s/products/%s", projectID, location, productID),
		ReferenceImage: &visionpb.ReferenceImage{
			Uri: gcsURI,
		},
		ReferenceImageId: referenceImageID,
	}

	resp, err := c.CreateReferenceImage(ctx, req)
	if err != nil {
		return fmt.Errorf("CreateReferenceImage: %v", err)
	}

	fmt.Fprintf(w, "Reference image name: %s\n", resp.Name)
	fmt.Fprintf(w, "Reference image uri: %s\n", resp.Uri)

	return nil
}

Java

/**
 * Create a reference image.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productId - Id of the product.
 * @param referenceImageId - Id of the image.
 * @param gcsUri - Google Cloud Storage path of the input image.
 * @throws IOException - on I/O errors.
 */
public static void createReferenceImage(
    String projectId,
    String computeRegion,
    String productId,
    String referenceImageId,
    String gcsUri)
    throws IOException {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // Get the full path of the product.
    String formattedParent =
        ProductSearchClient.formatProductName(projectId, computeRegion, productId);
    // Create a reference image.
    ReferenceImage referenceImage = ReferenceImage.newBuilder().setUri(gcsUri).build();

    ReferenceImage image =
        client.createReferenceImage(formattedParent, referenceImage, referenceImageId);
    // Display the reference image information.
    System.out.println(String.format("Reference image name: %s", image.getName()));
    System.out.println(String.format("Reference image uri: %s", image.getUri()));
  }
}

Node.js

const vision = require('@google-cloud/vision');

const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productId = 'Id of the product';
// const referenceImageId = 'Id of the reference image';
// const gcsUri = 'Google Cloud Storage path of the input image';

const formattedParent = client.productPath(projectId, location, productId);

const referenceImage = {
  uri: gcsUri,
};

const request = {
  parent: formattedParent,
  referenceImage: referenceImage,
  referenceImageId: referenceImageId,
};

const [response] = await client.createReferenceImage(request);
console.log(`response.name: ${response.name}`);
console.log(`response.uri: ${response.uri}`);

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\ReferenceImage;

/**
 * Create a reference image
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productId ID of the product
 * @param string $referenceImageId ID of the reference image
 * @param string $gcsUri Google Cloud Storage path of the input image
 */
function product_image_create($projectId, $location, $productId, $referenceImageId, $gcsUri)
{
    $client = new ProductSearchClient();

    # get the name of the product.
    $productPath = $client->productName($projectId, $location, $productId);

    # create a reference image.
    $referenceImage = (new ReferenceImage())
        ->setUri($gcsUri);

    # the response is the reference image with `name` populated.
    $image = $client->createReferenceImage($productPath, $referenceImage, ['referenceImageId' => $referenceImageId]);

    # display the reference image information
    printf('Reference image name: %s' . PHP_EOL, $image->getName());
    printf('Reference image uri: %s' . PHP_EOL, $image->getUri());

    $client->close();
}

Python

from google.cloud import vision

def create_reference_image(
        project_id, location, product_id, reference_image_id, gcs_uri):
    """Create a reference image.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_id: Id of the product.
        reference_image_id: Id of the reference image.
        gcs_uri: Google Cloud Storage path of the input image.
    """
    client = vision.ProductSearchClient()

    # Get the full path of the product.
    product_path = client.product_path(
        project=project_id, location=location, product=product_id)

    # Create a reference image.
    reference_image = vision.types.ReferenceImage(uri=gcs_uri)

    # The response is the reference image with `name` populated.
    image = client.create_reference_image(
        parent=product_path,
        reference_image=reference_image,
        reference_image_id=reference_image_id)

    # Display the reference image information.
    print('Reference image name: {}'.format(image.name))
    print('Reference image uri: {}'.format(image.uri))

Response

Reference image name: projects/prj-prod-search-tutorials/locations/us-east1/products/P_CLOTH-SHOE_46903668_070318/referenceImages/I_469a896b70ba11e8be97d20059124800_070418
Reference image id: I_469a896b70ba11e8be97d20059124800_070418
Reference image uri: gs://product-search-tutorial/dress-shoe-dataset/469a896b70ba11e8be97d20059124800.jpg
Reference image bounding polygons:
vertices {
  x: 80
  y: 50
}
vertices {
  x: 80
  y: 660
}
vertices {
  x: 300
  y: 50
}
vertices {
  x: 430
  y: 660
}

Step 2: Search For Matching Products

This interface allows you to query the product catalog you created by taking a new image as input and searching for the best matching product.

Similar to creating a reference image, when searching for matching images you have the option to include bounding poly coordinates. A bounding poly identifies the area of interest in the source image that you want to find matches for. For example, if your source image contains both a dress and a purse, and you only want to find matches for the dress, you can identify the bounding poly coordinates for the region of the picture that contains only the dress. By default, if no bounding poly is specified the API determines the largest bounding poly and queries it automatically.

A convenient way to get the bounding poly coordinates for an image is to use Vision API object localization. For more information on object localization, see Detecting Multiple Objects. For example, you can explicitly query a full image by specifying a bounding poly of the entire image box: [(0, 0), (0, 1), (1, 1), (1, 0)].

The request returns an API response that includes the best matching product for an image with the score and the matching image. This image is returned using the highest confidence value.

Command-line

To query the product set, make a POST request to the v1/images:annotate endpoint, specifying the PRODUCT_SEARCH feature and passing the product set's id in the imageContext field. You must use authentication for the same project as the product set being queried. For a full list of fields, refer to the ProductSearchParams reference.
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/images:annotate -d '{
  "requests": [
    {
      "image": {
        "content": BASE64_ENCODED_IMAGE
      },
      "features": [
        {
          "type": "PRODUCT_SEARCH"
        }
      ],
      "imageContext": {
        "productSearchParams": {
          "productSet": "projects/project-id/locations/location-id/productSets/product-set-id",
          "productCategories": [
               "homegoods"
             ],
        "filter": "style = womens"
        }
      }
    }
  ]
}

Go

import (
	"context"
	"fmt"
	"io"
	"os"

	vision "cloud.google.com/go/vision/apiv1"
	visionpb "google.golang.org/genproto/googleapis/cloud/vision/v1"
)

// getSimilarProducts searches for products from a product set similar to products in an image file.
func getSimilarProducts(w io.Writer, projectID string, location string, productSetID string, productCategory string, file string, filter string) error {
	ctx := context.Background()
	c, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return fmt.Errorf("NewImageAnnotatorClient: %v", err)
	}

	f, err := os.Open(file)
	if err != nil {
		return fmt.Errorf("Open: %v", err)
	}
	defer f.Close()

	image, err := vision.NewImageFromReader(f)
	if err != nil {
		return fmt.Errorf("NewImageFromReader: %v", err)
	}

	ictx := &visionpb.ImageContext{
		ProductSearchParams: &visionpb.ProductSearchParams{
			ProductSet:        fmt.Sprintf("projects/%s/locations/%s/productSets/%s", projectID, location, productSetID),
			ProductCategories: []string{productCategory},
			Filter:            filter,
		},
	}

	response, err := c.ProductSearch(ctx, image, ictx)
	if err != nil {
		return fmt.Errorf("ProductSearch: %v", err)
	}

	fmt.Fprintf(w, "Product set index time:\n")
	fmt.Fprintf(w, "seconds: %d\n", response.IndexTime.Seconds)
	fmt.Fprintf(w, "nanos: %d\n", response.IndexTime.Nanos)

	fmt.Fprintf(w, "Search results:\n")
	for _, result := range response.Results {
		fmt.Fprintf(w, "Score(Confidence): %f\n", result.Score)
		fmt.Fprintf(w, "Image name: %s\n", result.Image)

		fmt.Fprintf(w, "Prodcut name: %s\n", result.Product.Name)
		fmt.Fprintf(w, "Product display name: %s\n", result.Product.DisplayName)
		fmt.Fprintf(w, "Product labels: %s\n", result.Product.ProductLabels)
	}

	return nil
}

Java

/**
 * Search similar products to image in local file.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productSetId - Id of the product set.
 * @param productCategory - Category of the product.
 * @param filePath - Local file path of the image to be searched
 * @param filter - Condition to be applied on the labels. Example for filter: (color = red OR
 *     color = blue) AND style = kids It will search on all products with the following labels:
 *     color:red AND style:kids color:blue AND style:kids
 * @throws IOException - on I/O errors.
 */
public static void getSimilarProductsFile(
    String projectId,
    String computeRegion,
    String productSetId,
    String productCategory,
    String filePath,
    String filter)
    throws IOException {
  try (ImageAnnotatorClient queryImageClient = ImageAnnotatorClient.create()) {

    // Get the full path of the product set.
    String productSetPath =
        ProductSearchClient.formatProductSetName(projectId, computeRegion, productSetId);

    // Read the image as a stream of bytes.
    File imgPath = new File(filePath);
    byte[] content = Files.readAllBytes(imgPath.toPath());

    // Create annotate image request along with product search feature.
    Feature featuresElement = Feature.newBuilder().setType(Type.PRODUCT_SEARCH).build();
    // The input image can be a HTTPS link or Raw image bytes.
    // Example:
    // To use HTTP link replace with below code
    //  ImageSource source = ImageSource.newBuilder().setImageUri(imageUri).build();
    //  Image image = Image.newBuilder().setSource(source).build();
    Image image = Image.newBuilder().setContent(ByteString.copyFrom(content)).build();
    ImageContext imageContext =
        ImageContext.newBuilder()
            .setProductSearchParams(
                ProductSearchParams.newBuilder()
                    .setProductSet(productSetPath)
                    .addProductCategories(productCategory)
                    .setFilter(filter))
            .build();

    AnnotateImageRequest annotateImageRequest =
        AnnotateImageRequest.newBuilder()
            .addFeatures(featuresElement)
            .setImage(image)
            .setImageContext(imageContext)
            .build();
    List<AnnotateImageRequest> requests = Arrays.asList(annotateImageRequest);

    // Search products similar to the image.
    BatchAnnotateImagesResponse response = queryImageClient.batchAnnotateImages(requests);

    List<Result> similarProducts =
        response.getResponses(0).getProductSearchResults().getResultsList();
    System.out.println("Similar Products: ");
    for (Result product : similarProducts) {
      System.out.println(String.format("\nProduct name: %s", product.getProduct().getName()));
      System.out.println(
          String.format("Product display name: %s", product.getProduct().getDisplayName()));
      System.out.println(
          String.format("Product description: %s", product.getProduct().getDescription()));
      System.out.println(String.format("Score(Confidence): %s", product.getScore()));
      System.out.println(String.format("Image name: %s", product.getImage()));
    }
  }
}

Node.js

async function getSimilarProductsFile(
  projectId,
  location,
  productSetId,
  productCategory,
  filePath,
  filter
) {
  // Imports the Google Cloud client library
  const vision = require('@google-cloud/vision');
  const fs = require('fs');
  // Creates a client
  const productSearchClient = new vision.ProductSearchClient();
  const imageAnnotatorClient = new vision.ImageAnnotatorClient();

  /**
   * TODO(developer): Uncomment the following line before running the sample.
   */
  // const projectId = 'nodejs-docs-samples';
  // const location = 'us-west1';
  // const productSetId = 'indexed_product_set_id_for_testing';
  // const productCategory = 'apparel';
  // const filePath = './resources/shoes_1.jpg';
  // const filter = '';
  const productSetPath = productSearchClient.productSetPath(
    projectId,
    location,
    productSetId
  );
  const content = fs.readFileSync(filePath, 'base64');
  const request = {
    // The input image can be a GCS link or HTTPS link or Raw image bytes.
    // Example:
    // To use GCS link replace with below code
    // image: {source: {gcsImageUri: filePath}}
    // To use HTTP link replace with below code
    // image: {source: {imageUri: filePath}}
    image: {content: content},
    features: [{type: 'PRODUCT_SEARCH'}],
    imageContext: {
      productSearchParams: {
        productSet: productSetPath,
        productCategories: [productCategory],
        filter: filter,
      },
    },
  };
  const [response] = await imageAnnotatorClient.batchAnnotateImages({
    requests: [request],
  });
  console.log('Search Image:', filePath);
  const results = response['responses'][0]['productSearchResults']['results'];
  console.log('\nSimilar product information:');
  results.forEach(result => {
    console.log('Product id:', result['product'].name.split('/').pop(-1));
    console.log('Product display name:', result['product'].displayName);
    console.log('Product description:', result['product'].description);
    console.log('Product category:', result['product'].productCategory);
  });
}

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ImageAnnotatorClient;
use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\ProductSearchParams;

/**
 * Search similar products to image
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productSetId ID of the product set
 * @param string $productCategory Category of the product
 * @param string $filePath Local file path of the image to be searched
 * @param string $filter Condition to be applied on the labels
 */
function product_search_similar($projectId, $location, $productSetId, $productCategory, $filePath, $filter)
{
    $imageAnnotatorClient = new ImageAnnotatorClient();
    $productSearchClient = new ProductSearchClient();

    # read the image
    $image = file_get_contents($filePath);

    # get the name of the product set
    $productSetPath = $productSearchClient->productSetName($projectId, $location, $productSetId);

    # product search specific parameters
    $productSearchParams = (new ProductSearchParams())
        ->setProductSet($productSetPath)
        ->setProductCategories([$productCategory])
        ->setFilter($filter);

    # search products similar to the image
    $response = $imageAnnotatorClient->productSearch($image, $productSearchParams);

    if ($productSearchResults = $response->getProductSearchResults()) {
        $indexTime = $productSearchResults->getIndexTime();
        printf('Product set index time: %d seconds %d nanos' . PHP_EOL, $indexTime->getSeconds(), $indexTime->getNanos());

        $results = $productSearchResults->getResults();
        print('Search results: ' . PHP_EOL);
        foreach ($results as $result) {
            printf('Score (confidence): %d' . PHP_EOL, $result->getScore());

            # display the product information.
            $product = $result->getProduct();
            $productName = $product->getName();
            $productNameArray = explode('/', $productName);

            printf('Product name: %s' . PHP_EOL, $productName);
            printf('Product id: %s' . PHP_EOL, end($productNameArray));
            printf('Product display name: %s' . PHP_EOL, $product->getDisplayName());
            printf('Product description: %s' . PHP_EOL, $product->getDescription());
            printf('Product category: %s' . PHP_EOL, $product->getProductCategory());
            print('Product labels: ' . PHP_EOL);
            foreach ($product->getProductLabels() as $label) {
                printf('%s : %s' . PHP_EOL, $label->getKey(), $label->getValue());
            }
        }
    } else {
        print($response->getError()->getMessage());
    }

    $imageAnnotatorClient->close();
    $productSearchClient->close();
}

Python

from google.cloud import vision

def get_similar_products_file(
        project_id, location, product_set_id, product_category,
        file_path, filter):
    """Search similar products to image.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_set_id: Id of the product set.
        product_category: Category of the product.
        file_path: Local file path of the image to be searched.
        filter: Condition to be applied on the labels.
        Example for filter: (color = red OR color = blue) AND style = kids
        It will search on all products with the following labels:
        color:red AND style:kids
        color:blue AND style:kids
    """
    # product_search_client is needed only for its helper methods.
    product_search_client = vision.ProductSearchClient()
    image_annotator_client = vision.ImageAnnotatorClient()

    # Read the image as a stream of bytes.
    with open(file_path, 'rb') as image_file:
        content = image_file.read()

    # Create annotate image request along with product search feature.
    image = vision.types.Image(content=content)

    # product search specific parameters
    product_set_path = product_search_client.product_set_path(
        project=project_id, location=location,
        product_set=product_set_id)
    product_search_params = vision.types.ProductSearchParams(
        product_set=product_set_path,
        product_categories=[product_category],
        filter=filter)
    image_context = vision.types.ImageContext(
        product_search_params=product_search_params)

    # Search products similar to the image.
    response = image_annotator_client.product_search(
        image, image_context=image_context)

    index_time = response.product_search_results.index_time
    print('Product set index time:')
    print('  seconds: {}'.format(index_time.seconds))
    print('  nanos: {}\n'.format(index_time.nanos))

    results = response.product_search_results.results

    print('Search results:')
    for result in results:
        product = result.product

        print('Score(Confidence): {}'.format(result.score))
        print('Image name: {}'.format(result.image))

        print('Product name: {}'.format(product.name))
        print('Product display name: {}'.format(
            product.display_name))
        print('Product description: {}\n'.format(product.description))
        print('Product labels: {}\n'.format(product.product_labels))

Apparel Response Example

Search Image:
  D:/product/final/images-20180618T073733Z-01/images/469355b570ba11e88ff2d20059124800.jpg

Image search result

Similar product information:
 Product id: 46930b6b
 Product display name: Evening gown
 Product description: Blue evening gown in 1940s style
 Product category: apparel
 style: women
 category: dress
 color: blue

Searching with a Label

The following search example includes a filter based on color.

Request

Make a search request by executing the following request with method get_similar_products_file() or get_similar_products_uri(). Product set id, local image file path and filter are passed as arguments. This input image is also present in “resources/input/”.

Python

python product_search.py get_similar_products_file "12000002" "D:/product/final/images-20180618T073733Z-001/images/469355b570ba11e88ff2d20059124800.jpg" "color=white"

Response

Search Image:
  D:/product/final/images-20180618T073733Z-001/images/469355b570ba11e88ff2d20059124800.jpg

Image search result 2

Similar product information:
 Product id: p569d4e7a1
 Product display name: Wedding Dress
 Product description: Elegant Wedding Dress for women
 Product category: apparel
 style: women
 category: dress
 color: white
Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Vision API Product Search
Need help? Visit our support page.