Creating your reference images & indexing

Reference images are images containing various views of your products. The following recommendations apply:

  • Make sure the size of the file doesn't exceed the maximum size (20MB).
  • Consider viewpoints that logically highlight the product and contain relevant visual information.
  • Create reference images that supplement any missing viewpoints. For example, if you only have images of the right shoe in a pair, provide mirrored versions of those files as the left shoe.
  • Upload the highest resolution image available.
  • Show the product against a white background.
  • Convert PNGs with transparent backgrounds to a solid background.

Images must be stored in a Google Cloud Storage bucket. If you're authenticating your image create call with an API key, the bucket must be public. If you're authenticating with a service account, that service account must have read access on the bucket.

Creating a single reference image

You can add a reference image to an existing product. This then allows you to search for the product by the image.

Command-line

To create a reference image, send a POST request to the following URI.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
  • Replace product-id with the id of the product to associate the reference image with.
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/projects/project-id/locations/location-id/products/product-id/referenceImages -d '{
  'uri': 'gs://bucket/path/to/red-shoe.jpg',
  'boundingPolys': [
    {
      'vertices': [
        {
          'x': 44,
          'y': 252
        },
        {
          'x': 44,
          'y': 1351
        },
        {
          'x': 1161,
          'y': 252
        },
        {
          'x': 1161,
          'y': 1351
        }
      ]
    }
  ]
}
The boundingPolys field is optional. If omitted, the service will infer the values of the bounds for the image.

Java

/**
 * Create a reference image.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productId - Id of the product.
 * @param referenceImageId - Id of the image.
 * @param gcsUri - Google Cloud Storage path of the input image.
 * @throws IOException - on I/O errors.
 */
public static void createReferenceImage(
    String projectId,
    String computeRegion,
    String productId,
    String referenceImageId,
    String gcsUri)
    throws IOException {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // Get the full path of the product.
    String formattedParent =
        ProductSearchClient.formatProductName(projectId, computeRegion, productId);
    // Create a reference image.
    ReferenceImage referenceImage = ReferenceImage.newBuilder().setUri(gcsUri).build();

    ReferenceImage image =
        client.createReferenceImage(formattedParent, referenceImage, referenceImageId);
    // Display the reference image information.
    System.out.println(String.format("Reference image name: %s", image.getName()));
    System.out.println(String.format("Reference image uri: %s", image.getUri()));
  }
}

Node.js

const vision = require('@google-cloud/vision');

const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productId = 'Id of the product';
// const referenceImageId = 'Id of the reference image';
// const gcsUri = 'Google Cloud Storage path of the input image';

const formattedParent = client.productPath(projectId, location, productId);

const referenceImage = {
  uri: gcsUri,
};

const request = {
  parent: formattedParent,
  referenceImage: referenceImage,
  referenceImageId: referenceImageId,
};

const [response] = await client.createReferenceImage(request);
console.log(`response.name: ${response.name}`);
console.log(`response.uri: ${response.uri}`);

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\ReferenceImage;

/**
 * Create a reference image
 *
 * @param string $projectId Your Google Cloud project ID
 * @param string $location Google Cloud compute region name
 * @param string $productId ID of the product
 * @param string $referenceImageId ID of the reference image
 * @param string $gcsUri Google Cloud Storage path of the input image
 */
function product_image_create($projectId, $location, $productId, $referenceImageId, $gcsUri)
{
    $client = new ProductSearchClient();

    # get the name of the product.
    $productPath = $client->productName($projectId, $location, $productId);

    # create a reference image.
    $referenceImage = (new ReferenceImage())
        ->setUri($gcsUri);

    # the response is the reference image with `name` populated.
    $image = $client->createReferenceImage($productPath, $referenceImage, ['referenceImageId' => $referenceImageId]);

    # display the reference image information
    printf('Reference image name: %s' . PHP_EOL, $image->getName());
    printf('Reference image uri: %s' . PHP_EOL, $image->getUri());

    $client->close();
}

Python

from google.cloud import vision

def create_reference_image(
        project_id, location, product_id, reference_image_id, gcs_uri):
    """Create a reference image.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_id: Id of the product.
        reference_image_id: Id of the reference image.
        gcs_uri: Google Cloud Storage path of the input image.
    """
    client = vision.ProductSearchClient()

    # Get the full path of the product.
    product_path = client.product_path(
        project=project_id, location=location, product=product_id)

    # Create a reference image.
    reference_image = vision.types.ReferenceImage(uri=gcs_uri)

    # The response is the reference image with `name` populated.
    image = client.create_reference_image(
        parent=product_path,
        reference_image=reference_image,
        reference_image_id=reference_image_id)

    # Display the reference image information.
    print('Reference image name: {}'.format(image.name))
    print('Reference image uri: {}'.format(image.uri))

Creating multiple reference images with bulk import

You can also create reference images at the same time you create a product set and multiple products.

Create reference images in bulk by passing the Google Cloud Storage location of a CSV file to the import method. Thus, the CSV file and the images it points to must both be in a Google Cloud Storage bucket. This CSV source file must be public.

CSV format

image-uri,[image-id],product-set-id,product-id,product-category,[product-display-name],[label(s)],[bounding-poly]

See the CSV format how-to topic for more detailed information about formatting your CSV.

Bulk creation request

Command-line

To import the products in your product set, send a POST request to the following URI.
  • Replace project-id with the id of your Google Cloud Platform (GCP) project.
  • Replace location-id with a valid location identifier.Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.
curl -X POST \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
"https://vision.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION_ID/productSets:import" -d '{
  "inputConfig": {
    "gcsSource": {
      "csvFileUri": "gs://bucket/path/to/file.csv"
    }
  }
}'
A successful response contains a long-running operation object:
{
  "name": "locations/location-id/operations/operation-id"
}
To request the status of the operation, call the operations endpoint and pass the operation-id that was returned from your call to the import method. Replace location-id with the location id from your call to the import method.
curl -X GET -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://vision.googleapis.com/v1/locations/location-id/operations/operation-id
The response looks like:
{
  "name": "locations/us-west1/operations/eea5667a0ed419d8",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.BatchOperationMetadata",
    "state": "SUCCESSFUL",
    "endTime": "2018-04-18T21:45:14.926185506Z"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.ImportProductSetsResponse",
    "referenceImages": [
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_01/referenceImages/17196263426604931544-0",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_01.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_02/referenceImages/17196263426604931544-1",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_02.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_04/referenceImages/17196263426604931544-2",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_04.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_05/referenceImages/17196263426604931544-3",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_05.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_women_05/referenceImages/17196263426604931544-4",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_women_05.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_women_07/referenceImages/17196263426604931544-5",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_women_07.jpg"
      }
    ],
    "statuses": [
      {},
      {},
      {},
      {},
      {},
      {}
    ]
  }
}

Java

/**
 * Import images of different products in the product set.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param gcsUri - Google Cloud Storage URI.Target files must be in Product Search CSV format.
 * @throws Exception - on client errors.
 */
public static void importProductSets(String projectId, String computeRegion, String gcsUri)
    throws Exception {
  try (ProductSearchClient client = ProductSearchClient.create()) {

    // A resource that represents Google Cloud Platform location.
    String formattedParent = ProductSearchClient.formatLocationName(projectId, computeRegion);
    Builder gcsSource = ImportProductSetsGcsSource.newBuilder().setCsvFileUri(gcsUri);

    // Set the input configuration along with Google Cloud Storage URI
    ImportProductSetsInputConfig inputConfig =
        ImportProductSetsInputConfig.newBuilder().setGcsSource(gcsSource).build();

    // Import the product sets from the input URI.
    OperationFuture<ImportProductSetsResponse, BatchOperationMetadata> response =
        client.importProductSetsAsync(formattedParent, inputConfig);

    System.out.println(String.format("Processing operation name: %s", response.getName()));
    ImportProductSetsResponse results = response.get();
    System.out.println("Processing done.");
    System.out.println("Results of the processing:");

    for (int i = 0; i < results.getStatusesCount(); i++) {
      System.out.println(
          String.format(
              "Status of processing line %s of the csv: %s", i, results.getStatuses(i)));
      // Check the status of reference image.
      if (results.getStatuses(i).getCode() == 0) {
        ReferenceImage referenceImage = results.getReferenceImages(i);
        System.out.println(referenceImage);
      } else {
        System.out.println("No reference image.");
      }
    }
  }
}

Node.js

async function importProductSets(projectId, location, gcsUri) {
  // Imports the Google Cloud client library
  const vision = require('@google-cloud/vision');
  // Creates a client
  const client = new vision.ProductSearchClient();

  /**
   * TODO(developer): Uncomment the following line before running the sample.
   */
  // const projectId = 'Your Google Cloud project Id';
  // const location = 'A compute region name';
  // const gcsUri = 'Google Cloud Storage path of the input image'';

  // A resource that represents Google Cloud Platform location.
  const projectLocation = client.locationPath(projectId, location);

  // Set the input configuration along with Google Cloud Storage URI
  const inputConfig = {
    gcsSource: {
      csvFileUri: gcsUri,
    },
  };

  // Import the product sets from the input URI.
  const [response, operation] = await client.importProductSets({
    parent: projectLocation,
    inputConfig: inputConfig,
  });

  console.log('Processing operation name: ', operation.name);

  // synchronous check of operation status
  const [result] = await response.promise();
  console.log('Processing done.');
  console.log('Results of the processing:');

  for (const i in result.statuses) {
    console.log('Status of processing ', i, 'of the csv:', result.statuses[i]);

    // Check the status of reference image
    if (result.statuses[i].code === 0) {
      console.log(result.referenceImages[i]);
    } else {
      console.log('No reference image.');
    }
  }

PHP

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ProductSearchClient;
use Google\Cloud\Vision\V1\ImportProductSetsGcsSource;
use Google\Cloud\Vision\V1\ImportProductSetsInputConfig;


/**
 * Import images of different products in the product set.
 *
 * @param string $projectId Your Google Cloud Project ID
 * @param string $location Google Cloud compute region name
 * @param string $gcsUri Google Cloud Storage URI
 */
function product_set_import($projectId, $location, $gcsUri)
{
    $client = new ProductSearchClient();

    # a resource that represents Google Cloud Platform location.
    $locationPath = $client->locationName($projectId, $location);

    # set the input configuration along with Google Cloud Storage URI
    $gcsSource = (new ImportProductSetsGcsSource())
        ->setCsvFileUri($gcsUri);
    $inputConfig = (new ImportProductSetsInputConfig())
        ->setGcsSource($gcsSource);

    # import the product sets from the input URI
    $operation = $client->importProductSets($locationPath, $inputConfig);
    $operationName = $operation->getName();
    printf('Processing operation name: %s' . PHP_EOL, $operationName);

    $operation->pollUntilComplete();
    print('Processing done.' . PHP_EOL);

    if ($result = $operation->getResult()) {
        $referenceImages = $result->getReferenceImages();

        foreach ($result->getStatuses() as $count => $status) {
            printf('Status of processing line %d of the csv: ' . PHP_EOL, $count);
            # check the status of reference image
            # `0` is the code for OK in google.rpc.Code.
            if ($status->getCode() == 0) {
                $referenceImage = $referenceImages[$count];
                printf('name: %s' . PHP_EOL, $referenceImage->getName());
                printf('uri: %s' . PHP_EOL, $referenceImage->getUri());
            } else {
                printf('Status code not OK: %s' . PHP_EOL, $status->getMessage());
            }
        }
    } else{
        print($operation->getError()->getMessage());
    }

    $client->close();
}

Python

def import_product_sets(project_id, location, gcs_uri):
    """Import images of different products in the product set.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        gcs_uri: Google Cloud Storage URI.
            Target files must be in Product Search CSV format.
    """
    client = vision.ProductSearchClient()

    # A resource that represents Google Cloud Platform location.
    location_path = client.location_path(
        project=project_id, location=location)

    # Set the input configuration along with Google Cloud Storage URI
    gcs_source = vision.types.ImportProductSetsGcsSource(
        csv_file_uri=gcs_uri)
    input_config = vision.types.ImportProductSetsInputConfig(
        gcs_source=gcs_source)

    # Import the product sets from the input URI.
    response = client.import_product_sets(
        parent=location_path, input_config=input_config)

    print('Processing operation name: {}'.format(response.operation.name))
    # synchronous check of operation status
    result = response.result()
    print('Processing done.')

    for i, status in enumerate(result.statuses):
        print('Status of processing line {} of the csv: {}'.format(
            i, status))
        # Check the status of reference image
        # `0` is the code for OK in google.rpc.Code.
        if status.code == 0:
            reference_image = result.reference_images[i]
            print(reference_image)
        else:
            print('Status code not OK: {}'.format(status.message))

Common import error messages include:

  • The number of columns in the csv line should be 8: If you're not providing a bounding box, the line must end with a comma. For example:
    "gs://example-reference-images/10002-002/10002-002_B.jpg","sample-set","sample-product-123","apparel",,
  • Image not found: Your Google Cloud Storage URI does not resolve to an image file.
  • Access denied: Either your image or your CSV file does not have the correct permissions. Image files must be accessible by the authentication method used for your request. If you are authenticating with an API Key, then your CSV files must be public.

Indexing

The Product Search index of products is updated approximately every 30 minutes. When images are added or deleted, the change won't be reflected in your Product Search responses until the index is next updated.

To make sure that indexing has completed successfully, check the indexTime field of a product set.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Vision API Product Search
Need help? Visit our support page.