Creating your reference images & indexing

Reference images are images containing various views of your products. The following recommendations apply:

  • Make sure the size of the file doesn't exceed the maximum size (30MB).
  • Consider viewpoints that logically highlight the product and contain relevant visual information.
  • Create reference images that supplement any missing viewpoints. For example, if you only have images of the right shoe in a pair, provide mirrored versions of those files as the left shoe.
  • Upload the highest resolution image available.
  • Show the product against a white background.
  • Convert PNGs with transparent backgrounds to a solid background.

Images must be stored in a Google Cloud Storage bucket. If you're authenticating your image create call with an API key, the bucket must be public. If you're authenticating with a service account, that service account must have read access on the bucket.

Bulk creation

Bulk create reference images by passing the Google Cloud Storage location of a CSV file to the import method. The source file must be public.

The CSV file must contain one image per line and contain the following columns:

  • imageUri: The Google Cloud Storage URI of the reference image.
  • productSetId: The identifier for the product set to import the images to.
  • productId: A user-defined ID for the product identified by the reference image. A productId can be associated with multiple reference images.
  • productCategory: Optional. The category for the product identified by the reference image. Inferred by the system if not specified in the create request. Allowed values are homegoods and apparel. Allowed values are also listed in the productCategory reference documentation.
  • labels: Optional. Labels that describe the products in the reference image. For example: color=green.
  • boundingPoly: Optional. Specifies the area of interest in the reference image. If not specified: 1) bounding box for the image is inferred by the Vision API, and 2) the line must end with a comma. See the examples below.

    The boundingPoly column should contain an even number of comma-separated numbers, with the format p1_x,p1_y,p2_x,p2_y,...,pn_x,pn_y. For example, 0.1,0.1,0.9,0.1,0.9,0.9,0.1,0.9. Non-negative integers should be used for absolute bounding polygons, and float values in [0, 1] should be used for normalized bounding polygons.

For example:

"gs://example-reference-images/10001-001/10001-001_A.jpg","sample-set","sample-product-123","apparel","style=womens","1,1,100,1,100,100,1,100"
"gs://example-reference-images/10002-002/10002-002_B.jpg","sample-set","sample-product-123","apparel",,

To import the products in your product set, send a POST request to the following URI.

  • Replace project-id with the id of your Google Cloud Platform (GCP) project.

  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.

POST https://vision.googleapis.com/v1p3beta1/projects/project-id/locations/location-id/productSets:import
{
  "inputConfig": {
    "gcsSource": {
      "csvFileUri": "gs://bucket/path/to/file.csv"
    }
  }
}

A successful response contains a long-running operation object:

{
  "name": "locations/location-id/operations/operation-id"
}

To request the status of the operation, call the operations endpoint and pass the operation-id that was returned from your call to the import method. Replace location-id with the location id from your call to the import method.

GET https://vision.googleapis.com/v1p3beta1/locations/location-id/operations/operation-id

The response looks like:

{
  "name": "locations/us-west1/operations/eea5667a0ed419d8",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.BatchOperationMetadata",
    "state": "SUCCESSFUL",
    "endTime": "2018-04-18T21:45:14.926185506Z"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.vision.v1.ImportProductSetsResponse",
    "referenceImages": [
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_01/referenceImages/17196263426604931544-0",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_01.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_02/referenceImages/17196263426604931544-1",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_02.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_04/referenceImages/17196263426604931544-2",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_04.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_kids_05/referenceImages/17196263426604931544-3",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_kids_05.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_women_05/referenceImages/17196263426604931544-4",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_women_05.jpg"
      },
      {
        "name": "projects/media-demo-service/locations/us-west1/products/160421_women_07/referenceImages/17196263426604931544-5",
        "uri": "gs://cloud-ml-api-e2e-testing/vision/product_search/client-lib/160421_women_07.jpg"
      }
    ],
    "statuses": [
      {},
      {},
      {},
      {},
      {},
      {}
    ]
  }
}

Common import error messages include:

  • The number of columns in the csv line should be 6: If you're not providing a bounding box, the line must end with a comma. For example:

    "gs://example-reference-images/10002-002/10002-002_B.jpg","sample-set","sample-product-123","apparel",,

  • Image not found: Your Google Cloud Storage URI does not resolve to an image file.

  • Access denied: Either your image or your CSV file does not have the correct permissions. Image files must be accessible by the authentication method used for your request. If you are authenticating with an API Key, then your CSV files must be public.

Creating a single reference image

You can add a reference image to an existing product. This then allows you to search for the product by the image.

Command-line

To create a reference image, send a POST request to the following URI.

  • Replace project-id with the id of your Google Cloud Platform (GCP) project.

  • Replace location-id with a valid location identifier. Valid location identifiers are: us-west1, us-east1, europe-west1, and asia-east1.

  • Replace product-id with the id of the product to associate the reference image with.

POST https://vision.googleapis.com/v1p3beta1/projects/project-id/locations/location-id/products/product-id/referenceImages
{
  'uri': 'gs://bucket/path/to/red-shoe.jpg',
  'boundingPolys': [
    {
      'vertices': [
        {
          'x': 44,
          'y': 252
        },
        {
          'x': 44,
          'y': 1351
        },
        {
          'x': 1161,
          'y': 252
        },
        {
          'x': 1161,
          'y': 1351
        }
      ]
    }
  ]
}

The boundingPolys field is optional. If omitted, the service will infer the values of the bounds for the image.

Python

from google.cloud import vision_v1p3beta1 as vision

def create_reference_image(
        project_id, location, product_id, reference_image_id, gcs_uri):
    """Create a reference image.
    Args:
        project_id: Id of the project.
        location: A compute region name.
        product_id: Id of the product.
        reference_image_id: Id of the reference image.
        gcs_uri: Google Cloud Storage path of the input image.
    """
    client = vision.ProductSearchClient()

    # Get the full path of the product.
    product_path = client.product_path(
        project=project_id, location=location, product=product_id)

    # Create a reference image.
    reference_image = vision.types.ReferenceImage(uri=gcs_uri)

    # The response is the reference image with `name` populated.
    image = client.create_reference_image(
        parent=product_path,
        reference_image=reference_image,
        reference_image_id=reference_image_id)

    # Display the reference image information.
    print('Reference image name: {}'.format(image.name))
    print('Reference image uri: {}'.format(image.uri))

Java

/**
 * Create a reference image.
 *
 * @param projectId - Id of the project.
 * @param computeRegion - Region name.
 * @param productId - Id of the product.
 * @param referenceImageId - Id of the image.
 * @param gcsUri - Google Cloud Storage path of the input image.
 * @throws IOException - on I/O errors.
 */
public static void createReferenceImage(
    String projectId,
    String computeRegion,
    String productId,
    String referenceImageId,
    String gcsUri)
    throws IOException {
  ProductSearchClient client = ProductSearchClient.create();

  // Get the full path of the product.
  ProductName productPath = ProductName.of(projectId, computeRegion, productId);

  // Create a reference image.
  ReferenceImage referenceImage = ReferenceImage.newBuilder().setUri(gcsUri).build();
  ReferenceImage image =
      client.createReferenceImage(productPath, referenceImage, referenceImageId);

  // Display the reference image information.
  System.out.println(String.format("Reference image name: %s", image.getName()));
  System.out.println(String.format("Reference image uri: %s", image.getUri()));
}

Node.js

const vision = require('@google-cloud/vision').v1p3beta1;

const client = new vision.ProductSearchClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const projectId = 'Your Google Cloud project Id';
// const location = 'A compute region name';
// const productId = 'Id of the product';
// const referenceImageId = 'Id of the reference image';
// const gcsUri = 'Google Cloud Storage path of the input image';

const formattedParent = client.productPath(projectId, location, productId);

const referenceImage = {
  uri: gcsUri,
};

const request = {
  parent: formattedParent,
  referenceImage: referenceImage,
  referenceImageId: referenceImageId,
};

client
  .createReferenceImage(request)
  .then(responses => {
    const response = responses[0];
    console.log(`response.name: ${response.name}`);
    console.log(`response.uri: ${response.uri}`);
  })
  .catch(err => {
    console.error(err);
  });

Indexing

The Product Search index is updated approximately every 30 minutes. When images are added or deleted, the change won't be reflected in your Product Search responses until the index is next updated.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Vision API
Need help? Visit our support page.