The Tag Recognizer model helps you solve key problems in understanding your retail shelf, namely recognizing and parsing the tags (for example, the price tag or other label tags) according to user-defined key-value pair entity extraction schema.
This model can serve as the primary AI building block for analyzing and interpreting product image data in retail stores. For example, you can use this model on shelf images that are captured by local cameras or mobile devices.
Tag Recognizer and Product Recognizer use cases
The Product Recognizer model and the Tag Recognizer models can serve as the primary AI building blocks for analyzing and interpreting the image data around products and tags found in retail stores, such as the shelf scanning images captured by the installed cameras or mobile devices / platforms.
The Product recognizer and Tag recognizer models incorporate several core Google AI modeling and data capabilities to help the retailers and/or technical partners to solve key problems in understanding the retail shelf, including:
- Detecting, recognizing thus understanding what products are in the image or on the shelf.
- Detecting, recognizing, and parsing the tags (price-tag, for example, or any other text label tags) according to user defined key-value pair entity extraction schema.
In particular, several differentiating Google AI models are included in the shelf checking solution to support these use-cases problem solving, such as
- Product detection models (pre-trained by Google, but that you can still customize).
- Product thumbnail visual embedding model, which turns a product thumbnail image into a numerical feature space representation.
- Google OCR model, which extracts all texts visible in the image.
- Google entity extraction model (that you can customize), which turns the raw texts into the user defined key-value pair named entities.
In addition to these Google AI models, the shelf checking solution also leverages Google's large database of product information. The product data in this Product database includes the product's GTIN / UPC identity, product brand, title, and cross-language descriptions, product logo, and imagery with various packaging variations. The Product database with the previously mentioned product thumbnail visual embedding model enables the Product recognizer model to be able to recognize many products immediately.
For example, given a captured shelf image as follows, the shelf checking solution aims to:
- Detect and localize all product item boxes (visible, not severely occluded) in the image, and recognize the product identity of each individual product item box at the GTIN / UPC level.
- Detect and localize all tag boxes (visible) in the image, recognize all text strings in the tag, and then try to parse the text into the user-defined key-value pair entity extraction schema, such as product item description, price value.
The two major AI features to enable these solutions are Product Recognizer model and Tag Recognizer model, which we will provide more details in the following sections. For each of these two APIs that mainly provide the image inference services, there are one or more components in each API that you can customize. We will first describe the inference path of the API use, and then provide some short description of how the involved components you can customize, either through some user configuration or through some model training you carry out.
Tag Recognizer functionality
This model recognizes all text strings in the tag, and then tries to parse the text into the user-defined key-value pair entity extraction schema, such as product item description or price value. It includes the following differentiating Google AI models:
- The Google OCR technology, which extracts all visible text in the image.
The Google entity extraction model which turns the raw text into the user defined key-value pair named entities. Customize this model using Vertex AI. For example, if you mainly care about the product item description, product price value, or sale price, but nothing else, user can define their tag parsing schema as follows:
key: item_description value: string key: regular_price value: number key: sale_price value: number
Tag Parsing Schema
With the customized entity extraction model training, the detected tag item box will then be recognized and parsed in conformance with the user defined schema, for example as follows:
item_description: COLLECTION 18PC GFT BX
regular_price: 1099
sale_price: 999
Example Output JSON Object
{
"imageUri": "gs://test_bucket/test_image.jpg",
"tagRecognitionAnnotations": [
{
"entities": [
{
"confidence": 0.99646133,
"mentionText": "NISSIN TOP RAMEN\n\nBOW CHICKEN\n\n",
"region": {
"xMax": 0.4618055,
"xMin": 0.042725038,
"yMax": 0.45387268,
"yMin": 0.18415153
},
"type":"description"
},
{
"confidence": 0.95828205,
"mentionText": "$3.90\n",
"region": {
"xMax": 0.24819264,
"xMin": 0.04185935,
"yMax": 0.96134734,
"yMin": 0.80382305
},
"type":"unit_price"
},
{
"confidence": 0.60659707,
"mentionText": "$14.99\n",
"region": {
"xMax": 0.9754113,
"xMin": 0.3654699,
"yMax": 0.92825794,
"yMin": 0.40368474
},
"type":"price"
}
]
}
]
}
Environment setup
This section describes how to interact with Store Vision AI RESTful API.
API_ENDPOINT=visionai.googleapis.com
PROJECT_ID=your project ID
All the create methods require specifying the
to-be-created resource name/ID explicitly. You may use a meaningful string
identifier, for example, "product-ABC" or a randomly generated identifier for example,
UUID.
To grant a person role the editor access to use Store Vision API, please run the following iam binding command:
gcloud projects add-iam-policy-binding PROJECT_ID --member='user:USER_ACCOUNT' --role='roles/visionai.editor'
To grant a service account the editor access, please use the below command:
gcloud projects add-iam-policy-binding PROJECT_ID --member='serviceAccount:SERVICE_ACCOUNT' --role='roles/visionai.editor'
Tag Recognizer user journey
- Perform tag detection model customized training by using Vertex AI / AutoML Vision Object Detection feature.
- Perform tag entity parsing model training by using Vertex AI / AutoML Vision Object Detection feature with customized OCR engine.
- Create an Endpoint with desired Tag Recognition config.
- Perform BatchAnalyze with TagRecognition feature. In the backend, the system will identify tags from each input image, analyze the text on each detected tag to produce a structured parsing output. T ## Tag Detection & Entity Parsing model training
You can train the customized Tag Detection model using our existing Vertex AI / AutoML Vision product's image Object Detection model training feature. Though Vertex AI / AutoML Vision Object Detection model training feature provides a fully managed model training experience, it's still your responsibility to prepare a well sampled image dataset with fully labeled object bounding box annotations as the training dataset to feed into the model training console. Google Cloud provides the Vertex AI Data Labeling Service to let you create the data labeling task. Please follow the following Vertex AI data labeling job link for details: /vertex-ai/docs/datasets/data-labeling-job. Provide clear data labeling instructions to the human raters so that they know how to label the tag detection bounding boxes in the images as the training dataset preparation.
To train the Tag Entity Parsing model, you need to prepare a collection of training data, with images and their associated annotations.
- The image is the already cropped tag image.
- In each image, you need to define and provide the key entity field (such as product_title, price, unit_price fields.) they want to detect and recognize / parse, and their associated image bounding box coordinate location in this cropped image view.
- To support the correct recognition / parsing, we also require you to provide the regular expression syntax to characterize each field. This is needed to assist the tag parsing algorithm's training and inference routine.
Tag Entity Parsing Training Example
For example, with the previous tag entity parsing training example, you can provide one line of annotation info in the training data annotation CSV file as follows:
"image_filepath", "product_title", "(x0, y0, x1, y1)", "", "price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}", "unit_price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}"
From the previous example:
- The "product_title" field has its corresponding box image coordinate "(x0, y0, x1, y1)" and the regular expression constraint for this field is none "".
- The "price" field has its corresponding box image coordinate "(x0, y0, x1, y1)" and the regular expression constraint for this field is "\$\d+\.\d{2}", which is indicating we are looking to recognize and parse this field with the $ sign at the beginning of text entry, and a few number digits before the "." and two digits after the ".".
- The "unit_price" field has the same annotation syntax as the "price" field, for example, the box image coordinates "(x0, y0, x1, y1)" and the regular expression constraint for this field is "\$\d+\.\d{2}", which is indicating we are looking to recognize and parse this field with the $ sign at the beginning of text entry, and a few number digits before the "." and two digits after the ".".
Hence, a proper price-tag parsing / entity detection model training data will have a collection of price-tag images, with the annotation in a CSV file with each CSV row entry just like the entry in the previous example.
"image_filepath", "product_title", "(x0, y0, x1, y1)", "", "price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}", "unit_price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}"
"image_filepath", "product_title", "(x0, y0, x1, y1)", "", "price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}", "unit_price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}"
"image_filepath", "product_title", "(x0, y0, x1, y1)", "", "price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}", "unit_price", "(x0, y0, x1, y1)", "\\$\\d+\\.\\d{2}"
[...]
You can train the customized Tag Entity Parsing model using our existing Vertex AI /AutoML Vision product's Image Object Detection model training feature, plus the Google OCR engine customization.
Please note that though, as of 07/2022, customized Tag Detection and Tag Entity Parsing model training and deployment to Store Vision AI's BatchAnalyze API is not yet fully supported as an integrated console experience, you're still able to take advantage of this customized Tag Detection and Tag Entity Parsing model training (using Vertex AI Vision's Object Detection feature) and serving them in Store Vision AI's BatchAnalyze API by taking several manual self-serving steps.
API usage - batch analyze inference
Create Endpoint
- ENDPOINT_ID=Your endpoint id
- TAG_DETECTOR=Your tag detection model name
- TAG_PARSER=Your tag parsing model name
curl -sS -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" https://visionai.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/us-central1/retailEndpoints?retail_endpoint_id=ENDPOINT_ID \
-d '{
"tagRecognitionConfig": {
"tag_detection_model": "TAG_DETECTOR_NAME",
"tag_parsing_model": "TAG_PARSER_NAME"
}
}'
- INPUT_FILE_URI=Cloud Storage uri of your input file, each line in the input file is just a Cloud Storage uri of image to process, for example, gs://my-bucket/my-image.jpg
- OUTPUT_URI_PREFIX=Cloud Storage uri prefix for output results file, for example, gs://my-bucket/my-output-dir
curl -sS -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" https://visionai.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/us-central1/retailEndpoints/ENDPOINT_ID:batchAnalyze
-d '{
"gcsSource": {
"uris": ["INPUT_FILE_URI"]
},
"features": [
{
"type": "TYPE_TAG_RECOGNITION",
}
],
"outputGcsDestination": {
"outputUriPrefix": "OUTPUT_URI_PREFIX"
}
}'
"features": [
{
"type": "TYPE_TAG_RECOGNITION",
"tagRecognitionConfig": {
"tag_detection_model": "'TAG_DETECTOR_NAME'",
"tag_parsing_model": "TAG_PARSER_NAME"
}
}
],
There are also more fields that you can set and configure in the
tagRecognitionConfig, which is a RetailTagRecognitionConfig object. Please
see the resource description in the API reference for more details.
API reference
Resource: projects.locations.retailCatalogs
JSON Representation
{
"name": string,
"displayName": string,
"createTime": string,
"updateTime": string,
"resourceState": enum(RetailResourceState),
"labels": {
string: string,
...
}
}
Fields
name |
String |
Output only. Resource name of the RetailCatalog |
|---|---|---|
displayName |
String |
Optional. Display name of the RetailCatalog. |
createTime |
string (Timestamp format) |
Output only. Timestamp when this RetailCatalog was created. |
updateTime |
string (Timestamp format) |
Output only. The update timestamp. |
resourceState |
enum |
Output only. State of the RetailCatalog. |
labels |
map (key: string, value: string) |
The labels with user-defined metadata to organize your RetailCatalog. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. |
Method: projects.locations.retailCatalogs.create
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/us-central1}/retailCatalogs
Path parameters
| parent | string | Required. Parent identifier. |
Request body
The request body contains an instance of RetailCatalog.
Response body
If successful, the response body contains a newly created instance of
RetailCatalog.
Method: projects.locations.retailCatalogs.get
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/us-central1/retailCatalogs/*}
Path parameters
| name | string | Required. RetailCatalog identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of RetailCatalog.
Method: projects.locations.retailCatalogs.list
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/us-central1}/retailCatalogs
Path parameters
| parent | string | Required. Parent identifier. |
Query parameters
| filter | string | Optional. An expression for filtering the results of the request. |
|---|---|---|
| pageToken | string | Optional. A token identifying a page of results the server should return. |
| pageSize | integer | Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. |
| orderBy | string | Optional. A comma-separated list of fields to order by sorted in ascending order. Use "desc" after a field name for descending. |
Request body
The request body must be empty.
Response body
If successful, the response body contains data with the following structure: JSON representation
{
"retailCatalogs": [
{
object (RetailCatalog)
}
],
"nextPageToken": string
}
Method: projects.locations.retailCatalogs.delete
HTTP request
DELETE https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/us-central1/retailCatalogs/*}
Path parameters
| name | string | Required. RetailCatalog identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of Operation.
Method: projects.locations.retailCatalogs.importRetailProducts
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/us-central1/retailCatalogs/*}:importRetailProducts
Path parameters
| name | string | Required. RetailCatalog resource name. |
Request body
JSON representation
{
"gcsSource": { object(GcsSource) },
"format": enum(Format)
}
Fields
| gcsSource | object | Required. The Cloud Storage location for the input content.
Multiple input locations can be provided. The contents of all input
locations will be imported in one batch. Supported file extensions: 1.
A JSONL file. Each line is a JSON format of RetailProductIoFormat. 2. A TXT file. Each line is the gtin of a Product to be imported. |
| format | enum | Required. The import file format. |
Format ENUM values
| FORMAT_UNSPECIFIED | Should not be used. |
|---|---|
| FORMAT_TXT | TXT format. |
| FORMAT_JSONL | JSONL format. |
Response body
If successful, the response body contains an instance of Operation.
Resource: projects.locations.retailProducts
JSON Representation
{
"name": string,
"gtins": [string],
"normalizedGtins": [string],
"thirdPartyIds": [ { object(ThirdPartyId) }],
"locale": string,
"brand": string,
"title": string,
"productUri": string,
"resourceState": enum(RetailResourceState),
"labels": {
string: string,
...
}
"createTime": string,
"updateTime": string
}
Fields
name |
String |
Output only. Resource name of the RetailProductImage |
|---|---|---|
displayName |
String |
Optional. Display name of the RetailProductImage. |
sourceType |
enum |
Optional. Source type |
gcsUri |
string |
Optional. Cloud Storage location of the RetailProductImage. It should be set except when the image is provided by Google, for example, when the source type is SOURCE_TYPE_GOOGLE. |
resourceState |
enum |
Output only. State of the RetailProductImage. |
labels |
map (key: string, value: string) |
The labels with user-defined metadata to organize your RetailProductImage. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. |
createTime |
string (Timestamp format) |
Output only. The create Timestamp. |
updateTime |
string (Timestamp format) |
Output only. The update timestamp. |
RetailThirdPartyId JSON Presentation
{
"id": string,
"owner": string
}
Fields
| id | string | Third party id used by the retailer or manufacturer (for example,, SKU or MPN). |
| owner | string | The entity that 'owns' the third party identifier, for example, the manufacturer or the retailer selling this product. |
Method: projects.locations.retailCatalogs.retailProducts.create
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*}/retailProducts
Path parameters
| parent | string | Required. Parent identifier. |
Request body
The request body contains an instance of RetailProduct.
Response body
If successful, the response body contains a newly created instance of
RetailProduct.
Method: projects.locations.retailCatalogs.retailProducts.get
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProducts/*
Path parameters
| name | string | Required. RetailProduct identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of RetailProduct.
Method: projects.locations.retailCatalogs.retailProducts.list
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*}/retailProducts
Path parameters
| parent | string | Required. Parent identifier. |
Query parameters
| filter | string | Optional. An expression for filtering the results of the request. |
|---|---|---|
| pageToken | string | Optional. A token identifying a page of results the server should return. |
| pageSize | integer | Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. |
| orderBy | string | Optional. A comma-separated list of fields to order by sorted in ascending order. Use "desc" after a field name for descending. |
Request body
The request body must be empty.
Response body
If successful, the response body contains data with the following structure: JSON representation
{
"retailProducts": [
{
object (RetailProducts)
}
],
"nextPageToken": string
}
Method: projects.locations.retailCatalogs.retailProducts.delete
HTTP request
DELETE https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProducts/*
Path parameters
| name | string | Required. RetailProduct identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body is empty.
Resource: projects.locations.retailProductImages
JSON Representation
{
"name": string,
"displayName": string,
"sourceType": enum(SourceType),
"gcsUri": string,
"resourceState": enum(RetailResourceState),
"labels": {
string: string,
...
}
"createTime": string,
"updateTime": string
}
Fields
name |
String |
Output only. Resource name of the RetailProductImage |
|---|---|---|
displayName |
String |
Optional. Display name of the RetailProductImage. |
sourceType |
enum |
Optional. Source type |
gcsUri |
string |
Optional. Cloud Storage location of the RetailProductImage. It should be set except when the image is provided by Google, for example, when the source type is SOURCE_TYPE_GOOGLE. |
resourceState |
enum |
Output only. State of the RetailProductImage. |
labels |
map (key: string, value: string) |
The labels with user-defined metadata to organize your RetailProductImage. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. |
createTime |
string (Timestamp format) |
Output only. The create Timestamp. |
updateTime |
string (Timestamp format) |
Output only. The update timestamp. |
SourceType ENUM values
| SOURCE_TYPE_UNSPECIFIED | Unknown data source. Should not be used. |
|---|---|
| SOURCE_TYPE_FIXED_CAMERA | Image is captured from fixed camera. |
| SOURCE_TYPE_HAND_HELD_CAMERA | Image is captured from hand-held camera. |
| SOURCE_TYPE_CRAWLED | Image is crawled from the web. |
| SOURCE_TYPE_SYSTEM_GENERATED | Image is cropped from an original image with human labeling. |
Method: projects.locations.retailCatalogs.retailProducts.retailProductImages.create
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*/retailProducts/*}/retailProductImages
Path parameters
| parent | string | Required. Parent identifier. |
Request body
The request body contains an instance of RetailProductImage.
Response body
If successful, the response body contains a newly created instance of
RetailProductImage.
Method: projects.locations.retailCatalogs.retailProducts.retailProductImages.get
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProducts/*/retailProductImages/*
Path parameters
| name | string | Required. RetailProductImage identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of RetailProductImage.
Method: projects.locations.retailCatalogs.retailProducts.retailProductImages.list
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*/retailProducts/*}/retailProductImages
Path parameters
| parent | string | Required. Parent identifier. |
Query parameters
| filter | string | Optional. An expression for filtering the results of the request. |
|---|---|---|
| pageToken | string | Optional. A token identifying a page of results the server should return. |
| pageSize | integer | Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. |
| orderBy | string | Optional. A comma-separated list of fields to order by sorted in ascending order. Use "desc" after a field name for descending. |
Request body
The request body must be empty.
Response body
If successful, the response body contains data with the following structure:
JSON representation
{
"retailProductImages": [
{
object (RetailProductImages)
}
],
"nextPageToken": string
}
Method: projects.locations.retailCatalogs.retailProducts.retailProductImages.delete
HTTP request
DELETE https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProducts/*/retailProductImages/*
Path parameters
| name | string | Required. RetailProductImage identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body is empty.
Resource: projects.locations.retailCatalogs.retailProductSets
JSON representation
{
"name": string,
"displayName": string,
"retailProductIds": [string],
"resourceState": enum(RetailResourceState),
"labels": {
string: string,
...
}
"createTime": string,
"updateTime": string
}
Fields
name |
String |
Output only. Resource name of the RetailProductSet |
|---|---|---|
displayName |
String |
Optional. Display name of the RetailProductSet. |
retailProductIds [] |
String |
Output only. Resource ids of products belonging to this RetailProductSet. The products in a RetailProductSet should be in the same catalog. |
resourceState |
enum |
Output only. State of the RetailProductSet. |
labels |
map (key: string, value: string) |
The labels with user-defined metadata to organize your RetailProductSet. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. |
createTime |
string (Timestamp format) |
Output only. The create Timestamp. |
updateTime |
string (Timestamp format) |
Output only. The update timestamp. |
Method: projects.locations.retailCatalogs.retailProductSets.create
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*}/retailProductSets
Path parameters
| parent | string | Required. Parent identifier. |
Request body
The request body contains an instance of RetailProductSet.
Response body
If successful, the response body contains a newly created instance of RetailProductSet.
Method: projects.locations.retailCatalogs.retailProductSets.get
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProductSets/*
Path parameters
| name | string | Required. RetailProductSet identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of RetailProductSet.
Method: projects.locations.retailCatalogs.retailProductSets.list
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*}/retailProductSets
Path parameters
| parent | string | Required. Parent identifier. |
Query parameters
| filter | string | Optional. An expression for filtering the results of the request. |
|---|---|---|
| pageToken | string | Optional. A token identifying a page of results the server should return. |
| pageSize | integer | Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. |
| orderBy | string | Optional. A comma-separated list of fields to order by sorted in ascending order. Use "desc" after a field name for descending. |
Request body
The request body must be empty.
Response body
If successful, the response body contains data with the following structure:
JSON representation
{
"retailProductSets": [
{
object (RetailProductSets)
}
],
"nextPageToken": string
}
Method: projects.locations.retailCatalogs.retailProductSets.delete
HTTP request
DELETE https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProductSets/*
Path parameters
| name | string | Required. RetailProductSet identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body is empty.
Method: projects.locations.retailCatalogs.retailProductSets.add
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProductSets/*}:remove
Path parameters
| name | string | Required. RetailProductSet resource name. |
Request body
JSON representation
{
"productIds": [string],
"productFilter": string
}
Fields
productIds[ ] |
string |
Resource id of the RetailProducts to be added. They must all belong to the same RetailCatalog as the specified destination RetailProductSet. Up to 200 RetailProducts ids can be specified in one request. Can not be used together with productFilter. |
|---|---|---|
productFilter |
string |
A standard filter that will be applied to all RetailProducts in the parent RetailCatalog, select items which satisfy filter conditions and add them into the RetailProductSet. Cannot be used together with product_ids. Supported filters: https://google.aip.dev/160 |
Response body
If successful, the response body contains an instance of Operation.
Method: projects.locations.retailCatalogs.retailProductSets.remove
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProductSets/*}:add
Path parameters
| name | string | Required. RetailProductSet resource name. |
Request body
JSON representation
{
"productIds": [string],
"productFilter": string
}
Fields
productIds[ ] |
string |
Resource ids of the RetailProducts to be removed. If the specified RetailProducts does not belong to this RetailProductSet, it will be ignored. Up to 200 RetailProducts ids can be specified in one request. Can not be used together with products_filter. |
|---|---|---|
productFilter |
string |
A standard filter that will be applied to all RetailProducts in the specified RetailProductSet, select items which satisfy filter conditions and remove them from the RetailProductSet. Cannot be used together with product_ids. Supported filters:https://google.aip.dev/160 |
Response body
If successful, the response body contains an instance of Operation.
Resource: projects.locations.retailCatalogs.retailProductRecognitionIndexes
JSON representation
{
"name": string,
"displayName": string,
"description": string,
"retailProductSet": [string],
"resourceState": enum(RetailResourceState),
"labels": {
string: string,
...
}
"createTime": string,
"updateTime": string
}
Fields
name |
String |
Output only. Resource name of the RetailProductRecognitionIndex resource. |
|---|---|---|
displayName |
String |
Optional. Display name of the RetailProductRecognitionIndex. |
description |
String |
Optional. The description of the RetailProductRecognitionIndex. |
retailProductSet[] |
string |
Optional. The resource name of RetailProductSet to use for creating this resource. If set, the RetailProductRecognitionIndex will only contain products in the given RetailProductSet. If not set, all products in the parent catalog will be used. |
resourceState |
enum |
Output only. State of the RetailProductRecognitionIndex. |
labels |
map (key: string, value: string) |
The labels with user-defined metadata to organize your RetailProductRecognitionIndex. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. |
createTime |
string (Timestamp format) |
Output only. The create Timestamp. |
updateTime |
string (Timestamp format) |
Output only. The update timestamp. |
Method: projects.locations.retailCatalogs.retailProductRecognitionIndexes.create
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*}/retailProductRecognitionIndexes
Path parameters
| parent | string | Required. Parent identifier. |
Request body
The request body contains an instance of RetailProductRecognitionIndex.
Response body
If successful, the response body contains a newly created instance of
RetailProductRecognitionIndex.
Method: projects.locations.retailCatalogs.retailProductRecognitionIndexes.get
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProductRecognitionIndexes/*
Path parameters
| name | string | Required. RetailProductRecognitionIndex identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of
RetailProductRecognitionIndex.
Method: projects.locations.retailCatalogs.retailProductRecognitionIndexes.list
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*/retailCatalogs/*}/retailProductRecognitionIndexes
Path parameters
| parent | string | Required. Parent identifier. |
Query parameters
| filter | string | Optional. An expression for filtering the results of the request. |
|---|---|---|
| pageToken | string | Optional. A token identifying a page of results the server should return. |
| pageSize | integer | Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. |
| orderBy | string | Optional. A comma-separated list of fields to order by sorted in ascending order. Use "desc" after a field name for descending. |
Request body
The request body must be empty.
Response body
If successful, the response body contains data with the following structure:
JSON representation
{
"retailProductRecognitionIndexes": [
{
object (RetailProductRecognitionIndex)
}
],
"nextPageToken": string
}
Method: projects.locations.retailCatalogs.retailProductRecognitionIndexes.delete
HTTP request
DELETE https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailCatalogs/*/retailProductRecognitionIndexes/*
Path parameters
| name | string | Required. ProductRecognitionIndex identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of Operation.
Resource: projects.locations.retailEndpoints
JSON representation
{
"name": string,
"displayName": string,
"description": string,
"deployedProductRecognitionIndex": string,
"resourceState": enum(RetailResourceState),
"productRecognitionConfig": { object(RetailProductRecognitionConfig) },
"tagRecognitionConfig": { object(RetailTagRecognitionConfig) },
"labels": {
string: string,
...
}
"createTime": string,
"updateTime": string
}
Fields
name |
String |
Output only. Resource name of the RetailEndpoint resource. |
|---|---|---|
displayName |
String |
Optional. Display name of the RetailEndpoint. |
description |
String |
Optional. The description of the RetailEndpoint. |
deployedProductRecognitionIndex |
String |
Output only. Resource name of the ProductRecognitionIndex deployed to this RetailEndpoint. |
productRecognitionConfig |
object |
Optional. Configuration for product recognition. |
tagRecognitionConfig |
object |
Optional. Configuration for tag recognition. |
resourceState |
enum |
Output only. State of the RetailProductRecognitionIndex. |
labels |
map (key: string, value: string) |
The labels with user-defined metadata to organize your RetailProductRecognitionIndex. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. |
createTime |
string (Timestamp format) |
Output only. The create Timestamp. |
updateTime |
string (Timestamp format) |
Output only. The update timestamp. |
RetailProductRecognitionConfig
JSON representation
{
"productDetectionModel": string,
"detectionConfidenceThreshold": float,
"recognitionConfidenceThreshold": float,
"additionalConfig": { object }
}
Fields
|
| productDetectionModel | string | Required. Model to use to detect products in input images. Supported values: "builtin/stable" (the default) or Vertex AI model resource name. |
|---|---|---|
| detectionConfidenceThreshold | float | Optional. Confidence threshold to filter detection results. If not set, a system default value will be used. |
| recognitionConfidenceThreshold | float | Optional. Confidence threshold to filter recognition results. If not set, a system default value will be used. |
| additionalConfig | object (Struct format) | Optional. Additional configurations for product recognition. |
RetailTagRecognitionConfig
JSON representation
{
"tagDetectionModel": string,
"tagParsingModel": string,
"detectionConfidenceThreshold": float,
"parsingConfidenceThreshold": float,
"additionalConfig": { object }
}
Fields
| tagDetectionModel | string | Required. Model to use to detect tags in input images. Supported values: Vertex AI model resource. |
|---|---|---|
| tagParsingModel | string | Required. Model to parse text on detected tags. Supported values: Vertex AI model resource. |
| detectionConfidenceThreshold | float | Optional. Confidence threshold to filter detection results. If not set, a system default value will be used. |
| parsingConfidenceThreshold | float | Optional. Confidence threshold to filter text parsing results. If not set, a system default value will be used. |
| additionalConfig | object (Struct format) | Optional. Additional configurations for tag recognition. |
Method: projects.locations.retailEndpoints.create
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*}/retailEndpoints
Path parameters
| parent | string | Required. Parent identifier. |
Request body
The request body contains an instance of RetailEndpoint.
Response body
If successful, the response body contains a newly created instance of
RetailEndpoint.
Method: projects.locations.retailEndpoints.get
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailEndpoints/*}
Path parameters
| name | string | Required. RetailEndpoint identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of RetailEndpoint.
Method: projects.locations.retailEndpoints.list
HTTP request
GET https://visionai.googleapis.com/v1alpha1/{parent=projects/*/locations/*}/retailEndpoints
Path parameters
| parent | string | Required. Parent identifier. |
Query parameters
| filter | string | Optional. An expression for filtering the results of the request. |
|---|---|---|
| pageToken | string | Optional. A token identifying a page of results the server should return. |
| pageSize | integer | Optional. Requested page size. Server may return fewer items than requested. If unspecified, server will pick an appropriate default. |
| orderBy | string | Optional. A comma-separated list of fields to order by sorted in ascending order. Use "desc" after a field name for descending. |
Request body
The request body must be empty.
Response body
If successful, the response body contains data with the following structure:
JSON representation
{
"retailEndpoints": [
{
object (RetailEndpoint)
}
],
"nextPageToken": string
}
Method: projects.locations.retailEndpoints.delete
HTTP request
DELETE https://visionai.googleapis.com/v1alpha1/{name=projects/*/locations/*/retailEndpoints/*
Path parameters
| name | string | Required. RetailEndpoint identifier. |
Request body
The request body must be empty.
Response body
If successful, the response body is empty.
Method: projects.locations.retailEndpoints.deployRetailProductRecognitionIndex
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{retailEndpoint=projects/*/locations/*/retailEndpoints/*}:deployRetailProductRecognitionIndex
Path parameters
| retailEndpoint | string | Required. Resource name of the RetailEndpoint
resource into where the
RetailProductRecognitionIndex is deployed. |
Request body
JSON representation
{
"retailProductRecognitionIndex": string,
}
Fields
| retailProductRecognitionIndex | string | Required. The resource name of
RetailProductRecognitionIndex
to deploy. |
Response body
If successful, the response body contains an instance of Operation.
Method: projects.locations.retailEndpoints.undeployRetailProductRecognitionIndex
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{retailEndpoint=projects/*/locations/*/retailEndpoints/*}:undeployRetailProductRecognitionIndex
Path parameters
| retailEndpoint | string | Required. Resource name of the RetailEndpoint
resource on which the undeployment will act. |
Request body
The request body must be empty.
Response body
If successful, the response body contains an instance of Operation.
Method: projects.locations.retailEndpoints.batchAnalyze
HTTP request
POST https://visionai.googleapis.com/v1alpha1/{retailEndpoint=projects/*/locations/*/retailEndpoints/*}:batchAnalyze
Path parameters
| retailEndpoint | string | Required. Resource name of the RetailEndpoint
to serve the inference request. |
Request body
JSON representation
{
"gcsSource": string,
"features": { object(Feature) },
// Union field output can be only one of the following:
"outputGcsDestination": string,
"corpus": string,
// End of list of possible types for union field output.
"bigqueryTable": string
}
Fields
| gcsSource | string | Required. The Cloud Storage location for the input content. Multiple input locations can be provided. The contents of all input locations will be processed in one batch. Supported content: A TXT file, each line is the full path to an image. A maximum of 50k images can be supported in one request. |
|---|---|---|
| outputGcsDestination | string | Optional. The Cloud Storage location of the directory where the output is to be written to. |
| corpus | string | Optional. Resource name of image warehouse corpus. Not yet supported. |
| bigqueryTable | string | Optional. Resource name of the bigquery table for annotation exports. In the format of "projects/*/datasets/*/tables/*". If set, annotations generated from ML inference will also be exported to the given bigquery table. Not yet supported. |
| features[] | Object | Required. The type of ML inference to perform. |
Feature
JSON representation
{
"type": enum(Type),
"productRecognitionConfig": object(RetailProductRecognitionConfig),
"tagRecognitionConfig": object(RetailTagRecognitionConfig)
}
Fields
| type | enum | Required. The Feature Type. |
|---|---|---|
| productRecognitionConfig | object | Optional. Per request overrides for product recognition feature. It's effective only if type is set to TYPE_PRODUCT_RECOGNITION. |
| tagRecognitionConfig | object | Optional. Per request overrides for tag recognition feature. It's effective only if type is set to TYPE_TAG_RECOGNITION. |
Response body
If successful, the response body contains an instance of Operation.
Types
GcsSource
JSON representation
{
"uris": [string]
}
Fields
| uris[] | string | Required. References to a Cloud Storage paths. |
Type
ENUM values
| TYPE_UNSPECIFIED | The default value. Should not be used. |
|---|---|
| TYPE_PRODUCT_RECOGNITION | Product Recognition. Must be used on a
RetailEndpoint with deployed
RetailProductRecognitionIndex. |
| TYPE_TAG_RECOGNITION | Tag Detection and Parsing. Must be used on a
RetailEndpoint with
RetailTagRecognitionConfig. |
RetailProductIoFormat
JSON representation
{
"retailProduct": { object(RetailProduct) },
"retailProductImages": [ { object(RetailProductImage) }]
}
Fields
| retailProduct | object | Required. RetailProduct to be imported |
|---|---|---|
| retailProductImages[ ] | object | Optional. RetailProductImages of the
given RetailProduct to be imported. |
RetailResourceState
ENUM values
| RETAIL_RESOURCE_STATE_UNSPECIFIED | The default value. Should not be used. |
|---|---|
| RETAIL_RESOURCE_STATE_CREATING | State Creating. |
| RETAIL_RESOURCE_STATE_CREATED | State Created. |
| RETAIL_RESOURCE_STATE_UPDATING | State Updating. |
| RETAIL_RESOURCE_STATE_DELETED | State Deleted. |
| RETAIL_RESOURCE_STATE_ERROR | State Error. |