The Vision API can detect and extract information about entities in an image, across a broad group of categories.
Labels can identify general objects, locations, activities, animal species, products, and more. If you need targeted custom labels, Cloud AutoML Vision allows you to train a custom machine learning model to classify images.
Labels are returned in English only. The Cloud Translation API can translate English labels into any of a number of other languages.
For example, the image above may return the following list of labels:
Description | Score |
---|---|
Street | 0.872 |
Snapshot | 0.852 |
Town | 0.848 |
Night | 0.804 |
Alley | 0.713 |
Label detection requests
Set up your Google Cloud project and authentication
Detect Labels in a local image
The Vision API can perform feature detection on a local image file by sending the contents of the image file as a base64 encoded string in the body of your request.
REST
Before using any of the request data, make the following replacements:
- BASE64_ENCODED_IMAGE: The base64
representation (ASCII string) of your binary image data. This string should look similar to the
following string:
/9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
- RESULTS_INT: (Optional) An integer value of results to
return. If you omit the
"maxResults"
field and its value, the API returns the default value of 10 results. This field does not apply to the following feature types:TEXT_DETECTION
,DOCUMENT_TEXT_DETECTION
, orCROP_HINTS
. - PROJECT_ID: Your Google Cloud project ID.
HTTP method and URL:
POST https://vision.googleapis.com/v1/images:annotate
Request JSON body:
{ "requests": [ { "image": { "content": "BASE64_ENCODED_IMAGE" }, "features": [ { "maxResults": RESULTS_INT, "type": "LABEL_DETECTION" } ] } ] }
To send your request, choose one of these options:
curl
Save the request body in a file called request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: PROJECT_ID" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"
PowerShell
Save the request body in a file called request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "PROJECT_ID" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content
If the request is successful, the server returns a 200 OK
HTTP status code and
the response in JSON format.
A LABEL_DETECTION
response includes the detected labels, their score,
topicality, and an opaque label ID, where:
mid
- if present, contains a machine-generated identifier (MID) corresponding to the entity's Google Knowledge Graph entry. Note thatmid
values remain unique across different languages, so you can use these values to tie entities together from different languages. To inspect MID values, refer to the Google Knowledge Graph API documentation.description
- the label description.score
- the confidence score, which ranges from 0 (no confidence) to 1 (very high confidence).topicality
- The relevancy of the ICA (Image Content Annotation) label to the image. It measures how important/central a label is to the overall context of a page.
{ "responses": [ { "labelAnnotations": [ { "mid": "/m/01c8br", "description": "Street", "score": 0.87294734, "topicality": 0.87294734 }, { "mid": "/m/06pg22", "description": "Snapshot", "score": 0.8523099, "topicality": 0.8523099 }, { "mid": "/m/0dx1j", "description": "Town", "score": 0.8481104, "topicality": 0.8481104 }, { "mid": "/m/01d74z", "description": "Night", "score": 0.80408716, "topicality": 0.80408716 }, { "mid": "/m/01lwf0", "description": "Alley", "score": 0.7133322, "topicality": 0.7133322 } ] } ] }
Go
Before trying this sample, follow the Go setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in the Vision API Quickstart Using Client Libraries. For more information, see the Vision API Java reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Node.js API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Python API reference documentation.
Additional languages
C#: Please follow the C# setup instructions on the client libraries page and then visit the Vision reference documentation for .NET.
PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Vision reference documentation for PHP.
Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Vision reference documentation for Ruby.
Detect Labels in a remote image
The Vision API can perform feature detection on a remote image file located in Cloud Storage or on the Web. When you send a remote file request, you specify the file URL and don't need to send the contents of the image file in the request body.
REST
Before using any of the request data, make the following replacements:
- CLOUD_STORAGE_IMAGE_URI: the path to a valid
image file in a Cloud Storage bucket. You must at least have read privileges to the file.
Example:
gs://cloud-samples-data/vision/label/setagaya.jpeg
- RESULTS_INT: (Optional) An integer value of results to
return. If you omit the
"maxResults"
field and its value, the API returns the default value of 10 results. This field does not apply to the following feature types:TEXT_DETECTION
,DOCUMENT_TEXT_DETECTION
, orCROP_HINTS
. - PROJECT_ID: Your Google Cloud project ID.
HTTP method and URL:
POST https://vision.googleapis.com/v1/images:annotate
Request JSON body:
{ "requests": [ { "image": { "source": { "gcsImageUri": "CLOUD_STORAGE_IMAGE_URI" } }, "features": [ { "maxResults": RESULTS_INT, "type": "LABEL_DETECTION" }, ] } ] }
To send your request, choose one of these options:
curl
Save the request body in a file called request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "x-goog-user-project: PROJECT_ID" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://vision.googleapis.com/v1/images:annotate"
PowerShell
Save the request body in a file called request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "PROJECT_ID" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content
If the request is successful, the server returns a 200 OK
HTTP status code and
the response in JSON format.
A LABEL_DETECTION
response includes the detected labels, their score,
topicality, and an opaque label ID, where:
mid
- if present, contains a machine-generated identifier (MID) corresponding to the entity's Google Knowledge Graph entry. Note thatmid
values remain unique across different languages, so you can use these values to tie entities together from different languages. To inspect MID values, refer to the Google Knowledge Graph API documentation.description
- the label description.score
- the confidence score, which ranges from 0 (no confidence) to 1 (very high confidence).topicality
- The relevancy of the Image Content Annotation (ICA) label to the image. It measures how important/central a label is to the overall context of a page.
{ "responses": [ { "labelAnnotations": [ { "mid": "/m/01c8br", "description": "Street", "score": 0.87294734, "topicality": 0.87294734 }, { "mid": "/m/06pg22", "description": "Snapshot", "score": 0.8523099, "topicality": 0.8523099 }, { "mid": "/m/0dx1j", "description": "Town", "score": 0.8481104, "topicality": 0.8481104 }, { "mid": "/m/01d74z", "description": "Night", "score": 0.80408716, "topicality": 0.80408716 }, { "mid": "/m/01lwf0", "description": "Alley", "score": 0.7133322, "topicality": 0.7133322 } ] } ] }
Go
Before trying this sample, follow the Go setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in the Vision API Quickstart Using Client Libraries. For more information, see the Vision API Java reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Node.js API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Python API reference documentation.
gcloud CLI
To detect labels in an image, use the
gcloud ml vision detect-labels
command as shown in the following example:
gcloud ml vision detect-labels gs://cloud-samples-data/vision/label/setagaya.jpeg
Additional languages
C#: Please follow the C# setup instructions on the client libraries page and then visit the Vision reference documentation for .NET.
PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Vision reference documentation for PHP.
Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Vision reference documentation for Ruby.
Try it
Try label detection below. You can use the
image specified already (gs://cloud-samples-data/vision/label/setagaya.jpeg
)
or specify your own image in its place. Send the request by selecting
Execute.
Request body:
{ "requests": [ { "features": [ { "maxResults": 5, "type": "LABEL_DETECTION" } ], "image": { "source": { "imageUri": "gs://cloud-samples-data/vision/label/setagaya.jpeg" } } } ] }