Detect Labels

The Vision API can detect and extract information about entities within an image, across a broad group of categories.

Labels can identify objects, locations, activities, animal species, products, and more.

Labels are returned in English only. The Cloud Translation API can translate English labels into any of a number of other languages.

For example, the image above may return the following list of labels:

Description Score
ferris wheel 0.84832066
amusement park 0.8101249
night 0.8036025
outdoor recreation 0.68825835
fair 0.6566326

Code samples

For samples in a number of programming languages, see:

Label detection requests

Set up your GCP project and authentication

Detect Labels

Powershell

To make a label detection request using Windows Powershell, make a POST request to the https://vision.googleapis.com/v1/images:annotate endpoint and specify LABEL_DETECTION as the value of features.type, as shown in the following example:

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Post `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -Body "{
      'requests': [
        {
          'image': {
            'source': {
              'imageUri': 'https://cloud.google.com/vision/docs/images/ferris-wheel.jpg'
            }
          },
          'features': [
            {
              'type': 'LABEL_DETECTION'
            }
          ]
        }
      ]
    }" `
  -Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

Images can be passed in one of three ways: as a base64-encoded string; as a Google Cloud Storage URI; or as a publicly-accessible HTTPS or HTTP URL. See Making requests for more information.

See the AnnotateImageRequest reference documentation for more information on configuring the request body.

curl command

To make a label detection request using curl from the Linux or MacOS command line, make a POST request to the https://vision.googleapis.com/v1/images:annotate endpoint and specify LABEL_DETECTION as the value of features.type, as shown in the following example:

curl -X POST \
     -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
     -H "Content-Type: application/json; charset=utf-8" \
     --data "{
      'requests': [
        {
          'image': {
            'source': {
              'imageUri': 'https://cloud.google.com/vision/docs/images/ferris-wheel.jpg'
            }
          },
          'features': [
            {
              'type': 'LABEL_DETECTION'
            }
          ]
        }
      ]
    }" "https://vision.googleapis.com/v1/images:annotate"

Images can be passed in one of three ways: as a base64-encoded string; as a Google Cloud Storage URI; or as a publicly-accessible HTTPS or HTTP URL. See Making requests for more information.

See the AnnotateImageRequest reference documentation for more information on configuring the request body.

GCLOUD COMMAND

To detect labels in an image, use the gcloud ml vision detect-labels command as shown in the following example:

gcloud ml vision detect-labels https://cloud.google.com/vision/docs/images/ferris-wheel.jpg

Label detection responses

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

A LABEL_DETECTION response includes the detected labels, their score, and an opaque label ID:

{
  "responses": [
    {
      "labelAnnotations": [
        {
          "mid": "/m/017rgb",
          "description": "ferris wheel",
          "score": 0.84832066
        },
        {
          "mid": "/m/010jjr",
          "description": "amusement park",
          "score": 0.8101249
        },
        {
          "mid": "/m/01d74z",
          "description": "night",
          "score": 0.8036025
        },
        {
          "mid": "/m/05b0n7k",
          "description": "outdoor recreation",
          "score": 0.68825835
        },
        {
          "mid": "/m/02jf28",
          "description": "fair",
          "score": 0.6566326
        }
      ]
    }
  ]
}

Where:

  • mid, if present, contains a machine-generated identifier (MID) corresponding to the entity's Google Knowledge Graph entry. Note that mid values remain unique across different languages, so you can use these values to tie entities together from different languages. To inspect MID values, refer to the Google Knowledge Graph API documentation.

  • description is the label description.

  • score is the confidence score, which ranges from 0 (no confidence) to 1 (very high confidence).

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Vision API Documentation
Need help? Visit our support page.