Detect Labels

The Vision API can detect and extract information about entities within an image, across a broad group of categories.

Labels can identify objects, locations, activities, animal species, products, and more.

Labels are returned in English only. The Cloud Translate API can translate English labels into any of a number of other languages.

For example, the image above may return the following list of labels:

Description Score
ferris wheel 0.84832066
amusement park 0.8101249
night 0.8036025
outdoor recreation 0.68825835
fair 0.6566326

Label detection requests

To make a label detection request, make a POST request to:

POST https://vision.googleapis.com/v1/images:annotate?key=YOUR_API_KEY

Specify LABEL_DETECTION as the value of features.type:

{
  "requests": [
    {
      "image": {
        "source": {
          "imageUri": "https://cloud.google.com/vision/docs/images/ferris-wheel.jpg"
        }
      },
      "features": [
        {
          "type": "LABEL_DETECTION"
        }
      ]
    }
  ]
}

Images can be passed in one of three ways: as a base64-encoded string; as a Google Cloud Storage URI; or as a publicly-accessible HTTPS or HTTP URL. See Making requests for more information.

See the AnnotateImageRequest reference documentation for more information on configuring the request body.

Code samples

For samples in a number of programming languages, see:

Label detection responses

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

A LABEL_DETECTION response includes the detected labels, their score, and an opaque label ID:

{
  "responses": [
    {
      "labelAnnotations": [
        {
          "mid": "/m/017rgb",
          "description": "ferris wheel",
          "score": 0.84832066
        },
        {
          "mid": "/m/010jjr",
          "description": "amusement park",
          "score": 0.8101249
        },
        {
          "mid": "/m/01d74z",
          "description": "night",
          "score": 0.8036025
        },
        {
          "mid": "/m/05b0n7k",
          "description": "outdoor recreation",
          "score": 0.68825835
        },
        {
          "mid": "/m/02jf28",
          "description": "fair",
          "score": 0.6566326
        }
      ]
    }
  ]
}

Where:

  • mid, if present, contains a machine-generated identifier (MID) corresponding to the entity's Google Knowledge Graph entry. Note that mid values remain unique across different languages, so you can use these values to tie entities together from different languages. To inspect MID values, refer to the Google Knowledge Graph API documentation.

  • description is the label description.

  • score is the confidence score, which ranges from 0 (no confidence) to 1 (very high confidence).

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Google Cloud Vision API Documentation