Detect image properties

The Image Properties feature detects general attributes of the image, such as dominant color.

Bali image — *Image credit*: Jeremy Bishop on Unsplash.

Dominant colors detected:

Image property detection requests

Set up your GCP project and authentication

If you have not created a Google Cloud Platform (GCP) project and service account credentials, do so now. Expand this section for instructions.

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
Set up a Cloud Console project.
Set up a project
Click to:
- Create or select a project.
- Enable the Vision API for that project.
- Create a service account.
- Download a private key as JSON.
You can view and manage these resources at any time in the Cloud Console.

Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
Example: Linux or macOS
```
export GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"
```
Replace KEY_PATH with the path of the JSON file that contains your service account key.

For example:
```
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"
```
Example: Windows

For PowerShell:
```
$env:GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"
```
Replace KEY_PATH with the path of the JSON file that contains your service account key.

For example:
```
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\service-account-file.json"
```
For command prompt:
```
set GOOGLE_APPLICATION_CREDENTIALS=KEY_PATH
```
Replace KEY_PATH with the path of the JSON file that contains your service account key.
Install and initialize the Cloud SDK.

Detect Image Properties in a local image

The Vision API can perform feature detection on a local image file by sending the contents of the image file as a base64 encoded string in the body of your request.

The ColorInfo field does not carry information about the absolute color space that should be used to interpret the RGB value (e.g. sRGB, Adobe RGB, DCI-P3, BT.2020, etc.). By default, applications should assume the sRGB color space.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

base64-encoded-image: The base64 representation (ASCII string) of your binary image data. This string should look similar to the following string:
- /9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
Visit the base64 encode topic for more information.

HTTP method and URL:

POST https://vision.googleapis.com/v1/images:annotate

Request JSON body:

{
  "requests": [
    {
      "image": {
        "content": "base64-encoded-image"
      },
      "features": [
        {
          "maxResults": 10,
          "type": "IMAGE_PROPERTIES"
        },
      ]
    }
  ]
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://vision.googleapis.com/v1/images:annotate

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

Response:

Response

{
  "responses": [
    {
      "imagePropertiesAnnotation": {
        "dominantColors": {
          "colors": [
            {
              "color": {
                "red": 243,
                "green": 177,
                "blue": 133
              },
              "score": 0.18074834,
              "pixelFraction": 0.013533333
            },
            {
              "color": {
                "red": 204,
                "green": 205,
                "blue": 213
              },
              "score": 0.092455424,
              "pixelFraction": 0.19266666
            },
            {
              "color": {
                "red": 114,
                "green": 77,
                "blue": 64
              },
              "score": 0.090447456,
              "pixelFraction": 0.034133334
            },
            {
              "color": {
                "red": 224,
                "green": 57,
                "blue": 64
              },
              "score": 0.010952942,
              "pixelFraction": 0.014266667
            },
            {
              "color": {
                "red": 248,
                "green": 125,
                "blue": 130
              },
              "score": 0.006984347,
              "pixelFraction": 0.0057333335
            },
            {
              "color": {
                "red": 150,
                "green": 107,
                "blue": 92
              },
              "score": 0.081589326,
              "pixelFraction": 0.019666666
            },
            {
              "color": {
                "red": 233,
                "green": 185,
                "blue": 158
              },
              "score": 0.08035342,
              "pixelFraction": 0.0122
            },
            {
              "color": {
                "red": 221,
                "green": 221,
                "blue": 226
              },
              "score": 0.045200635,
              "pixelFraction": 0.202
            },
            {
              "color": {
                "red": 105,
                "green": 77,
                "blue": 75
              },
              "score": 0.030223774,
              "pixelFraction": 0.013866667
            },
            {
              "color": {
                "red": 189,
                "green": 145,
                "blue": 123
              },
              "score": 0.028689377,
              "pixelFraction": 0.0069333334
            }
          ]
        }
      },
      "cropHintsAnnotation": {
        "cropHints": [
          {
            "boundingPoly": {
              "vertices": [
                {},
                {
                  "x": 2549
                },
                {
                  "x": 2549,
                  "y": 1699
                },
                {
                  "y": 1699
                }
              ]
            },
            "confidence": 0.79999995,
            "importanceFraction": 1
          }
        ]
      }
    }
  ]
}

Go

Before trying this sample, follow the Go setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Go API reference documentation.

View on GitHub Feedback


// detectProperties gets image properties from the Vision API for an image at the given file path.
func detectProperties(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	f, err := os.Open(file)
	if err != nil {
		return err
	}
	defer f.Close()

	image, err := vision.NewImageFromReader(f)
	if err != nil {
		return err
	}
	props, err := client.DetectImageProperties(ctx, image, nil)
	if err != nil {
		return err
	}

	fmt.Fprintln(w, "Dominant colors:")
	for _, quantized := range props.DominantColors.Colors {
		color := quantized.Color
		r := int(color.Red) & 0xff
		g := int(color.Green) & 0xff
		b := int(color.Blue) & 0xff
		fmt.Fprintf(w, "%2.1f%% - #%02x%02x%02x\n", quantized.PixelFraction*100, r, g, b)
	}

	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Vision API Quickstart Using Client Libraries. For more information, see the Vision API Java API reference documentation.

View on GitHub Feedback


import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.AnnotateImageResponse;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.ColorInfo;
import com.google.cloud.vision.v1.DominantColorsAnnotation;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class DetectProperties {
  public static void detectProperties() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String filePath = "path/to/your/image/file.jpg";
    detectProperties(filePath);
  }

  // Detects image properties such as color frequency from the specified local image.
  public static void detectProperties(String filePath) throws IOException {
    List<AnnotateImageRequest> requests = new ArrayList<>();

    ByteString imgBytes = ByteString.readFrom(new FileInputStream(filePath));

    Image img = Image.newBuilder().setContent(imgBytes).build();
    Feature feat = Feature.newBuilder().setType(Feature.Type.IMAGE_PROPERTIES).build();
    AnnotateImageRequest request =
        AnnotateImageRequest.newBuilder().addFeatures(feat).setImage(img).build();
    requests.add(request);

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
      BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
      List<AnnotateImageResponse> responses = response.getResponsesList();

      for (AnnotateImageResponse res : responses) {
        if (res.hasError()) {
          System.out.format("Error: %s%n", res.getError().getMessage());
          return;
        }

        // For full list of available annotations, see http://g.co/cloud/vision/docs
        DominantColorsAnnotation colors = res.getImagePropertiesAnnotation().getDominantColors();
        for (ColorInfo color : colors.getColorsList()) {
          System.out.format(
              "fraction: %f%nr: %f, g: %f, b: %f%n",
              color.getPixelFraction(),
              color.getColor().getRed(),
              color.getColor().getGreen(),
              color.getColor().getBlue());
        }
      }
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Node.js API reference documentation.

View on GitHub Feedback

const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const fileName = 'Local image file, e.g. /path/to/image.png';

// Performs property detection on the local file
const [result] = await client.imageProperties(fileName);
const colors = result.imagePropertiesAnnotation.dominantColors.colors;
colors.forEach(color => console.log(color));

Python

Before trying this sample, follow the Python setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Python API reference documentation.

View on GitHub Feedback

def detect_properties(path):
    """Detects image properties in the file."""
    from google.cloud import vision
    import io
    client = vision.ImageAnnotatorClient()

    with io.open(path, 'rb') as image_file:
        content = image_file.read()

    image = vision.Image(content=content)

    response = client.image_properties(image=image)
    props = response.image_properties_annotation
    print('Properties:')

    for color in props.dominant_colors.colors:
        print('fraction: {}'.format(color.pixel_fraction))
        print('\tr: {}'.format(color.color.red))
        print('\tg: {}'.format(color.color.green))
        print('\tb: {}'.format(color.color.blue))
        print('\ta: {}'.format(color.color.alpha))

    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))

Additional languages

C#: Please follow the C# setup instructions on the client libraries page and then visit the Vision reference documentation for .NET.

PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Vision reference documentation for PHP.

Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Vision reference documentation for Ruby.

Detect Image Properties in a remote image

For your convenience, the Vision API can perform feature detection directly on an image file located in Google Cloud Storage or on the Web without the need to send the contents of the image file in the body of your request.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

cloud-storage-image-uri: the path to a valid image file in a Cloud Storage bucket. You must at least have read privileges to the file. Example:
- ```
gs://cloud-samples-data/vision/image_properties/bali.jpeg
```

HTTP method and URL:

POST https://vision.googleapis.com/v1/images:annotate

Request JSON body:

{
  "requests": [
    {
      "image": {
        "source": {
          "gcsImageUri": "cloud-storage-image-uri"
        }
      },
      "features": [
        {
          "maxResults": 10,
          "type": "IMAGE_PROPERTIES"
        },
      ]
    }
  ]
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://vision.googleapis.com/v1/images:annotate

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

Response:

Response

{
  "responses": [
    {
      "imagePropertiesAnnotation": {
        "dominantColors": {
          "colors": [
            {
              "color": {
                "red": 243,
                "green": 177,
                "blue": 133
              },
              "score": 0.18074834,
              "pixelFraction": 0.013533333
            },
            {
              "color": {
                "red": 204,
                "green": 205,
                "blue": 213
              },
              "score": 0.092455424,
              "pixelFraction": 0.19266666
            },
            {
              "color": {
                "red": 114,
                "green": 77,
                "blue": 64
              },
              "score": 0.090447456,
              "pixelFraction": 0.034133334
            },
            {
              "color": {
                "red": 224,
                "green": 57,
                "blue": 64
              },
              "score": 0.010952942,
              "pixelFraction": 0.014266667
            },
            {
              "color": {
                "red": 248,
                "green": 125,
                "blue": 130
              },
              "score": 0.006984347,
              "pixelFraction": 0.0057333335
            },
            {
              "color": {
                "red": 150,
                "green": 107,
                "blue": 92
              },
              "score": 0.081589326,
              "pixelFraction": 0.019666666
            },
            {
              "color": {
                "red": 233,
                "green": 185,
                "blue": 158
              },
              "score": 0.08035342,
              "pixelFraction": 0.0122
            },
            {
              "color": {
                "red": 221,
                "green": 221,
                "blue": 226
              },
              "score": 0.045200635,
              "pixelFraction": 0.202
            },
            {
              "color": {
                "red": 105,
                "green": 77,
                "blue": 75
              },
              "score": 0.030223774,
              "pixelFraction": 0.013866667
            },
            {
              "color": {
                "red": 189,
                "green": 145,
                "blue": 123
              },
              "score": 0.028689377,
              "pixelFraction": 0.0069333334
            }
          ]
        }
      },
      "cropHintsAnnotation": {
        "cropHints": [
          {
            "boundingPoly": {
              "vertices": [
                {},
                {
                  "x": 2549
                },
                {
                  "x": 2549,
                  "y": 1699
                },
                {
                  "y": 1699
                }
              ]
            },
            "confidence": 0.79999995,
            "importanceFraction": 1
          }
        ]
      }
    }
  ]
}

Go

Before trying this sample, follow the Go setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Go API reference documentation.

View on GitHub Feedback


// detectProperties gets image properties from the Vision API for an image at the given file path.
func detectPropertiesURI(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	image := vision.NewImageFromURI(file)
	props, err := client.DetectImageProperties(ctx, image, nil)
	if err != nil {
		return err
	}

	fmt.Fprintln(w, "Dominant colors:")
	for _, quantized := range props.DominantColors.Colors {
		color := quantized.Color
		r := int(color.Red) & 0xff
		g := int(color.Green) & 0xff
		b := int(color.Blue) & 0xff
		fmt.Fprintf(w, "%2.1f%% - #%02x%02x%02x\n", quantized.PixelFraction*100, r, g, b)
	}

	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Java API reference documentation.

View on GitHub Feedback


import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.AnnotateImageResponse;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.ColorInfo;
import com.google.cloud.vision.v1.DominantColorsAnnotation;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.ImageSource;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class DetectPropertiesGcs {

  public static void detectPropertiesGcs() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String filePath = "gs://your-gcs-bucket/path/to/image/file.jpg";
    detectPropertiesGcs(filePath);
  }

  // Detects image properties such as color frequency from the specified remote image on Google
  // Cloud Storage.
  public static void detectPropertiesGcs(String gcsPath) throws IOException {
    List<AnnotateImageRequest> requests = new ArrayList<>();

    ImageSource imgSource = ImageSource.newBuilder().setGcsImageUri(gcsPath).build();
    Image img = Image.newBuilder().setSource(imgSource).build();
    Feature feat = Feature.newBuilder().setType(Feature.Type.IMAGE_PROPERTIES).build();
    AnnotateImageRequest request =
        AnnotateImageRequest.newBuilder().addFeatures(feat).setImage(img).build();
    requests.add(request);

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
      BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
      List<AnnotateImageResponse> responses = response.getResponsesList();

      for (AnnotateImageResponse res : responses) {
        if (res.hasError()) {
          System.out.format("Error: %s%n", res.getError().getMessage());
          return;
        }

        // For full list of available annotations, see http://g.co/cloud/vision/docs
        DominantColorsAnnotation colors = res.getImagePropertiesAnnotation().getDominantColors();
        for (ColorInfo color : colors.getColorsList()) {
          System.out.format(
              "fraction: %f%nr: %f, g: %f, b: %f%n",
              color.getPixelFraction(),
              color.getColor().getRed(),
              color.getColor().getGreen(),
              color.getColor().getBlue());
        }
      }
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Node.js API reference documentation.

View on GitHub Feedback

// Imports the Google Cloud client libraries
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// const bucketName = 'Bucket where the file resides, e.g. my-bucket';
// const fileName = 'Path to file within bucket, e.g. path/to/image.png';

// Performs property detection on the gcs file
const [result] = await client.imageProperties(
  `gs://${bucketName}/${fileName}`
);
const colors = result.imagePropertiesAnnotation.dominantColors.colors;
colors.forEach(color => console.log(color));

Python

Before trying this sample, follow the Python setup instructions in the Vision quickstart using client libraries. For more information, see the Vision Python API reference documentation.

View on GitHub Feedback

def detect_properties_uri(uri):
    """Detects image properties in the file located in Google Cloud Storage or
    on the Web."""
    from google.cloud import vision
    client = vision.ImageAnnotatorClient()
    image = vision.Image()
    image.source.image_uri = uri

    response = client.image_properties(image=image)
    props = response.image_properties_annotation
    print('Properties:')

    for color in props.dominant_colors.colors:
        print('frac: {}'.format(color.pixel_fraction))
        print('\tr: {}'.format(color.color.red))
        print('\tg: {}'.format(color.color.green))
        print('\tb: {}'.format(color.color.blue))
        print('\ta: {}'.format(color.color.alpha))

    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))

gcloud

To perform image property detection, use the gcloud ml vision detect-image-properties command as shown in the following example:

gcloud ml vision detect-image-properties gs://cloud-samples-data/vision/image_properties/bali.jpeg

Additional languages

C#: Please follow the C# setup instructions on the client libraries page and then visit the Vision reference documentation for .NET.

PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Vision reference documentation for PHP.

Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Vision reference documentation for Ruby.

Try it

Try image property detection below. You can use the image specified already (gs://cloud-samples-data/vision/image_properties/bali.jpeg) or specify your own image in its place. Send the request by selecting Execute.

Request body:

{
  "requests": [
    {
      "features": [
        {
          "maxResults": 10,
          "type": "IMAGE_PROPERTIES"
        }
      ],
      "image": {
        "source": {
          "imageUri": "gs://cloud-samples-data/vision/image_properties/bali.jpeg"
        }
      }
    }
  ]
}