Cloud Vision API

Objectives

In this sample, you'll use the Google Cloud Vision API to detect faces in an image. To prove to yourself that the faces were detected correctly, you'll then use that data to draw a box around each face.

Costs

This tutorial uses billable components of Cloud Platform, including:

  • Google Cloud Vision API

Use the Pricing Calculator to generate a cost estimate based on your projected usage. New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Google Cloud Platform project.

    Go to the Manage resources page

  3. Verifica che la fatturazione sia attivata per il tuo progetto.

    scopri come attivare la fatturazione

  4. Enable the Google Cloud Vision API.

    Enable the API

  5. Set up your environment for using Application Default Credentials.
  6. Set up language-specific tasks and tools:

    C#

    Java

    • Install Java.
    • API reference.
    • Download and install the Apache Maven build system. Maven will ensure that the Google API Client Library and the Vision API client libraries are installed when the project is built, because we have included them in the pom.xml.

      <dependency>
        <groupId>com.google.apis</groupId>
        <artifactId>google-api-services-vision</artifactId>
        <version>v1-rev20190412-1.28.0</version>
      </dependency>
      <dependency>
        <groupId>com.google.api-client</groupId>
        <artifactId>google-api-client</artifactId>
        <version>1.29.2</version>
        <exclusions>
          <exclusion> <!-- exclude an old version of Guava -->
            <groupId>com.google.guava</groupId>
            <artifactId>guava-jdk5</artifactId>
          </exclusion>
        </exclusions>
      </dependency>
      <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>25.1-android</version>
      </dependency>
      ...

    Node.js

    • Install the Google Client Library
    • Install node.js.
    • API reference.
    • Install npm and node-canvas. The sample code includes a package.json to install all dependencies using the command: npm install. Note that node-canvas has additional dependencies you may need to install - see the node-canvas installation doc for more information.

      {
        "name": "nodejs-docs-samples-vision",
        "private": true,
        "license": "Apache-2.0",
        "author": "Google Inc.",
        "engines": {
          "node": ">=8"
        },
        "files": [
          "*.js"
        ],
        "scripts": {
          "test": "mocha system-test --timeout 600000"
        },
        "dependencies": {
          "@google-cloud/vision": "^1.1.1",
          "mathjs": "^6.0.0",
          "natural": "^0.6.1",
          "redis": "^2.8.0",
          "yargs": "^13.0.0",
          "canvas": "^2.0.0"
        },
        "devDependencies": {
          "@google-cloud/storage": "^2.0.0",
          "chai": "^4.2.0",
          "mocha": "^6.0.0",
          "uuid": "^3.2.1"
        }
      }
      

    PHP

    Python

    Ruby

Create the service object

To access Google APIs using the official client SDKs, you create a service object based on the API's discovery document, which describes the API to the SDK. You'll need to fetch it from the Vision API's discovery service, using your credentials:

C#

using Google.Cloud.Vision.V1;
using System;
using System.Linq;
var client = ImageAnnotatorClient.Create();

Java

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.vision.v1.Vision;
import com.google.api.services.vision.v1.VisionScopes;
import com.google.api.services.vision.v1.model.AnnotateImageRequest;
import com.google.api.services.vision.v1.model.AnnotateImageResponse;
import com.google.api.services.vision.v1.model.BatchAnnotateImagesRequest;
import com.google.api.services.vision.v1.model.BatchAnnotateImagesResponse;
import com.google.api.services.vision.v1.model.FaceAnnotation;
import com.google.api.services.vision.v1.model.Feature;
import com.google.api.services.vision.v1.model.Image;
import com.google.api.services.vision.v1.model.Vertex;
import com.google.common.collect.ImmutableList;
import java.awt.BasicStroke;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.Polygon;
import java.awt.image.BufferedImage;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.GeneralSecurityException;
import java.util.List;
import javax.imageio.ImageIO;
/**
 * Connects to the Vision API using Application Default Credentials.
 */
public static Vision getVisionService() throws IOException, GeneralSecurityException {
  GoogleCredential credential =
      GoogleCredential.getApplicationDefault().createScoped(VisionScopes.all());
  JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
  return new Vision.Builder(GoogleNetHttpTransport.newTrustedTransport(), jsonFactory, credential)
          .setApplicationName(APPLICATION_NAME)
          .build();
}

Node.js

// By default, the client will authenticate using the service account file
// specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable and use
// the project specified by the GCLOUD_PROJECT environment variable. See
// https://googlecloudplatform.github.io/gcloud-node/#/docs/google-cloud/latest/guides/authentication
const vision = require('@google-cloud/vision');
// Creates a client
const client = new vision.ImageAnnotatorClient();

const fs = require('fs');

PHP

use Google\Cloud\Vision\V1\ImageAnnotatorClient;
$imageAnnotator = new ImageAnnotatorClient();

Python

from google.cloud import vision
from google.cloud.vision import types
from PIL import Image, ImageDraw
client = vision.ImageAnnotatorClient()

Ruby

require "google/cloud/vision"
image_annotator = Google::Cloud::Vision::ImageAnnotator.new

Send a face detection request

To construct a request to the Vision API, first consult the API documentation. In this case, you'll be asking the images resource to annotate your image. A request to this API takes the form of an object with a requests list. Each item in this list contains two bits of information:

  • The base64-encoded image data
  • A list of features you'd like annotated about that image.

For this example, you'll simply request FACE_DETECTION annotation on one image, and return the relevant portion of the response:

C#

var response = client.DetectFaces(Image.FromFile(args[0]));

Java

/**
 * Gets up to {@code maxResults} faces for an image stored at {@code path}.
 */
public List<FaceAnnotation> detectFaces(Path path, int maxResults) throws IOException {
  byte[] data = Files.readAllBytes(path);

  AnnotateImageRequest request =
      new AnnotateImageRequest()
          .setImage(new Image().encodeContent(data))
          .setFeatures(ImmutableList.of(
              new Feature()
                  .setType("FACE_DETECTION")
                  .setMaxResults(maxResults)));
  Vision.Images.Annotate annotate =
      vision.images()
          .annotate(new BatchAnnotateImagesRequest().setRequests(ImmutableList.of(request)));
  // Due to a bug: requests to Vision API containing large images fail when GZipped.
  annotate.setDisableGZipContent(true);

  BatchAnnotateImagesResponse batchResponse = annotate.execute();
  assert batchResponse.getResponses().size() == 1;
  AnnotateImageResponse response = batchResponse.getResponses().get(0);
  if (response.getFaceAnnotations() == null) {
    throw new IOException(
        response.getError() != null
            ? response.getError().getMessage()
            : "Unknown error getting image annotations");
  }
  return response.getFaceAnnotations();
}

Node.js

async function detectFaces(inputFile) {
  // Make a call to the Vision API to detect the faces
  const request = {image: {source: {filename: inputFile}}};
  const results = await client.faceDetection(request);
  const faces = results[0].faceAnnotations;
  const numFaces = faces.length;
  console.log(`Found ${numFaces} face${numFaces === 1 ? '' : 's'}.`);
  return faces;
}

PHP

# annotate the image
// $path = 'path/to/your/image.jpg'
$image = file_get_contents($path);
$response = $imageAnnotator->faceDetection($image);
$faces = $response->getFaceAnnotations();

Python

def detect_face(face_file, max_results=4):
    """Uses the Vision API to detect faces in the given file.

    Args:
        face_file: A file-like object containing an image with faces.

    Returns:
        An array of Face objects with information about the picture.
    """
    client = vision.ImageAnnotatorClient()

    content = face_file.read()
    image = types.Image(content=content)

    return client.face_detection(image=image, max_results=max_results).face_annotations

Ruby

response = image_annotator.face_detection image: path_to_image_file

Process the response

Congratulations - you've detected the faces in your image! The response to our face annotation request includes a bunch of metadata about the detected faces, which include coordinates of a polygon encompassing the face. At this point, though, this is only a list of numbers. Let's use them to confirm that you have, in fact, found the faces in your image. We'll draw polygons onto a copy of the image, using the coordinates returned by the Vision API:

C#

using (var image = System.Drawing.Image.FromFile(args[0]))
using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(image))
{
    var cyanPen = new System.Drawing.Pen(System.Drawing.Color.Cyan, 3);
    foreach (var annotation in response)
    {
        g.DrawPolygon(cyanPen, annotation.BoundingPoly.Vertices.Select(
            (vertex) => new System.Drawing.Point(vertex.X, vertex.Y)).ToArray());
        // ...
    }
    // ...
}

Java

/**
 * Reads image {@code inputPath} and writes {@code outputPath} with {@code faces} outlined.
 */
private static void writeWithFaces(Path inputPath, Path outputPath, List<FaceAnnotation> faces)
    throws IOException {
  BufferedImage img = ImageIO.read(inputPath.toFile());
  annotateWithFaces(img, faces);
  ImageIO.write(img, "jpg", outputPath.toFile());
}

/**
 * Annotates an image {@code img} with a polygon around each face in {@code faces}.
 */
public static void annotateWithFaces(BufferedImage img, List<FaceAnnotation> faces) {
  for (FaceAnnotation face : faces) {
    annotateWithFace(img, face);
  }
}

/**
 * Annotates an image {@code img} with a polygon defined by {@code face}.
 */
private static void annotateWithFace(BufferedImage img, FaceAnnotation face) {
  Graphics2D gfx = img.createGraphics();
  Polygon poly = new Polygon();
  for (Vertex vertex : face.getFdBoundingPoly().getVertices()) {
    poly.addPoint(vertex.getX(), vertex.getY());
  }
  gfx.setStroke(new BasicStroke(5));
  gfx.setColor(new Color(0x00ff00));
  gfx.draw(poly);
}

Node.js

We use the node-canvas library to draw onto images.

async function highlightFaces(inputFile, faces, outputFile, Canvas) {
  const {promisify} = require('util');
  const readFile = promisify(fs.readFile);
  const image = await readFile(inputFile);
  const Image = Canvas.Image;
  // Open the original image into a canvas
  const img = new Image();
  img.src = image;
  const canvas = new Canvas.Canvas(img.width, img.height);
  const context = canvas.getContext('2d');
  context.drawImage(img, 0, 0, img.width, img.height);

  // Now draw boxes around all the faces
  context.strokeStyle = 'rgba(0,255,0,0.8)';
  context.lineWidth = '5';

  faces.forEach(face => {
    context.beginPath();
    let origX = 0;
    let origY = 0;
    face.boundingPoly.vertices.forEach((bounds, i) => {
      if (i === 0) {
        origX = bounds.x;
        origY = bounds.y;
      }
      context.lineTo(bounds.x, bounds.y);
    });
    context.lineTo(origX, origY);
    context.stroke();
  });

  // Write the result to a file
  console.log(`Writing to file ${outputFile}`);
  const writeStream = fs.createWriteStream(outputFile);
  const pngStream = canvas.pngStream();

  await new Promise((resolve, reject) => {
    pngStream
      .on('data', chunk => writeStream.write(chunk))
      .on('error', reject)
      .on('end', resolve);
  });
}

PHP

We use the GD extension to draw onto images.

# draw box around faces
if ($faces && $outFile) {
    $imageCreateFunc = [
        'png' => 'imagecreatefrompng',
        'gd' => 'imagecreatefromgd',
        'gif' => 'imagecreatefromgif',
        'jpg' => 'imagecreatefromjpeg',
        'jpeg' => 'imagecreatefromjpeg',
    ];
    $imageWriteFunc = [
        'png' => 'imagepng',
        'gd' => 'imagegd',
        'gif' => 'imagegif',
        'jpg' => 'imagejpeg',
        'jpeg' => 'imagejpeg',
    ];

    copy($path, $outFile);
    $ext = strtolower(pathinfo($path, PATHINFO_EXTENSION));
    if (!array_key_exists($ext, $imageCreateFunc)) {
        throw new \Exception('Unsupported image extension');
    }
    $outputImage = call_user_func($imageCreateFunc[$ext], $outFile);

    foreach ($faces as $face) {
        $vertices = $face->getBoundingPoly()->getVertices();
        if ($vertices) {
            $x1 = $vertices[0]->getX();
            $y1 = $vertices[0]->getY();
            $x2 = $vertices[2]->getX();
            $y2 = $vertices[2]->getY();
            imagerectangle($outputImage, $x1, $y1, $x2, $y2, 0x00ff00);
        }
    }

Python

def highlight_faces(image, faces, output_filename):
    """Draws a polygon around the faces, then saves to output_filename.

    Args:
      image: a file containing the image with the faces.
      faces: a list of faces found in the file. This should be in the format
          returned by the Vision API.
      output_filename: the name of the image file to be created, where the
          faces have polygons drawn around them.
    """
    im = Image.open(image)
    draw = ImageDraw.Draw(im)
    # Sepecify the font-family and the font-size
    for face in faces:
        box = [(vertex.x, vertex.y)
               for vertex in face.bounding_poly.vertices]
        draw.line(box + [box[0]], width=5, fill='#00ff00')
        # Place the confidence value/score of the detected faces above the
        # detection box in the output image
        draw.text(((face.bounding_poly.vertices)[0].x,
                   (face.bounding_poly.vertices)[0].y - 30),
                  str(format(face.detection_confidence, '.3f')) + '%',
                  fill='#FF0000')
    im.save(output_filename)

Ruby

We use the rmagick gem to draw onto images.

require "rmagick"
  response.responses.each do |res|
    res.face_annotations.each do |annotation|
      puts "Face bounds:"
      annotation.bounding_poly.vertices.each do |vertex|
        puts "(#{vertex.x}, #{vertex.y})"
      end

      x1 = annotation.bounding_poly.vertices[0].x.to_i
      y1 = annotation.bounding_poly.vertices[0].y.to_i
      x2 = annotation.bounding_poly.vertices[2].x.to_i
      y2 = annotation.bounding_poly.vertices[2].y.to_i

      photo = Magick::Image.read(path_to_image_file).first
      draw = Magick::Draw.new
      draw.stroke = "green"
      draw.stroke_width 5
      draw.fill_opacity 0
      draw.rectangle x1, y1, x2, y2
      draw.draw photo

      photo.write path_to_output_file
    end
  end

  puts "Output file: #{path_to_output_file}"

Put it all together

C#

static readonly string s_usage = @"dotnet run image-file

Use the Google Cloud Vision API to detect faces in the image.
Writes an output file called image-file.faces.
";
public static void Main(string[] args)
{
    if (args.Length < 1)
    {
        Console.WriteLine(s_usage);
        return;
    }

    var client = ImageAnnotatorClient.Create();
    var response = client.DetectFaces(Image.FromFile(args[0]));

    int numberOfFacesFound = 0;
    using (var image = System.Drawing.Image.FromFile(args[0]))
    using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(image))
    {
        var cyanPen = new System.Drawing.Pen(System.Drawing.Color.Cyan, 3);
        foreach (var annotation in response)
        {
            g.DrawPolygon(cyanPen, annotation.BoundingPoly.Vertices.Select(
                (vertex) => new System.Drawing.Point(vertex.X, vertex.Y)).ToArray());
            // ...
        }
        // ...
    }
    // ...
}

To build the sample, open the Solution file dotnet-doc-samples/vision/api/Vision.sln in Visual Studio and build the solution.

To run the sample:

C:\...\> cd dotnet-docs-samples\vision\api\DetectFaces\bin\Debug
C:\...\bin\Debug> DetectFaces ..\..\..\VisionTest\data\face.png

Java

/**
 * Annotates an image using the Vision API.
 */
public static void main(String[] args) throws IOException, GeneralSecurityException {
  if (args.length != 2) {
    System.err.println("Usage:");
    System.err.printf(
        "\tjava %s inputImagePath outputImagePath\n",
        FaceDetectApp.class.getCanonicalName());
    System.exit(1);
  }
  Path inputPath = Paths.get(args[0]);
  Path outputPath = Paths.get(args[1]);
  if (!outputPath.toString().toLowerCase().endsWith(".jpg")) {
    System.err.println("outputImagePath must have the file extension 'jpg'.");
    System.exit(1);
  }

  FaceDetectApp app = new FaceDetectApp(getVisionService());
  List<FaceAnnotation> faces = app.detectFaces(inputPath, MAX_RESULTS);
  System.out.printf("Found %d face%s\n", faces.size(), faces.size() == 1 ? "" : "s");
  System.out.printf("Writing to file %s\n", outputPath);
  app.writeWithFaces(inputPath, outputPath, faces);
}
...

To build and run the sample, run the following commands from the sample code directory:

mvn clean compile assembly:single
java -cp target/vision-face-detection-1.0-SNAPSHOT-jar-with-dependencies.jar \
    com.google.cloud.vision.samples.facedetect.FaceDetectApp \
    data/face.jpg \
    output.jpg

Node.js

async function main(inputFile, outputFile) {
  const Canvas = require('canvas');
  outputFile = outputFile || 'out.png';
  const faces = await detectFaces(inputFile);
  console.log('Highlighting...');
  await highlightFaces(inputFile, faces, outputFile, Canvas);
  console.log('Finished!');
}

To run the sample, run the following command from the sample code directory:

node faceDetection resources/face.png

PHP

call_user_func($imageWriteFunc[$ext], $outputImage, $outFile);
printf('Output image written to %s' . PHP_EOL, $outFile);

To run the sample, run the following command from the sample code directory:

composer install
php vision.php face test/data/face.png output-image.png

Python

def main(input_filename, output_filename, max_results):
    with open(input_filename, 'rb') as image:
        faces = detect_face(image, max_results)
        print('Found {} face{}'.format(
            len(faces), '' if len(faces) == 1 else 's'))

        print('Writing to file {}'.format(output_filename))
        # Reset the file pointer, so we can read the file again
        image.seek(0)
        highlight_faces(image, faces, output_filename)

Ruby

if $PROGRAM_NAME == __FILE__
  if ARGV.size == 2
    draw_box_around_faces path_to_image_file:  ARGV.shift,
                          path_to_output_file: ARGV.shift
  else
    puts <<~USAGE
      Usage: ruby draw_box_around_faces.rb [input-file] [output-file]
       Example:
        ruby draw_box_around_faces.rb images/face.png output-image.png
    USAGE
  end
end

To run the sample, run the following command from the sample code directory:

bundle install
bundle exec ruby draw_box_around_faces.rb resources/face_no_surprise.png output-image.png

Input image Output image

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete .
  3. In the dialog, type the project ID, and then click Shut down to delete the project.
Hai trovato utile questa pagina? Facci sapere cosa ne pensi:

Invia feedback per...

Cloud Vision API Documentation
Hai bisogno di assistenza? Visita la nostra pagina di assistenza.