Face Detection Tutorial

Objectives

In this sample, you'll use the Google Cloud Vision API to detect faces in an image. To prove to yourself that the faces were detected correctly, you'll then use that data to draw a box around each face.

Costs

This tutorial uses billable components of Cloud Platform, including:

  • Google Cloud Vision API

Use the Pricing Calculator to generate a cost estimate based on your projected usage. New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Cloud Platform project.

    Go to the Projects page

  3. Enable billing for your project.

    Enable billing

  4. Enable the Google Cloud Vision API.

    Enable the API

  5. Set up your environment for using Application Default Credentials.
  6. Set up language-specific tasks and tools:

    C#

    Node.js

    • Install the Google Client Library
    • Install node.js.
    • API reference.
    • Install npm and node-canvas. The sample code includes a package.json to install all dependencies using the command: npm install. Note that node-canvas has additional dependencies you may need to install - see the node-canvas installation doc for more information.

      {
        "name": "nodejs-docs-samples-vision",
        "version": "0.0.1",
        "private": true,
        "license": "Apache-2.0",
        "author": "Google Inc.",
        "repository": {
          "type": "git",
          "url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
        },
        "engines": {
          "node": ">=4.3.2"
        },
        "scripts": {
          "lint": "samples lint",
          "pretest": "npm run lint",
          "system-test": "ava -T 1m --verbose system-test/*.test.js",
          "test": "npm run system-test"
        },
        "dependencies": {
          "@google-cloud/storage": "1.1.0",
          "@google-cloud/vision": "0.11.2",
          "async": "2.3.0",
          "natural": "0.5.1",
          "redis": "2.7.1",
          "yargs": "7.1.0"
        },
        "devDependencies": {
          "@google-cloud/nodejs-repo-tools": "1.4.14",
          "ava": "0.19.1",
          "proxyquire": "1.7.11",
          "sinon": "2.1.0"
        },
        "optionalDependencies": {
          "canvas": "1.6.5"
        },
        "cloud-repo-tools": {
          "requiresKeyFile": true,
          "requiresProjectId": true
        }
      }
      

    PHP

    Ruby

Create the service object

To access Google APIs using the official client SDKs, you create a service object based on the API's discovery document, which describes the API to the SDK. You'll need to fetch it from the Vision API's discovery service, using your credentials:

C#

var client = ImageAnnotatorClient.Create();
...

Node.js

// By default, the client will authenticate using the service account file
// specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable and use
// the project specified by the GCLOUD_PROJECT environment variable. See
// https://googlecloudplatform.github.io/gcloud-node/#/docs/google-cloud/latest/guides/authentication
var Vision = require('@google-cloud/vision');

// Instantiate a vision client
var vision = Vision();
...

PHP

use Google\Cloud\Vision\VisionClient;

// $projectId = 'YOUR_PROJECT_ID';
// $path = 'path/to/your/image.jpg'

$vision = new VisionClient([
    'projectId' => $projectId,
]);
...

Ruby

require "google/cloud/vision"
...
vision = Google::Cloud::Vision.new project: project_id
...

Send a face detection request

To construct a request to the Vision API, first consult the API documentation. In this case, you'll be asking the images resource to annotate your image. A request to this API takes the form of an object with a requests list. Each item in this list contains two bits of information:

  • The base64-encoded image data
  • A list of features you'd like annotated about that image.

For this example, you'll simply request FACE_DETECTION annotation on one image, and return the relevant portion of the response:

C#

var response = client.DetectFaces(Image.FromFile(args[0]));
...

Node.js

function detectFaces (inputFile, callback) {
  // Make a call to the Vision API to detect the faces
  vision.detectFaces(inputFile, function (err, faces) {
    if (err) {
      return callback(err);
    }
    var numFaces = faces.length;
    console.log('Found ' + numFaces + (numFaces === 1 ? ' face' : ' faces'));
    callback(null, faces);
  });
}
...

PHP

$image = $vision->image(file_get_contents($path), ['FACE_DETECTION']);
$result = $vision->annotate($image);
...

Ruby

image = vision.image path_to_image_file
faces = image.faces
...

Process the response

Congratulations - you've detected the faces in your image! The response to our face annotation request includes a bunch of metadata about the detected faces, which include coordinates of a polygon encompassing the face. At this point, though, this is only a list of numbers. Let's use them to confirm that you have, in fact, found the faces in your image. We'll draw polygons onto a copy of the image, using the coordinates returned by the Vision API:

C#

using (var image = System.Drawing.Image.FromFile(args[0]))
using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(image))
{
    var cyanPen = new System.Drawing.Pen(System.Drawing.Color.
        FromKnownColor(System.Drawing.KnownColor.Cyan), 3);
    foreach (var annotation in response)
    {
        g.DrawPolygon(cyanPen, annotation.BoundingPoly.Vertices.Select(
            (vertex) => new System.Drawing.Point(vertex.X, vertex.Y)).ToArray());
    }
...

Node.js

We use the node-canvas library to draw onto images.

function highlightFaces (inputFile, faces, outputFile, Canvas, callback) {
  fs.readFile(inputFile, function (err, image) {
    if (err) {
      return callback(err);
    }

    var Image = Canvas.Image;
    // Open the original image into a canvas
    var img = new Image();
    img.src = image;
    var canvas = new Canvas(img.width, img.height);
    var context = canvas.getContext('2d');
    context.drawImage(img, 0, 0, img.width, img.height);

    // Now draw boxes around all the faces
    context.strokeStyle = 'rgba(0,255,0,0.8)';
    context.lineWidth = '5';

    faces.forEach(function (face) {
      context.beginPath();
      face.bounds.face.forEach(function (bounds) {
        context.lineTo(bounds.x, bounds.y);
      });
      context.lineTo(face.bounds.face[0].x, face.bounds.face[0].y);
      context.stroke();
    });

    // Write the result to a file
    console.log('Writing to file ' + outputFile);
    var writeStream = fs.createWriteStream(outputFile);
    var pngStream = canvas.pngStream();

    pngStream.on('data', function (chunk) {
      writeStream.write(chunk);
    });
    pngStream.on('error', console.log);
    pngStream.on('end', callback);
  });
}
...

PHP

We use the GD extension to draw onto images.

foreach ($result->info()['faceAnnotations'] as $annotation) {
    if (isset($annotation['boundingPoly'])) {
        $verticies = $annotation['boundingPoly']['vertices'];
        $x1 = isset($verticies[0]['x']) ? $verticies[0]['x'] : 0;
        $y1 = isset($verticies[0]['y']) ? $verticies[0]['y'] : 0;
        $x2 = isset($verticies[2]['x']) ? $verticies[2]['x'] : 0;
        $y2 = isset($verticies[2]['y']) ? $verticies[2]['y'] : 0;
        imagerectangle($outputImage, $x1, $y1, $x2, $y2, 0x00ff00);
    }
}
...

Ruby

We use the rmagick gem to draw onto images.

require "rmagick"
...
image = Magick::Image.read(path_to_image_file).first

faces.each do |face|
  puts "Face bounds:"
  face.bounds.face.each do |vector|
    puts "(#{vector.x}, #{vector.y})"
  end

  draw = Magick::Draw.new
  draw.stroke = "green"
  draw.stroke_width 5
  draw.fill_opacity 0

  x1 = face.bounds.face[0].x.to_i
  y1 = face.bounds.face[0].y.to_i
  x2 = face.bounds.face[2].x.to_i
  y2 = face.bounds.face[2].y.to_i

  draw.rectangle x1, y1, x2, y2
  draw.draw image
end

image.write path_to_output_file

puts "Output file: #{path_to_output_file}"
...

Put it all together

C#

static readonly string s_usage = @"DetectFaces image-file

Use the Google Cloud Vision API to detect faces in the image.
Writes an output file called image-file.faces.
";
public static void Main(string[] args)
{
    if (args.Length < 1)
    {
        Console.WriteLine(s_usage);
        return;
    }
...

To build the sample, open the Solution file dotnet-doc-samples/vision/api/Vision.sln in Visual Studio and build the solution.

To run the sample:

C:\...\> cd dotnet-docs-samples\vision\api\DetectFaces\bin\Debug
C:\...\bin\Debug> DetectFaces ..\..\..\VisionTest\data\face.png

Node.js

function main (inputFile, outputFile, Canvas, callback) {
  outputFile = outputFile || 'out.png';
  detectFaces(inputFile, function (err, faces) {
    if (err) {
      return callback(err);
    }

    console.log('Highlighting...');
    highlightFaces(inputFile, faces, outputFile, Canvas, function (err) {
      if (err) {
        return callback(err);
      }
      console.log('Finished!');
      callback(null, faces);
    });
  });
}
...

To run the sample, run the following command from the sample code directory:

node faceDetection face.png

PHP

To run the sample, run the following command from the sample code directory:

composer install
php vision.php face images/face.png output-image.png

Ruby

if __FILE__ == $PROGRAM_NAME
  project_id = ENV["GOOGLE_CLOUD_PROJECT"]

  if ARGV.size == 2
    draw_box_around_faces path_to_image_file:  ARGV.shift,
                          path_to_output_file: ARGV.shift,
                          project_id:          project_id
  else
    puts <<-usage
Usage: ruby draw_box_around_faces.rb [input-file] [output-file]

Example:
  ruby draw_box_around_faces.rb images/face.png output-image.png
    usage
  end
end
...

To run the sample, run the following command from the sample code directory:

bundle install
bundle exec ruby draw_box_around_faces.rb images/face.png output-image.png

Input image Output image

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

  1. In the Cloud Platform Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click
      Delete project
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Send feedback about...

Google Cloud Vision API Documentation