Detect Web entities and pages

Web Detection detects Web references to an image.

Carnaval image
Image credit: Quinten de Graaf on Unsplash.

Category Responses
Web entities
  • entityId: /m/02p7_j8, score: 1.3225499, description: Carnival in Rio de Janeiro
  • entityId: /m/06gmr, score: 1.1684971, description: Rio de Janeiro
  • entityId: /m/04cx88, score: 1.05945, description: Brazilian Carnival
...
Full matching images
  • url: https://1000lugaresparair.files.wordpress.com/2017/11/quinten-de-graaf-278848.jpg
  • url: https://freewalkingtourrotterdam.com/wp-content/uploads/2017/07/quinten-de-graaf-278848.jpg
...
Partial matching images
  • url: https://www.linnanneito.fi/wp-content/uploads/sambakarnevaali-riossa.jpg
  • url: https://static.airhelp.com/wp-content/uploads/2019/02/26105557/two-women-in-carnival-costumes.jpg
...
Pages with matching images
  • url: https://travelnoire.com/best-carnival-celebrations-around-the-world/,
    pageTitle: Best \u003cb\u003eCarnival\u003c/b\u003e Celebrations Around The World - Travel Noire,
    fullMatchingImages: [{url: https://travelnoire.com/wp-content/uploads/2019/02/quinten-de-graaf-278848-unsplash.jpg}]
  • url: https://bespokebrazil.com/rio-carnival-2019/,
    pageTitle: Visit \u003cb\u003eRio Carnival 2019\u003c/b\u003e with the Brazil Specialists - Bespoke Brazil,
    partialMatchingImages: [{ url: https://bespoke-brazil-2018-bespokebrazil.netdna-ssl.com/wp-content/uploads/2019/01/Carnival-1.jpg}]
...
Visually similar images
  • url: https://www.brazilbookers.com/_images/photos/rio-carnival-images/rio-carnival-2016-carnival-date.jpg
  • url: https://image.redbull.com/rbcom/010/2017-02-08/1331843859949_3/0100/0/1/watch-rio-carnival-2017-live-on-red-bull-tv.jpg
...
Best guess labels rio carnival 2019 dancers

Web detection requests

Set up your GCP project and authentication

Detect Web entities with a local image

The Vision API can perform feature detection on a local image file by sending the contents of the image file as a base64 encoded string in the body of your request.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • base64-encoded-image: The base64 representation (ASCII string) of your binary image data. This string should look similar to the following string:
    • /9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==
    Visit the base64 encode topic for more information.

HTTP method and URL:

POST https://vision.googleapis.com/v1/images:annotate

Request JSON body:

{
  "requests": [
    {
      "image": {
        "content": "base64-encoded-image"
      },
      "features": [
        {
          "maxResults": 10,
          "type": "WEB_DETECTION"
        },
      ]
    }
  ]
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://vision.googleapis.com/v1/images:annotate

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

Response:

C#

Before trying this sample, follow the C# setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision C# API reference documentation.

// Load an image from a local file.
var image = Image.FromFile(filePath);
var client = ImageAnnotatorClient.Create();
WebDetection annotation = client.DetectWebInformation(image);
foreach (var matchingImage in annotation.FullMatchingImages)
{
    Console.WriteLine("MatchingImage Score:\t{0}\tUrl:\t{1}",
        matchingImage.Score, matchingImage.Url);
}
foreach (var page in annotation.PagesWithMatchingImages)
{
    Console.WriteLine("PageWithMatchingImage Score:\t{0}\tUrl:\t{1}",
        page.Score, page.Url);
}
foreach (var matchingImage in annotation.PartialMatchingImages)
{
    Console.WriteLine("PartialMatchingImage Score:\t{0}\tUrl:\t{1}",
        matchingImage.Score, matchingImage.Url);
}
foreach (var entity in annotation.WebEntities)
{
    Console.WriteLine("WebEntity Score:\t{0}\tId:\t{1}\tDescription:\t{2}",
        entity.Score, entity.EntityId, entity.Description);
}

Go

Before trying this sample, follow the Go setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Go API reference documentation.


// detectWeb gets image properties from the Vision API for an image at the given file path.
func detectWeb(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	f, err := os.Open(file)
	if err != nil {
		return err
	}
	defer f.Close()

	image, err := vision.NewImageFromReader(f)
	if err != nil {
		return err
	}
	web, err := client.DetectWeb(ctx, image, nil)
	if err != nil {
		return err
	}

	fmt.Fprintln(w, "Web properties:")
	if len(web.FullMatchingImages) != 0 {
		fmt.Fprintln(w, "\tFull image matches:")
		for _, full := range web.FullMatchingImages {
			fmt.Fprintf(w, "\t\t%s\n", full.Url)
		}
	}
	if len(web.PagesWithMatchingImages) != 0 {
		fmt.Fprintln(w, "\tPages with this image:")
		for _, page := range web.PagesWithMatchingImages {
			fmt.Fprintf(w, "\t\t%s\n", page.Url)
		}
	}
	if len(web.WebEntities) != 0 {
		fmt.Fprintln(w, "\tEntities:")
		fmt.Fprintln(w, "\t\tEntity\t\tScore\tDescription")
		for _, entity := range web.WebEntities {
			fmt.Fprintf(w, "\t\t%-14s\t%-2.4f\t%s\n", entity.EntityId, entity.Score, entity.Description)
		}
	}
	if len(web.BestGuessLabels) != 0 {
		fmt.Fprintln(w, "\tBest guess labels:")
		for _, label := range web.BestGuessLabels {
			fmt.Fprintf(w, "\t\t%s\n", label.Label)
		}
	}

	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Vision API Quickstart Using Client Libraries. For more information, see the Vision API Java API reference documentation.


import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.AnnotateImageResponse;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Feature.Type;
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.WebDetection;
import com.google.cloud.vision.v1.WebDetection.WebEntity;
import com.google.cloud.vision.v1.WebDetection.WebImage;
import com.google.cloud.vision.v1.WebDetection.WebLabel;
import com.google.cloud.vision.v1.WebDetection.WebPage;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class DetectWebDetections {

  public static void detectWebDetections() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String filePath = "path/to/your/image/file.jpg";
    detectWebDetections(filePath);
  }

  // Finds references to the specified image on the web.
  public static void detectWebDetections(String filePath) throws IOException {
    List<AnnotateImageRequest> requests = new ArrayList<>();

    ByteString imgBytes = ByteString.readFrom(new FileInputStream(filePath));

    Image img = Image.newBuilder().setContent(imgBytes).build();
    Feature feat = Feature.newBuilder().setType(Type.WEB_DETECTION).build();
    AnnotateImageRequest request =
        AnnotateImageRequest.newBuilder().addFeatures(feat).setImage(img).build();
    requests.add(request);

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
      BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
      List<AnnotateImageResponse> responses = response.getResponsesList();

      for (AnnotateImageResponse res : responses) {
        if (res.hasError()) {
          System.out.format("Error: %s%n", res.getError().getMessage());
          return;
        }

        // Search the web for usages of the image. You could use these signals later
        // for user input moderation or linking external references.
        // For a full list of available annotations, see http://g.co/cloud/vision/docs
        WebDetection annotation = res.getWebDetection();
        System.out.println("Entity:Id:Score");
        System.out.println("===============");
        for (WebEntity entity : annotation.getWebEntitiesList()) {
          System.out.println(
              entity.getDescription() + " : " + entity.getEntityId() + " : " + entity.getScore());
        }
        for (WebLabel label : annotation.getBestGuessLabelsList()) {
          System.out.format("%nBest guess label: %s", label.getLabel());
        }
        System.out.println("%nPages with matching images: Score%n==");
        for (WebPage page : annotation.getPagesWithMatchingImagesList()) {
          System.out.println(page.getUrl() + " : " + page.getScore());
        }
        System.out.println("%nPages with partially matching images: Score%n==");
        for (WebImage image : annotation.getPartialMatchingImagesList()) {
          System.out.println(image.getUrl() + " : " + image.getScore());
        }
        System.out.println("%nPages with fully matching images: Score%n==");
        for (WebImage image : annotation.getFullMatchingImagesList()) {
          System.out.println(image.getUrl() + " : " + image.getScore());
        }
        System.out.println("%nPages with visually similar images: Score%n==");
        for (WebImage image : annotation.getVisuallySimilarImagesList()) {
          System.out.println(image.getUrl() + " : " + image.getScore());
        }
      }
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Node.js API reference documentation.


// Imports the Google Cloud client library
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const fileName = 'Local image file, e.g. /path/to/image.png';

// Detect similar images on the web to a local file
const [result] = await client.webDetection(fileName);
const webDetection = result.webDetection;
if (webDetection.fullMatchingImages.length) {
  console.log(
    `Full matches found: ${webDetection.fullMatchingImages.length}`
  );
  webDetection.fullMatchingImages.forEach(image => {
    console.log(`  URL: ${image.url}`);
    console.log(`  Score: ${image.score}`);
  });
}

if (webDetection.partialMatchingImages.length) {
  console.log(
    `Partial matches found: ${webDetection.partialMatchingImages.length}`
  );
  webDetection.partialMatchingImages.forEach(image => {
    console.log(`  URL: ${image.url}`);
    console.log(`  Score: ${image.score}`);
  });
}

if (webDetection.webEntities.length) {
  console.log(`Web entities found: ${webDetection.webEntities.length}`);
  webDetection.webEntities.forEach(webEntity => {
    console.log(`  Description: ${webEntity.description}`);
    console.log(`  Score: ${webEntity.score}`);
  });
}

if (webDetection.bestGuessLabels.length) {
  console.log(
    `Best guess labels found: ${webDetection.bestGuessLabels.length}`
  );
  webDetection.bestGuessLabels.forEach(label => {
    console.log(`  Label: ${label.label}`);
  });
}

PHP

Before trying this sample, follow the PHP setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision PHP API reference documentation.

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ImageAnnotatorClient;

// $path = 'path/to/your/image.jpg'

function detect_web($path)
{
    $imageAnnotator = new ImageAnnotatorClient();

    # annotate the image
    $image = file_get_contents($path);
    $response = $imageAnnotator->webDetection($image);
    $web = $response->getWebDetection();

    // Print best guess labels
    printf('%d best guess labels found' . PHP_EOL,
        count($web->getBestGuessLabels()));
    foreach ($web->getBestGuessLabels() as $label) {
        printf('Best guess label: %s' . PHP_EOL, $label->getLabel());
    }
    print(PHP_EOL);

    // Print pages with matching images
    printf('%d pages with matching images found' . PHP_EOL,
        count($web->getPagesWithMatchingImages()));
    foreach ($web->getPagesWithMatchingImages() as $page) {
        printf('URL: %s' . PHP_EOL, $page->getUrl());
    }
    print(PHP_EOL);

    // Print full matching images
    printf('%d full matching images found' . PHP_EOL,
        count($web->getFullMatchingImages()));
    foreach ($web->getFullMatchingImages() as $fullMatchingImage) {
        printf('URL: %s' . PHP_EOL, $fullMatchingImage->getUrl());
    }
    print(PHP_EOL);

    // Print partial matching images
    printf('%d partial matching images found' . PHP_EOL,
        count($web->getPartialMatchingImages()));
    foreach ($web->getPartialMatchingImages() as $partialMatchingImage) {
        printf('URL: %s' . PHP_EOL, $partialMatchingImage->getUrl());
    }
    print(PHP_EOL);

    // Print visually similar images
    printf('%d visually similar images found' . PHP_EOL,
        count($web->getVisuallySimilarImages()));
    foreach ($web->getVisuallySimilarImages() as $visuallySimilarImage) {
        printf('URL: %s' . PHP_EOL, $visuallySimilarImage->getUrl());
    }
    print(PHP_EOL);

    // Print web entities
    printf('%d web entities found' . PHP_EOL,
        count($web->getWebEntities()));
    foreach ($web->getWebEntities() as $entity) {
        printf('Description: %s, Score %s' . PHP_EOL,
            $entity->getDescription(),
            $entity->getScore());
    }

    $imageAnnotator->close();
}

Python

Before trying this sample, follow the Python setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Python API reference documentation.

def detect_web(path):
    """Detects web annotations given an image."""
    from google.cloud import vision
    import io
    client = vision.ImageAnnotatorClient()

    with io.open(path, 'rb') as image_file:
        content = image_file.read()

    image = vision.types.Image(content=content)

    response = client.web_detection(image=image)
    annotations = response.web_detection

    if annotations.best_guess_labels:
        for label in annotations.best_guess_labels:
            print('\nBest guess label: {}'.format(label.label))

    if annotations.pages_with_matching_images:
        print('\n{} Pages with matching images found:'.format(
            len(annotations.pages_with_matching_images)))

        for page in annotations.pages_with_matching_images:
            print('\n\tPage url   : {}'.format(page.url))

            if page.full_matching_images:
                print('\t{} Full Matches found: '.format(
                       len(page.full_matching_images)))

                for image in page.full_matching_images:
                    print('\t\tImage url  : {}'.format(image.url))

            if page.partial_matching_images:
                print('\t{} Partial Matches found: '.format(
                       len(page.partial_matching_images)))

                for image in page.partial_matching_images:
                    print('\t\tImage url  : {}'.format(image.url))

    if annotations.web_entities:
        print('\n{} Web entities found: '.format(
            len(annotations.web_entities)))

        for entity in annotations.web_entities:
            print('\n\tScore      : {}'.format(entity.score))
            print(u'\tDescription: {}'.format(entity.description))

    if annotations.visually_similar_images:
        print('\n{} visually similar images found:\n'.format(
            len(annotations.visually_similar_images)))

        for image in annotations.visually_similar_images:
            print('\tImage url    : {}'.format(image.url))

    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))

Ruby

Before trying this sample, follow the Ruby setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Ruby API reference documentation.

# image_path = "Path to local image file, eg. './image.png'"

require "google/cloud/vision"

image_annotator = Google::Cloud::Vision.image_annotator

response = image_annotator.web_detection(
  image:       image_path,
  max_results: 15 # optional, defaults to 10
)

response.responses.each do |res|
  res.web_detection.web_entities.each do |entity|
    puts entity.description
  end

  res.web_detection.full_matching_images.each do |match|
    puts match.url
  end
end

Detect Web entities with a remote image

For your convenience, the Vision API can perform feature detection directly on an image file located in Google Cloud Storage or on the Web without the need to send the contents of the image file in the body of your request.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • cloud-storage-image-uri: the path to a valid image file in a Cloud Storage bucket. You must at least have read privileges to the file. Example:
    • gs://cloud-samples-data/vision/web/carnaval.jpeg

HTTP method and URL:

POST https://vision.googleapis.com/v1/images:annotate

Request JSON body:

{
  "requests": [
    {
      "image": {
        "source": {
          "gcsImageUri": "cloud-storage-image-uri"
        }
      },
      "features": [
        {
          "maxResults": 10,
          "type": "WEB_DETECTION"
        },
      ]
    }
  ]
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://vision.googleapis.com/v1/images:annotate

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

Response:

C#

Before trying this sample, follow the C# setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision C# API reference documentation.

// Specify a Google Cloud Storage uri for the image
// or a publicly accessible HTTP or HTTPS uri.
var image = Image.FromUri(uri);
var client = ImageAnnotatorClient.Create();
WebDetection annotation = client.DetectWebInformation(image);
foreach (var matchingImage in annotation.FullMatchingImages)
{
    Console.WriteLine("MatchingImage Score:\t{0}\tUrl:\t{1}",
        matchingImage.Score, matchingImage.Url);
}
foreach (var page in annotation.PagesWithMatchingImages)
{
    Console.WriteLine("PageWithMatchingImage Score:\t{0}\tUrl:\t{1}",
        page.Score, page.Url);
}
foreach (var matchingImage in annotation.PartialMatchingImages)
{
    Console.WriteLine("PartialMatchingImage Score:\t{0}\tUrl:\t{1}",
        matchingImage.Score, matchingImage.Url);
}
foreach (var entity in annotation.WebEntities)
{
    Console.WriteLine("WebEntity Score:\t{0}\tId:\t{1}\tDescription:\t{2}",
        entity.Score, entity.EntityId, entity.Description);
}

Go

Before trying this sample, follow the Go setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Go API reference documentation.


// detectWeb gets image properties from the Vision API for an image at the given file path.
func detectWebURI(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	image := vision.NewImageFromURI(file)
	web, err := client.DetectWeb(ctx, image, nil)
	if err != nil {
		return err
	}

	fmt.Fprintln(w, "Web properties:")
	if len(web.FullMatchingImages) != 0 {
		fmt.Fprintln(w, "\tFull image matches:")
		for _, full := range web.FullMatchingImages {
			fmt.Fprintf(w, "\t\t%s\n", full.Url)
		}
	}
	if len(web.PagesWithMatchingImages) != 0 {
		fmt.Fprintln(w, "\tPages with this image:")
		for _, page := range web.PagesWithMatchingImages {
			fmt.Fprintf(w, "\t\t%s\n", page.Url)
		}
	}
	if len(web.WebEntities) != 0 {
		fmt.Fprintln(w, "\tEntities:")
		fmt.Fprintln(w, "\t\tEntity\t\tScore\tDescription")
		for _, entity := range web.WebEntities {
			fmt.Fprintf(w, "\t\t%-14s\t%-2.4f\t%s\n", entity.EntityId, entity.Score, entity.Description)
		}
	}
	if len(web.BestGuessLabels) != 0 {
		fmt.Fprintln(w, "\tBest guess labels:")
		for _, label := range web.BestGuessLabels {
			fmt.Fprintf(w, "\t\t%s\n", label.Label)
		}
	}

	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Java API reference documentation.


import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.AnnotateImageResponse;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.ImageSource;
import com.google.cloud.vision.v1.WebDetection;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class DetectWebDetectionsGcs {

  public static void detectWebDetectionsGcs() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String filePath = "gs://your-gcs-bucket/path/to/image/file.jpg";
    detectWebDetectionsGcs(filePath);
  }

  // Detects whether the remote image on Google Cloud Storage has features you would want to
  // moderate.
  public static void detectWebDetectionsGcs(String gcsPath) throws IOException {
    List<AnnotateImageRequest> requests = new ArrayList<>();

    ImageSource imgSource = ImageSource.newBuilder().setGcsImageUri(gcsPath).build();
    Image img = Image.newBuilder().setSource(imgSource).build();
    Feature feat = Feature.newBuilder().setType(Feature.Type.WEB_DETECTION).build();
    AnnotateImageRequest request =
        AnnotateImageRequest.newBuilder().addFeatures(feat).setImage(img).build();
    requests.add(request);

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
      BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
      List<AnnotateImageResponse> responses = response.getResponsesList();

      for (AnnotateImageResponse res : responses) {
        if (res.hasError()) {
          System.out.format("Error: %s%n", res.getError().getMessage());
          return;
        }

        // Search the web for usages of the image. You could use these signals later
        // for user input moderation or linking external references.
        // For a full list of available annotations, see http://g.co/cloud/vision/docs
        WebDetection annotation = res.getWebDetection();
        System.out.println("Entity:Id:Score");
        System.out.println("===============");
        for (WebDetection.WebEntity entity : annotation.getWebEntitiesList()) {
          System.out.println(
              entity.getDescription() + " : " + entity.getEntityId() + " : " + entity.getScore());
        }
        for (WebDetection.WebLabel label : annotation.getBestGuessLabelsList()) {
          System.out.format("%nBest guess label: %s", label.getLabel());
        }
        System.out.println("%nPages with matching images: Score%n==");
        for (WebDetection.WebPage page : annotation.getPagesWithMatchingImagesList()) {
          System.out.println(page.getUrl() + " : " + page.getScore());
        }
        System.out.println("%nPages with partially matching images: Score%n==");
        for (WebDetection.WebImage image : annotation.getPartialMatchingImagesList()) {
          System.out.println(image.getUrl() + " : " + image.getScore());
        }
        System.out.println("%nPages with fully matching images: Score%n==");
        for (WebDetection.WebImage image : annotation.getFullMatchingImagesList()) {
          System.out.println(image.getUrl() + " : " + image.getScore());
        }
        System.out.println("%nPages with visually similar images: Score%n==");
        for (WebDetection.WebImage image : annotation.getVisuallySimilarImagesList()) {
          System.out.println(image.getUrl() + " : " + image.getScore());
        }
      }
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Node.js API reference documentation.


// Imports the Google Cloud client libraries
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// const bucketName = 'Bucket where the file resides, e.g. my-bucket';
// const fileName = 'Path to file within bucket, e.g. path/to/image.png';

// Detect similar images on the web to a remote file
const [result] = await client.webDetection(`gs://${bucketName}/${fileName}`);
const webDetection = result.webDetection;
if (webDetection.fullMatchingImages.length) {
  console.log(
    `Full matches found: ${webDetection.fullMatchingImages.length}`
  );
  webDetection.fullMatchingImages.forEach(image => {
    console.log(`  URL: ${image.url}`);
    console.log(`  Score: ${image.score}`);
  });
}

if (webDetection.partialMatchingImages.length) {
  console.log(
    `Partial matches found: ${webDetection.partialMatchingImages.length}`
  );
  webDetection.partialMatchingImages.forEach(image => {
    console.log(`  URL: ${image.url}`);
    console.log(`  Score: ${image.score}`);
  });
}

if (webDetection.webEntities.length) {
  console.log(`Web entities found: ${webDetection.webEntities.length}`);
  webDetection.webEntities.forEach(webEntity => {
    console.log(`  Description: ${webEntity.description}`);
    console.log(`  Score: ${webEntity.score}`);
  });
}

if (webDetection.bestGuessLabels.length) {
  console.log(
    `Best guess labels found: ${webDetection.bestGuessLabels.length}`
  );
  webDetection.bestGuessLabels.forEach(label => {
    console.log(`  Label: ${label.label}`);
  });
}

PHP

Before trying this sample, follow the PHP setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision PHP API reference documentation.

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ImageAnnotatorClient;

// $path = 'gs://path/to/your/image.jpg'

function detect_web_gcs($path)
{
    $imageAnnotator = new ImageAnnotatorClient();

    # annotate the image
    $response = $imageAnnotator->webDetection($path);
    $web = $response->getWebDetection();

    if ($web) {
        printf('%d best guess labels found' . PHP_EOL,
            count($web->getPagesWithMatchingImages()));
        foreach ($web->getBestGuessLabels() as $label) {
            printf('Best guess label: %s' . PHP_EOL, $label->getLabel());
        }
        print(PHP_EOL);

        // Print pages with matching images
        printf('%d pages with matching images found' . PHP_EOL,
            count($web->getPagesWithMatchingImages()));
        foreach ($web->getPagesWithMatchingImages() as $page) {
            printf('URL: %s' . PHP_EOL, $page->getUrl());
        }
        print(PHP_EOL);

        // Print full matching images
        printf('%d full matching images found' . PHP_EOL,
            count($web->getFullMatchingImages()));
        foreach ($web->getFullMatchingImages() as $fullMatchingImage) {
            printf('URL: %s' . PHP_EOL, $fullMatchingImage->getUrl());
        }
        print(PHP_EOL);

        // Print partial matching images
        printf('%d partial matching images found' . PHP_EOL,
            count($web->getPartialMatchingImages()));
        foreach ($web->getPartialMatchingImages() as $partialMatchingImage) {
            printf('URL: %s' . PHP_EOL, $partialMatchingImage->getUrl());
        }
        print(PHP_EOL);

        // Print visually similar images
        printf('%d visually similar images found' . PHP_EOL,
            count($web->getVisuallySimilarImages()));
        foreach ($web->getVisuallySimilarImages() as $visuallySimilarImage) {
            printf('URL: %s' . PHP_EOL, $visuallySimilarImage->getUrl());
        }
        print(PHP_EOL);

        // Print web entities
        printf('%d web entities found' . PHP_EOL,
            count($web->getWebEntities()));
        foreach ($web->getWebEntities() as $entity) {
            printf('Description: %s, Score: %f' . PHP_EOL,
                $entity->getDescription(),
                $entity->getScore());
        }
    } else {
        print('No Results.' . PHP_EOL);
    }

    $imageAnnotator->close();
}

Python

Before trying this sample, follow the Python setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Python API reference documentation.

def detect_web_uri(uri):
    """Detects web annotations in the file located in Google Cloud Storage."""
    from google.cloud import vision
    client = vision.ImageAnnotatorClient()
    image = vision.types.Image()
    image.source.image_uri = uri

    response = client.web_detection(image=image)
    annotations = response.web_detection

    if annotations.best_guess_labels:
        for label in annotations.best_guess_labels:
            print('\nBest guess label: {}'.format(label.label))

    if annotations.pages_with_matching_images:
        print('\n{} Pages with matching images found:'.format(
            len(annotations.pages_with_matching_images)))

        for page in annotations.pages_with_matching_images:
            print('\n\tPage url   : {}'.format(page.url))

            if page.full_matching_images:
                print('\t{} Full Matches found: '.format(
                       len(page.full_matching_images)))

                for image in page.full_matching_images:
                    print('\t\tImage url  : {}'.format(image.url))

            if page.partial_matching_images:
                print('\t{} Partial Matches found: '.format(
                       len(page.partial_matching_images)))

                for image in page.partial_matching_images:
                    print('\t\tImage url  : {}'.format(image.url))

    if annotations.web_entities:
        print('\n{} Web entities found: '.format(
            len(annotations.web_entities)))

        for entity in annotations.web_entities:
            print('\n\tScore      : {}'.format(entity.score))
            print(u'\tDescription: {}'.format(entity.description))

    if annotations.visually_similar_images:
        print('\n{} visually similar images found:\n'.format(
            len(annotations.visually_similar_images)))

        for image in annotations.visually_similar_images:
            print('\tImage url    : {}'.format(image.url))

    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))

Ruby

Before trying this sample, follow the Ruby setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Ruby API reference documentation.

# image_path = "Google Cloud Storage URI, eg. 'gs://my-bucket/image.png'"

require "google/cloud/vision"

image_annotator = Google::Cloud::Vision.image_annotator

response = image_annotator.web_detection(
  image:       image_path,
  max_results: 15 # optional, defaults to 10
)

response.responses.each do |res|
  res.web_detection.web_entities.each do |entity|
    puts entity.description
  end

  res.web_detection.full_matching_images.each do |match|
    puts match.url
  end
end

gcloud command

To perform Web detection, use the gcloud ml vision detect-web command as shown in the following example:

gcloud ml vision detect-web gs://cloud-samples-data/vision/web/carnaval.jpeg

Using geographic metadata with a local image

The Vision API can access geotag metadata in your image files to return more relevant web entities and pages. To allow geotag usage, specify 'includeGeoResults': true in your request.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • cloud-storage-image-uri: the path to a valid image file in a Cloud Storage bucket. You must at least have read privileges to the file. Example:
    • gs://cloud-samples-data/vision/web/carnaval.jpeg

HTTP method and URL:

POST https://vision.googleapis.com/v1/images:annotate

Request JSON body:

{
  "requests": [
    {
      "image": {
        "source": {
          "gcsImageUri": "cloud-storage-image-uri"
        }
      },
      "features": [
        {
          "type": "WEB_DETECTION"
        }
      ],
      "imageContext": {
        "webDetectionParams": {
          "includeGeoResults": true
          }
        }
    }
  ]
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://vision.googleapis.com/v1/images:annotate

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

If the request is successful, the server returns a 200 OK HTTP status code and the response in JSON format.

Response:

Go

Before trying this sample, follow the Go setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Go API reference documentation.


// detectWebGeo detects geographic metadata from the Vision API for an image at the given file path.
func detectWebGeo(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	f, err := os.Open(file)
	if err != nil {
		return err
	}
	defer f.Close()

	image, err := vision.NewImageFromReader(f)
	if err != nil {
		return err
	}
	imageContext := &visionpb.ImageContext{
		WebDetectionParams: &visionpb.WebDetectionParams{
			IncludeGeoResults: true,
		},
	}
	web, err := client.DetectWeb(ctx, image, imageContext)
	if err != nil {
		return err
	}

	if len(web.WebEntities) != 0 {
		fmt.Fprintln(w, "Entities:")
		fmt.Fprintln(w, "\tEntity\t\tScore\tDescription")
		for _, entity := range web.WebEntities {
			fmt.Fprintf(w, "\t%-14s\t%-2.4f\t%s\n", entity.EntityId, entity.Score, entity.Description)
		}
	}

	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Java API reference documentation.


import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Feature.Type;
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.ImageContext;
import com.google.cloud.vision.v1.WebDetectionParams;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Arrays;

public class DetectWebEntitiesIncludeGeoResults {

  public static void detectWebEntitiesIncludeGeoResults() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String filePath = "path/to/your/image/file.jpg";
    detectWebEntitiesIncludeGeoResults(filePath);
  }

  // Find web entities given a local image.
  public static void detectWebEntitiesIncludeGeoResults(String filePath) throws IOException {

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
      // Read in the local image
      ByteString contents = ByteString.readFrom(new FileInputStream(filePath));

      // Build the image
      Image image = Image.newBuilder().setContent(contents).build();

      // Enable `IncludeGeoResults`
      WebDetectionParams webDetectionParams =
          WebDetectionParams.newBuilder().setIncludeGeoResults(true).build();

      // Set the parameters for the image
      ImageContext imageContext =
          ImageContext.newBuilder().setWebDetectionParams(webDetectionParams).build();

      // Create the request with the image, imageContext, and the specified feature: web detection
      AnnotateImageRequest request =
          AnnotateImageRequest.newBuilder()
              .addFeatures(Feature.newBuilder().setType(Type.WEB_DETECTION))
              .setImage(image)
              .setImageContext(imageContext)
              .build();

      // Perform the request
      BatchAnnotateImagesResponse response = client.batchAnnotateImages(Arrays.asList(request));

      // Display the results
      response.getResponsesList().stream()
          .forEach(
              r ->
                  r.getWebDetection().getWebEntitiesList().stream()
                      .forEach(
                          entity -> {
                            System.out.format("Description: %s%n", entity.getDescription());
                            System.out.format("Score: %f%n", entity.getScore());
                          }));
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Node.js API reference documentation.

// Imports the Google Cloud client library
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following line before running the sample.
 */
// const fileName = 'Local image file, e.g. /path/to/image.png';

const request = {
  image: {
    source: {
      filename: fileName,
    },
  },
  imageContext: {
    webDetectionParams: {
      includeGeoResults: true,
    },
  },
};

// Detect similar images on the web to a local file
const [result] = await client.webDetection(request);
const webDetection = result.webDetection;
webDetection.webEntities.forEach(entity => {
  console.log(`Score: ${entity.score}`);
  console.log(`Description: ${entity.description}`);
});

PHP

Before trying this sample, follow the PHP setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision PHP API reference documentation.

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ImageAnnotatorClient;
use Google\Cloud\Vision\V1\ImageContext;
use Google\Cloud\Vision\V1\WebDetectionParams;

// $path = 'path/to/your/image.jpg'

/**
 * Detect web entities on an image and include the image's geo metadata
 * to improve the quality of the detection.
 */
function detect_web_with_geo_metadata($path)
{
    $imageAnnotator = new ImageAnnotatorClient();

    # enable include geo results
    $params = new WebDetectionParams();
    $params->setIncludeGeoResults(true);
    $imageContext = new ImageContext();
    $imageContext-> setWebDetectionParams($params);

    # annotate the image
    $image = file_get_contents($path);
    $response = $imageAnnotator->webDetection($image, ['imageContext' => $imageContext]);
    $web = $response->getWebDetection();

    if ($web->getWebEntities()) {
        printf('%d web entities found:' . PHP_EOL,
            count($web->getWebEntities()));
        foreach ($web->getWebEntities() as $entity) {
            printf('Description: %s ' . PHP_EOL, $entity->getDescription());
            printf('Score: %f' . PHP_EOL, $entity->getScore());
            print(PHP_EOL);
        }
    }

    $imageAnnotator->close();
}

Python

Before trying this sample, follow the Python setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Python API reference documentation.

def web_entities_include_geo_results(path):
    """Detects web annotations given an image, using the geotag metadata
    in the image to detect web entities."""
    from google.cloud import vision
    import io
    client = vision.ImageAnnotatorClient()

    with io.open(path, 'rb') as image_file:
        content = image_file.read()

    image = vision.types.Image(content=content)

    web_detection_params = vision.types.WebDetectionParams(
        include_geo_results=True)
    image_context = vision.types.ImageContext(
        web_detection_params=web_detection_params)

    response = client.web_detection(image=image, image_context=image_context)

    for entity in response.web_detection.web_entities:
        print('\n\tScore      : {}'.format(entity.score))
        print(u'\tDescription: {}'.format(entity.description))

    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))

gcloud command

To perform Web detection, use the gcloud ml vision detect-web command as shown in the following example:

gcloud ml vision detect-web gs://cloud-samples-data/vision/web/carnaval.jpeg

Using geographic metadata with a remote image

Go

Before trying this sample, follow the Go setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Go API reference documentation.


// detectWebGeo detects geographic metadata from the Vision API for an image at the given file path.
func detectWebGeoURI(w io.Writer, file string) error {
	ctx := context.Background()

	client, err := vision.NewImageAnnotatorClient(ctx)
	if err != nil {
		return err
	}

	image := vision.NewImageFromURI(file)
	imageContext := &visionpb.ImageContext{
		WebDetectionParams: &visionpb.WebDetectionParams{
			IncludeGeoResults: true,
		},
	}
	web, err := client.DetectWeb(ctx, image, imageContext)
	if err != nil {
		return err
	}

	if len(web.WebEntities) != 0 {
		fmt.Fprintln(w, "Entities:")
		fmt.Fprintln(w, "\tEntity\t\tScore\tDescription")
		for _, entity := range web.WebEntities {
			fmt.Fprintf(w, "\t%-14s\t%-2.4f\t%s\n", entity.EntityId, entity.Score, entity.Description)
		}
	}

	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Java API reference documentation.


import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.ImageContext;
import com.google.cloud.vision.v1.ImageSource;
import com.google.cloud.vision.v1.WebDetectionParams;
import java.io.IOException;
import java.util.Arrays;

public class DetectWebEntitiesIncludeGeoResultsGcs {

  public static void detectWebEntitiesIncludeGeoResultsGcs() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String filePath = "gs://your-gcs-bucket/path/to/image/file.jpg";
    detectWebEntitiesIncludeGeoResultsGcs(filePath);
  }

  // Find web entities given the remote image on Google Cloud Storage.
  public static void detectWebEntitiesIncludeGeoResultsGcs(String gcsPath) throws IOException {

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
      // Set the image source to the given gs uri
      ImageSource imageSource = ImageSource.newBuilder().setGcsImageUri(gcsPath).build();
      // Build the image
      Image image = Image.newBuilder().setSource(imageSource).build();

      // Enable `IncludeGeoResults`
      WebDetectionParams webDetectionParams =
          WebDetectionParams.newBuilder().setIncludeGeoResults(true).build();

      // Set the parameters for the image
      ImageContext imageContext =
          ImageContext.newBuilder().setWebDetectionParams(webDetectionParams).build();

      // Create the request with the image, imageContext, and the specified feature: web detection
      AnnotateImageRequest request =
          AnnotateImageRequest.newBuilder()
              .addFeatures(Feature.newBuilder().setType(Feature.Type.WEB_DETECTION))
              .setImage(image)
              .setImageContext(imageContext)
              .build();

      // Perform the request
      BatchAnnotateImagesResponse response = client.batchAnnotateImages(Arrays.asList(request));

      // Display the results
      response.getResponsesList().stream()
          .forEach(
              r ->
                  r.getWebDetection().getWebEntitiesList().stream()
                      .forEach(
                          entity -> {
                            System.out.format("Description: %s%n", entity.getDescription());
                            System.out.format("Score: %f%n", entity.getScore());
                          }));
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Node.js API reference documentation.

// Imports the Google Cloud client library
const vision = require('@google-cloud/vision');

// Creates a client
const client = new vision.ImageAnnotatorClient();

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// const bucketName = 'Bucket where the file resides, e.g. my-bucket';
// const fileName = 'Path to file within bucket, e.g. path/to/image.png';

const request = {
  image: {
    source: {
      imageUri: `gs://${bucketName}/${fileName}`,
    },
  },
  imageContext: {
    webDetectionParams: {
      includeGeoResults: true,
    },
  },
};

// Detect similar images on the web to a remote file
const [result] = await client.webDetection(request);
const webDetection = result.webDetection;
webDetection.webEntities.forEach(entity => {
  console.log(`Score: ${entity.score}`);
  console.log(`Description: ${entity.description}`);
});

PHP

Before trying this sample, follow the PHP setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision PHP API reference documentation.

namespace Google\Cloud\Samples\Vision;

use Google\Cloud\Vision\V1\ImageAnnotatorClient;
use Google\Cloud\Vision\V1\ImageContext;
use Google\Cloud\Vision\V1\WebDetectionParams;

// $path = 'gs://path/to/your/image.jpg'

/**
 * Detect web entities on an image and include the image's geo metadata
 * to improve the quality of the detection.
 */
function detect_web_with_geo_metadata_gcs($path)
{
    $imageAnnotator = new ImageAnnotatorClient();

    # enable include geo results
    $params = new WebDetectionParams();
    $params->setIncludeGeoResults(true);
    $imageContext = new ImageContext();
    $imageContext-> setWebDetectionParams($params);

    # annotate the image
    $response = $imageAnnotator->webDetection($path, ['imageContext' => $imageContext]);
    $web = $response->getWebDetection();

    if ($web) {
        printf('%d web entities found:' . PHP_EOL,
            count($web->getWebEntities()));
        foreach ($web->getWebEntities() as $entity) {
            printf('Description: %s ' . PHP_EOL, $entity->getDescription());
            printf('Score: %f' . PHP_EOL, $entity->getScore());
            print(PHP_EOL);
        }
    } else {
        print('No Results.' . PHP_EOL);
    }

    $imageAnnotator->close();
}

Python

Before trying this sample, follow the Python setup instructions in the Vision Quickstart Using Client Libraries. For more information, see the Vision Python API reference documentation.

def web_entities_include_geo_results_uri(uri):
    """Detects web annotations given an image in the file located in
    Google Cloud Storage., using the geotag metadata in the image to
    detect web entities."""
    from google.cloud import vision
    client = vision.ImageAnnotatorClient()

    image = vision.types.Image()
    image.source.image_uri = uri

    web_detection_params = vision.types.WebDetectionParams(
        include_geo_results=True)
    image_context = vision.types.ImageContext(
        web_detection_params=web_detection_params)

    response = client.web_detection(image=image, image_context=image_context)

    for entity in response.web_detection.web_entities:
        print('\n\tScore      : {}'.format(entity.score))
        print(u'\tDescription: {}'.format(entity.description))

    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))

Try it

Try Web entities detection below. You can use the image specified already (gs://cloud-samples-data/vision/web/carnaval.jpeg) or specify your own image in its place. Send the request by selecting Execute.

Try repeating the request with includeGeoResults set to false.

Carnaval image
Image credit: Quinten de Graaf on Unsplash.