Making individual predictions

After you have created (trained) a model and deployed it you can make online (or synchronous) prediction requests to it.

Online (individual) prediction example

After you have deployed your trained model, you can request a prediction for an image using the predict method, or use the UI to get prediction annotations. The predict method applies labels to object bounding boxes in your image.

Your model incurs charges while it is deployed. After making predictions with your trained model you can undeploy your model if you no longer want to incur model hosting usage charges.

Web UI

  1. Open the Cloud AutoML Vision Object Detection UI and click the Models tab (with lightbulb icon) in the left navigation bar to display the available models.

    To view the models for a different project, select the project from the drop-down list in the upper right of the title bar.

  2. Click the row for the model you want to use to label your images.

  3. If your model is not yet deployed, deploy it now by selecting Deploy model.

    Your model must be deployed to use online predictions. Deploying your model incurs costs. For more information, see the pricing page.

  4. Click the Test & Use tab just below the title bar.

    Test and use model page

  5. Click Upload Images to upload the images that you want to label.

    Prediction on uploaded image page

REST & CMD LINE

To test prediction you must first deploy your Cloud-hosted model.

Before using any of the request data below, make the following replacements:

  • project-id: your GCP project ID.
  • model-id: the ID of your model, from the response when you created the model. The ID is the last element of the name of your model. For example:
    • model name: projects/project-id/locations/location-id/models/IOD4412217016962778756
    • model id: IOD4412217016962778756
  • base64-encoded-image: the base64 representation (ASCII string) of your binary image data. This string should look similar to the following string: /9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==. Visit the base64 encode topic for more information.

Field-specific considerations:

  • scoreThreshold - A value from 0 to 1. Only values with score thresholds of at least this value will be displayed. The default value is 0.5.
  • maxBoundingBoxCount - The greatest number (upper bound) of bounding boxes to be returned in a response. The default value is 100 and the maximum is 500. This value is subject to resource constraints, and may be limited by the server.

HTTP method and URL:

POST https://automl.googleapis.com/v1/projects/project-id/locations/us-central1/models/model-id:predict

Request JSON body:

{
  "payload": {
    "image": {
      "imageBytes": "base64-encoded-image"
    }
  },
  "params": {
    "scoreThreshold": "0.5",
    "maxBoundingBoxCount": "100"
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://automl.googleapis.com/v1/projects/project-id/locations/us-central1/models/model-id:predict

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://automl.googleapis.com/v1/projects/project-id/locations/us-central1/models/model-id:predict" | Select-Object -Expand Content

Output is returned in JSON form. The predictions from your AutoML Vision Object Detection model are contained in the payload field:

  • boundingBox of an object is specified by diagonally opposed vertices.
  • displayName is the object's label predicted by the AutoML Vision Object Detection model.
  • score represents a confidence level that the specified label applies to the image. It ranges from 0 (no confidence) to 1 (high confidence).

{
  "payload": [
    {
      "imageObjectDetection": {
        "boundingBox": {
          "normalizedVertices": [
            {
              "x": 0.034553755,
              "y": 0.015524037
            },
            {
              "x": 0.941527,
              "y": 0.9912563
            }
          ]
        },
        "score": 0.9997793
      },
      "displayName": "Salad"
    },
    {
      "imageObjectDetection": {
        "boundingBox": {
          "normalizedVertices": [
            {
              "x": 0.11737197,
              "y": 0.7098793
            },
            {
              "x": 0.510878,
              "y": 0.87987
            }
          ]
        },
        "score": 0.63219965
      },
      "displayName": "Tomato"
    }
  ]
}

Go

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

import (
	"context"
	"fmt"
	"io"
	"io/ioutil"
	"os"

	automl "cloud.google.com/go/automl/apiv1"
	automlpb "google.golang.org/genproto/googleapis/cloud/automl/v1"
)

// visionObjectDetectionPredict does a prediction for image classification.
func visionObjectDetectionPredict(w io.Writer, projectID string, location string, modelID string, filePath string) error {
	// projectID := "my-project-id"
	// location := "us-central1"
	// modelID := "IOD123456789..."
	// filePath := "path/to/image.jpg"

	ctx := context.Background()
	client, err := automl.NewPredictionClient(ctx)
	if err != nil {
		return fmt.Errorf("NewPredictionClient: %v", err)
	}
	defer client.Close()

	file, err := os.Open(filePath)
	if err != nil {
		return fmt.Errorf("Open: %v", err)
	}
	defer file.Close()
	bytes, err := ioutil.ReadAll(file)
	if err != nil {
		return fmt.Errorf("ReadAll: %v", err)
	}

	req := &automlpb.PredictRequest{
		Name: fmt.Sprintf("projects/%s/locations/%s/models/%s", projectID, location, modelID),
		Payload: &automlpb.ExamplePayload{
			Payload: &automlpb.ExamplePayload_Image{
				Image: &automlpb.Image{
					Data: &automlpb.Image_ImageBytes{
						ImageBytes: bytes,
					},
				},
			},
		},
		// Params is additional domain-specific parameters.
		Params: map[string]string{
			// score_threshold is used to filter the result.
			"score_threshold": "0.8",
		},
	}

	resp, err := client.Predict(ctx, req)
	if err != nil {
		return fmt.Errorf("Predict: %v", err)
	}

	for _, payload := range resp.GetPayload() {
		fmt.Fprintf(w, "Predicted class name: %v\n", payload.GetDisplayName())
		fmt.Fprintf(w, "Predicted class score: %v\n", payload.GetImageObjectDetection().GetScore())
		boundingBox := payload.GetImageObjectDetection().GetBoundingBox()
		fmt.Fprintf(w, "Normalized vertices:\n")
		for _, vertex := range boundingBox.GetNormalizedVertices() {
			fmt.Fprintf(w, "\tX: %v, Y: %v\n", vertex.GetX(), vertex.GetY())
		}
	}

	return nil
}

Java

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.


import com.google.cloud.automl.v1.AnnotationPayload;
import com.google.cloud.automl.v1.BoundingPoly;
import com.google.cloud.automl.v1.ExamplePayload;
import com.google.cloud.automl.v1.Image;
import com.google.cloud.automl.v1.ImageObjectDetectionAnnotation;
import com.google.cloud.automl.v1.ModelName;
import com.google.cloud.automl.v1.NormalizedVertex;
import com.google.cloud.automl.v1.PredictRequest;
import com.google.cloud.automl.v1.PredictResponse;
import com.google.cloud.automl.v1.PredictionServiceClient;
import com.google.protobuf.ByteString;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

class VisionObjectDetectionPredict {

  static void predict() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    String modelId = "YOUR_MODEL_ID";
    String filePath = "path_to_local_file.jpg";
    predict(projectId, modelId, filePath);
  }

  static void predict(String projectId, String modelId, String filePath) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (PredictionServiceClient client = PredictionServiceClient.create()) {
      // Get the full path of the model.
      ModelName name = ModelName.of(projectId, "us-central1", modelId);
      ByteString content = ByteString.copyFrom(Files.readAllBytes(Paths.get(filePath)));
      Image image = Image.newBuilder().setImageBytes(content).build();
      ExamplePayload payload = ExamplePayload.newBuilder().setImage(image).build();
      PredictRequest predictRequest =
          PredictRequest.newBuilder()
              .setName(name.toString())
              .setPayload(payload)
              .putParams(
                  "score_threshold", "0.5") // [0.0-1.0] Only produce results higher than this value
              .build();

      PredictResponse response = client.predict(predictRequest);
      for (AnnotationPayload annotationPayload : response.getPayloadList()) {
        System.out.format("Predicted class name: %s%n", annotationPayload.getDisplayName());
        ImageObjectDetectionAnnotation imageObjectDetectionAnnotation =
            annotationPayload.getImageObjectDetection();
        System.out.format(
            "Predicted class score: %.2f%n", imageObjectDetectionAnnotation.getScore());
        BoundingPoly boundingPoly = imageObjectDetectionAnnotation.getBoundingBox();
        System.out.println("Normalized Vertices:");
        for (NormalizedVertex vertex : boundingPoly.getNormalizedVerticesList()) {
          System.out.format("\tX: %.2f, Y: %.2f%n", vertex.getX(), vertex.getY());
        }
      }
    }
  }
}

Node.js

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const modelId = 'YOUR_MODEL_ID';
// const filePath = 'path_to_local_file.jpg';

// Imports the Google Cloud AutoML library
const {PredictionServiceClient} = require('@google-cloud/automl').v1;
const fs = require('fs');

// Instantiates a client
const client = new PredictionServiceClient();

// Read the file content for translation.
const content = fs.readFileSync(filePath);

async function predict() {
  // Construct request
  // params is additional domain-specific parameters.
  // score_threshold is used to filter the result
  const request = {
    name: client.modelPath(projectId, location, modelId),
    payload: {
      image: {
        imageBytes: content,
      },
    },
    params: {
      score_threshold: '0.8',
    },
  };

  const [response] = await client.predict(request);

  for (const annotationPayload of response.payload) {
    console.log(`Predicted class name: ${annotationPayload.displayName}`);
    console.log(
      `Predicted class score: ${annotationPayload.imageObjectDetection.score}`
    );
    console.log('Normalized vertices:');
    for (const vertex of annotationPayload.imageObjectDetection.boundingBox
      .normalizedVertices) {
      console.log(`\tX: ${vertex.x}, Y: ${vertex.y}`);
    }
  }
}

predict();

PHP

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

use Google\Cloud\AutoMl\V1\ExamplePayload;
use Google\Cloud\AutoMl\V1\Image;
use Google\Cloud\AutoMl\V1\PredictionServiceClient;

/** Uncomment and populate these variables in your code */
// $projectId = '[Google Cloud Project ID]';
// $location = 'us-central1';
// $modelId = 'my_model_id_123';
// $filePath = 'path_to_local_file.jpg';

$client = new PredictionServiceClient();

try {
    // get full path of model
    $formattedName = $client->modelName(
        $projectId,
        $location,
        $modelId);

    // read the file
    $content = file_get_contents($filePath);
    $image = (new Image())
        ->setImageBytes($content);
    // create payload
    $payload = (new ExamplePayload())
        ->setImage($image);

    // params is additional domain-specific parameters
    // score_threshold is used to filter the result
    $params = ['score_threshold' => '0.8']; // value between 0.0 and 1.0

    // predict with above model and payload
    $response = $client->predict($formattedName, $payload, $params);
    $annotations = $response->getPayload();

    // display results
    foreach ($annotations as $annotation) {
        $imageObjectDetection = $annotation->getImageObjectDetection();
        printf('Predicted class name: %s' . PHP_EOL, $annotation->getDisplayName());
        printf('Predicted class score: %s' . PHP_EOL, $imageObjectDetection->getScore());
        $vertices = $imageObjectDetection->getBoundingBox()
            ->getNormalizedVertices();
        print('Normalized bounding box vertices: ');
        foreach ($vertices as $vertex) {
            printf(' (%f, %f)', $vertex->getX(), $vertex->getY());
        }
        print(PHP_EOL);
    }
} finally {
    $client->close();
}

Python

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

from google.cloud import automl

# TODO(developer): Uncomment and set the following variables
# project_id = "YOUR_PROJECT_ID"
# model_id = "YOUR_MODEL_ID"
# file_path = "path_to_local_file.jpg"

prediction_client = automl.PredictionServiceClient()

# Get the full path of the model.
model_full_id = automl.AutoMlClient.model_path(
    project_id, "us-central1", model_id
)

# Read the file.
with open(file_path, "rb") as content_file:
    content = content_file.read()

image = automl.Image(image_bytes=content)
payload = automl.ExamplePayload(image=image)

# params is additional domain-specific parameters.
# score_threshold is used to filter the result
# https://cloud.google.com/automl/docs/reference/rpc/google.cloud.automl.v1#predictrequest
params = {"score_threshold": "0.8"}

request = automl.PredictRequest(
    name=model_full_id,
    payload=payload,
    params=params
)

response = prediction_client.predict(request=request)
print("Prediction results:")
for result in response.payload:
    print("Predicted class name: {}".format(result.display_name))
    print(
        "Predicted class score: {}".format(
            result.image_object_detection.score
        )
    )
    bounding_box = result.image_object_detection.bounding_box
    print("Normalized Vertices:")
    for vertex in bounding_box.normalized_vertices:
        print("\tX: {}, Y: {}".format(vertex.x, vertex.y))

Ruby

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

require "google/cloud/automl"

project_id = "YOUR_PROJECT_ID"
model_id = "YOUR_MODEL_ID"
file_path = "path_to_local_file.txt"

prediction_client = Google::Cloud::AutoML.prediction_service

# Get the full path of the model.
model_full_id = prediction_client.model_path project: project_id,
                                             location: "us-central1",
                                             model: model_id

# Read the file.
content = File.binread file_path
payload = {
  image: {
    image_bytes: content
  }
}
# params is additional domain-specific parameters.
# score_threshold is used to filter the result
params = { "score_threshold" => "0.8" }

response = prediction_client.predict name: model_full_id,
                                     payload: payload,
                                     params: params

puts "Prediction results:"
response.payload.each do |result|
  puts "Predicted class name: #{result.display_name}"
  puts "Predicted class score: #{result.image_object_detection.score}"
  bounding_box = result.image_object_detection.bounding_box
  puts "Normalized Vertices:"
  bounding_box.normalized_vertices.each do |vertex|
    puts "\tX: #{vertex.x}, Y: #{vertex.y}"
  end
end