Making individual predictions

After you have created (trained) a model, you can request a prediction for an image using the predict method. The predict method applies labels to your image based on the primary object of the image that your model predicts.

Online (individual) prediction

This section describes sending an individual file for annotation. This request will immediately return a response.

You can also send a batch of files for annotation. Batch file annotation is a long-running operation, and stores results in a Cloud Storage bucket of your choosing.

Web UI

  1. Open the Vision Dashboard and click the lightbulb icon in the left navigation bar to display the available models.

    To view the models for a different project, select the project from the drop-down list in the upper right of the title bar.

  2. Click the row for the model you want to use to label your images.

  3. Click the Test & Use tab just below the title bar.

  4. Click Upload Images to upload the images that you want to label.

    updated predict page

REST & CMD LINE

To test prediction you must first deploy your Cloud-hosted model.

Before using any of the request data below, make the following replacements:

  • project-id: your GCP project ID.
  • model-id: the ID of your model, from the response when you created the model. The ID is the last element of the name of your model. For example:
    • model name: projects/project-id/locations/location-id/models/IOD4412217016962778756
    • model id: IOD4412217016962778756
  • base64-encoded-image: the base64 representation (ASCII string) of your binary image data. This string should look similar to the following string: /9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==. Visit the base64 encode topic for more information.

Field-specific considerations:

  • scoreThreshold - A value from 0 to 1. Only values with score thresholds of at least this value will be displayed. The default value is 0.5.

HTTP method and URL:

POST https://automl.googleapis.com/v1/projects/project-id/locations/us-central1/models/model-id:predict

Request JSON body:

{
  "payload": {
    "image": {
      "imageBytes": "base64-encoded-image"
    }
  },
  "params": {
    "scoreThreshold": "0.5"
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://automl.googleapis.com/v1/projects/project-id/locations/us-central1/models/model-id:predict

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://automl.googleapis.com/v1/projects/project-id/locations/us-central1/models/model-id:predict" | Select-Object -Expand Content

Output is returned in JSON form. The predictions from your AutoML Vision model are contained in the payload field:

  • displayName is the object's label predicted by the AutoML Vision model.
  • score represents a confidence level that the specified label applies to the image. It ranges from 0 (no confidence) to 1 (high confidence).
{
  "payload": [
    {
      "annotationSpecId": "7922029656637702144",
      "classification": {
        "score": 0.9960259
      },
      "displayName": "roses"
    }
  ]
}

Go

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

import (
	"context"
	"fmt"
	"io"
	"io/ioutil"
	"os"

	automl "cloud.google.com/go/automl/apiv1"
	automlpb "google.golang.org/genproto/googleapis/cloud/automl/v1"
)

// visionClassificationPredict does a prediction for image classification.
func visionClassificationPredict(w io.Writer, projectID string, location string, modelID string, filePath string) error {
	// projectID := "my-project-id"
	// location := "us-central1"
	// modelID := "ICN123456789..."
	// filePath := "path/to/image.jpg"

	ctx := context.Background()
	client, err := automl.NewPredictionClient(ctx)
	if err != nil {
		return fmt.Errorf("NewPredictionClient: %v", err)
	}
	defer client.Close()

	file, err := os.Open(filePath)
	if err != nil {
		return fmt.Errorf("Open: %v", err)
	}
	defer file.Close()
	bytes, err := ioutil.ReadAll(file)
	if err != nil {
		return fmt.Errorf("ReadAll: %v", err)
	}

	req := &automlpb.PredictRequest{
		Name: fmt.Sprintf("projects/%s/locations/%s/models/%s", projectID, location, modelID),
		Payload: &automlpb.ExamplePayload{
			Payload: &automlpb.ExamplePayload_Image{
				Image: &automlpb.Image{
					Data: &automlpb.Image_ImageBytes{
						ImageBytes: bytes,
					},
				},
			},
		},
		// Params is additional domain-specific parameters.
		Params: map[string]string{
			// score_threshold is used to filter the result.
			"score_threshold": "0.8",
		},
	}

	resp, err := client.Predict(ctx, req)
	if err != nil {
		return fmt.Errorf("Predict: %v", err)
	}

	for _, payload := range resp.GetPayload() {
		fmt.Fprintf(w, "Predicted class name: %v\n", payload.GetDisplayName())
		fmt.Fprintf(w, "Predicted class score: %v\n", payload.GetClassification().GetScore())
	}

	return nil
}

Java

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

import com.google.cloud.automl.v1.AnnotationPayload;
import com.google.cloud.automl.v1.ExamplePayload;
import com.google.cloud.automl.v1.Image;
import com.google.cloud.automl.v1.ModelName;
import com.google.cloud.automl.v1.PredictRequest;
import com.google.cloud.automl.v1.PredictResponse;
import com.google.cloud.automl.v1.PredictionServiceClient;
import com.google.protobuf.ByteString;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

class VisionClassificationPredict {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    String modelId = "YOUR_MODEL_ID";
    String filePath = "path_to_local_file.jpg";
    predict(projectId, modelId, filePath);
  }

  static void predict(String projectId, String modelId, String filePath) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (PredictionServiceClient client = PredictionServiceClient.create()) {
      // Get the full path of the model.
      ModelName name = ModelName.of(projectId, "us-central1", modelId);
      ByteString content = ByteString.copyFrom(Files.readAllBytes(Paths.get(filePath)));
      Image image = Image.newBuilder().setImageBytes(content).build();
      ExamplePayload payload = ExamplePayload.newBuilder().setImage(image).build();
      PredictRequest predictRequest =
          PredictRequest.newBuilder()
              .setName(name.toString())
              .setPayload(payload)
              .putParams(
                  "score_threshold", "0.8") // [0.0-1.0] Only produce results higher than this value
              .build();

      PredictResponse response = client.predict(predictRequest);

      for (AnnotationPayload annotationPayload : response.getPayloadList()) {
        System.out.format("Predicted class name: %s\n", annotationPayload.getDisplayName());
        System.out.format(
            "Predicted class score: %.2f\n", annotationPayload.getClassification().getScore());
      }
    }
  }
}

Node.js

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const modelId = 'YOUR_MODEL_ID';
// const filePath = 'path_to_local_file.jpg';

// Imports the Google Cloud AutoML library
const {PredictionServiceClient} = require('@google-cloud/automl').v1;
const fs = require('fs');

// Instantiates a client
const client = new PredictionServiceClient();

// Read the file content for translation.
const content = fs.readFileSync(filePath);

async function predict() {
  // Construct request
  // params is additional domain-specific parameters.
  // score_threshold is used to filter the result
  const request = {
    name: client.modelPath(projectId, location, modelId),
    payload: {
      image: {
        imageBytes: content,
      },
    },
  };

  const [response] = await client.predict(request);

  for (const annotationPayload of response.payload) {
    console.log(`Predicted class name: ${annotationPayload.displayName}`);
    console.log(
      `Predicted class score: ${annotationPayload.classification.score}`
    );
  }
}

predict();

PHP

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

use Google\Cloud\AutoMl\V1\ExamplePayload;
use Google\Cloud\AutoMl\V1\Image;
use Google\Cloud\AutoMl\V1\PredictionServiceClient;

/** Uncomment and populate these variables in your code */
// $projectId = '[Google Cloud Project ID]';
// $location = 'us-central1';
// $modelId = 'my_model_id_123';
// $filePath = 'path_to_local_file.jpg';

$client = new PredictionServiceClient();

try {
    // get full path of model
    $formattedName = $client->modelName(
        $projectId,
        $location,
        $modelId);

    // read the file
    $content = file_get_contents($filePath);
    $image = (new Image())
        ->setImageBytes($content);
    // create payload
    $payload = (new ExamplePayload())
        ->setImage($image);

    // params is additional domain-specific parameters
    // score_threshold is used to filter the result
    $params = ['score_threshold' => '0.8']; // value between 0.0 and 1.0

    // predict with above model and payload
    $response = $client->predict($formattedName, $payload, $params);
    $annotations = $response->getPayload();

    // display results
    foreach ($annotations as $annotation) {
        $classification = $annotation->getClassification();
        printf('Predicted class name: %s' . PHP_EOL, $annotation->getDisplayName());
        printf('Predicted class score: %s' . PHP_EOL, $classification->getScore());
    }
} finally {
    $client->close();
}

Python

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

from google.cloud import automl

# TODO(developer): Uncomment and set the following variables
# project_id = "YOUR_PROJECT_ID"
# model_id = "YOUR_MODEL_ID"
# file_path = "path_to_local_file.jpg"

prediction_client = automl.PredictionServiceClient()

# Get the full path of the model.
model_full_id = automl.AutoMlClient.model_path(project_id, "us-central1", model_id)

# Read the file.
with open(file_path, "rb") as content_file:
    content = content_file.read()

image = automl.Image(image_bytes=content)
payload = automl.ExamplePayload(image=image)

# params is additional domain-specific parameters.
# score_threshold is used to filter the result
# https://cloud.google.com/automl/docs/reference/rpc/google.cloud.automl.v1#predictrequest
params = {"score_threshold": "0.8"}

request = automl.PredictRequest(name=model_full_id, payload=payload, params=params)
response = prediction_client.predict(request=request)

print("Prediction results:")
for result in response.payload:
    print("Predicted class name: {}".format(result.display_name))
    print("Predicted class score: {}".format(result.classification.score))

Ruby

Before trying this sample, follow the setup instructions for this language on the Client Libraries page.

require "google/cloud/automl"

project_id = "YOUR_PROJECT_ID"
model_id = "YOUR_MODEL_ID"
file_path = "path_to_local_file.jpg"

prediction_client = Google::Cloud::AutoML.prediction_service

# Get the full path of the model.
model_full_id = prediction_client.model_path project: project_id,
                                             location: "us-central1",
                                             model: model_id

# Read the file.
content = File.binread file_path
payload = {
  image: {
    image_bytes: content
  }
}
# params is additional domain-specific parameters.
# score_threshold is used to filter the result
params = { "score_threshold" => "0.8" }

response = prediction_client.predict name: model_full_id,
                                     payload: payload,
                                     params: params

puts "Prediction results:"
response.payload.each do |result|
  puts "Predicted class name: #{result.display_name}"
  puts "Predicted class score: #{result.classification.score}"
end