Annotating images

After you have created (trained) a model, you can request a prediction for an image using the predict method. The predict method applies labels to your image based on the primary object of the image that your model predicts.

The maximum lifespan for a custom model is two years. You must create and train a new model to continue classifying content after that amount of time.

Using curl

To make it more convenient to run the curl samples in this topic, set the following environment variable. Replace project-id with the name of your GCP project.

export PROJECT_ID="project-id"

Predict Example

Web UI

  1. Open the AutoML Vision UI and click the lightbulb icon in the left navigation bar to display the available models.

    To view the models for a different project, select the project from the drop-down list in the upper right of the title bar.

  2. Click the row for the model you want to use to label your images.

  3. Click the Predict tab just below the title bar.

  4. Click Upload Images to upload the images that you want to label.


To make a prediction for an image, supply the contents of your image in the imageBytes field. The contents of your image must be base64-encoded. For information on base64-encoding the contents of an image, see Base64 Encoding.

The following example requests a prediction for an image.

  • Replace model-id with the ID of your model. The ID is the last element of the name of your model. For example, if the name of your model is projects/434039606874/locations/us-central1/models/3745331181667467569, then the ID of your model is 3745331181667467569.
curl -X POST \
  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \${PROJECT_ID}/locations/us-central1/models/model-id:predict \
  -d '{
        "payload" : {
          "image": {
            "imageBytes" : "/9j/4AAQSkZJRgABAQAAAQ … "

You should see output similar to the following:

  "payload": [
      "displayName": "Lily",
      "classification": {
        "score": 0.8989502
      "displayName": "Daisy",
      "classification": {
        "score": 0.10098731
      "displayName": "--other--",
      "classification": {
        "score": 6.2490464e-05

The predictions from your AutoML Vision model are contained in the payload field.

  • displayName is the label predicted by the AutoML Vision model.
  • score represents a confidence level that the specified label applies to the image. It ranges from 0 (no confidence) to 1 (high confidence).


Before you can run this code example, you must install the Python Client Libraries.

  • The model_full_id parameter is the full name of your model. For example: projects/434039606874/locations/us-central1/models/3745331181667467569.
# TODO(developer): Uncomment and set the following variables
# project_id = 'PROJECT_ID_HERE'
# compute_region = 'COMPUTE_REGION_HERE'
# model_id = 'MODEL_ID_HERE'
# file_path = '/local/path/to/file'
# score_threshold = 'value from 0.0 to 0.5'

from import automl_v1beta1 as automl

automl_client = automl.AutoMlClient()

# Get the full path of the model.
model_full_id = automl_client.model_path(
    project_id, compute_region, model_id

# Create client for prediction service.
prediction_client = automl.PredictionServiceClient()

# Read the image and assign to payload.
with open(file_path, "rb") as image_file:
    content =
payload = {"image": {"image_bytes": content}}

# params is additional domain-specific parameters.
# score_threshold is used to filter the result
# Initialize params
params = {}
if score_threshold:
    params = {"score_threshold": score_threshold}

response = prediction_client.predict(model_full_id, payload, params)
print("Prediction results:")
for result in response.payload:
    print("Predicted class name: {}".format(result.display_name))
    print("Predicted class score: {}".format(result.classification.score))


 * Demonstrates using the AutoML client to predict an image.
 * @param projectId the Id of the project.
 * @param computeRegion the Region name.
 * @param modelId the Id of the model which will be used for text classification.
 * @param filePath the Local text file path of the content to be classified.
 * @param scoreThreshold the Confidence score. Only classifications with confidence score above
 *     scoreThreshold are displayed.
 * @throws IOException on Input/Output errors.
public static void predict(
    String projectId,
    String computeRegion,
    String modelId,
    String filePath,
    String scoreThreshold)
    throws IOException {

  // Instantiate client for prediction service.
  PredictionServiceClient predictionClient = PredictionServiceClient.create();

  // Get the full path of the model.
  ModelName name = ModelName.of(projectId, computeRegion, modelId);

  // Read the image and assign to payload.
  ByteString content = ByteString.copyFrom(Files.readAllBytes(Paths.get(filePath)));
  Image image = Image.newBuilder().setImageBytes(content).build();
  ExamplePayload examplePayload = ExamplePayload.newBuilder().setImage(image).build();

  // Additional parameters that can be provided for prediction e.g. Score Threshold
  Map<String, String> params = new HashMap<>();
  if (scoreThreshold != null) {
    params.put("score_threshold", scoreThreshold);
  // Perform the AutoML Prediction request
  PredictResponse response = predictionClient.predict(name, examplePayload, params);

  System.out.println("Prediction results:");
  for (AnnotationPayload annotationPayload : response.getPayloadList()) {
    System.out.println("Predicted class name :" + annotationPayload.getDisplayName());
        "Predicted class score :" + annotationPayload.getClassification().getScore());


  const automl = require('@google-cloud/automl').v1beta1;
  const fs = require('fs');

  // Create client for prediction service.
  const client = new automl.PredictionServiceClient();

   * TODO(developer): Uncomment the following line before running the sample.
  // const projectId = `The GCLOUD_PROJECT string, e.g. "my-gcloud-project"`;
  // const computeRegion = `region-name, e.g. "us-central1"`;
  // const modelId = `id of the model, e.g. “ICN12345”`;
  // const filePath = `local text file path of content to be classified, e.g. "./resources/test.txt"`;
  // const scoreThreshold = `value between 0.0 and 1.0, e.g. "0.5"';

  // Get the full path of the model.
  const modelFullId = client.modelPath(projectId, computeRegion, modelId);

  // Read the file content for prediction.
  const content = fs.readFileSync(filePath, 'base64');

  const params = {};

  if (scoreThreshold) {
    params.scoreThreshold = scoreThreshold;

  // Set the payload by giving the content and type of the file.
  const payload = {};
  payload.image = {imageBytes: content};

  // params is additional domain-specific parameters.
  // currently there is no additional parameters supported.
    .predict({name: modelFullId, payload: payload, params: params})
    .then(responses => {
      console.log(`Prediction results:`);
      responses[0].payload.forEach(result => {
        console.log(`Predicted class name: ${result.displayName}`);
        console.log(`Predicted class score: ${result.classification.score}`);
    .catch(err => {

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud AutoML Vision
Need help? Visit our support page.