Managing models

A custom model is trained using a prepared dataset that you provide. AutoML Video Intelligence Classification uses the items from the your dataset to train, test, and evaluate the model's performance. Next, you should review the results, adjust the training dataset as needed, and train a new model using the improved dataset.

Training a model can take several hours to complete. The AutoML API enables you to check the status of training.

Since AutoML Video Intelligence Classification creates a new model each time you start training, your project may include numerous models. You can get a list of the models in your project and can delete models you no longer need.

The maximum lifespan for a custom model is two years. You must create and train a new model to continue classifying content after that amount of time.

Using curl or PowerShell

To make it more convenient to run the curl (or PowerShell) samples in this topic, set the following environment variable. Replace project-id with the name of your GCP project.

export PROJECT_ID="project-id"

Training models

When you have a dataset with a solid set of labeled training items, you are ready to create and train the custom model.

Web UI

  1. Open the AutoML Video Classification UI and navigate to the Datasets page.

    Datasets page in Google Cloud Console
  2. Select the dataset that you want to use to train the custom model.

    The display name of the selected dataset appears in the title bar, and the page lists the individual items in the dataset along with their labels.

    Videos tab with two videos shown
  3. When you are done reviewing the dataset, click the Train tab just below the title bar.

    The training page provides a basic analysis of your dataset and advises you about whether it is adequate for training. If AutoML Video Classification suggests changes, consider returning to the Videos page and adding items or labels.

  4. When the dataset is ready, click Start Training to create a new model or Train New Model if you want to create an additional model.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • dataset-id: The ID is the last element of the name of your dataset. For example, if the name of your dataset is projects/434039606874/locations/us-central1/datasets/VCN3104518874390609379, then the ID of your dataset is VCN3104518874390609379.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

Request JSON body:

{
  "displayName": "test_model",
  "dataset_id": "dataset-id",
  "videoClassificationModelMetadata": {}
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models" | Select-Object -Expand Content
The operation-id is provided in the response when you started the operation, for example VCN123....
{
  "name": "projects/project-number/locations/location-id/operations/operation-id",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.automl.v1beta1.OperationMetadata",
    "progressPercentage": 100,
    "createTime": "2020-02-27T01:56:28.395640Z",
    "updateTime": "2020-02-27T02:04:12.336070Z"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.automl.v1beta1.Model",
    "name": "projects/project-number/locations/location-id/models/operation-id",
    "createTime": "2020-02-27T02:00:22.329970Z",
    "videoClassificationModelMetadata": {
      "trainBudget": "1",
      "trainCost": "1",
      "stopReason": "BUDGET_REACHED"
    },
    "displayName": "a_98487760535e48319dd204e6394670"
  }
}

Java

import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.automl.v1beta1.AutoMlClient;
import com.google.cloud.automl.v1beta1.LocationName;
import com.google.cloud.automl.v1beta1.Model;
import com.google.cloud.automl.v1beta1.OperationMetadata;
import com.google.cloud.automl.v1beta1.VideoClassificationModelMetadata;
import java.io.IOException;
import java.util.concurrent.ExecutionException;

class VideoClassificationCreateModel {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    String datasetId = "YOUR_DATASET_ID";
    String displayName = "YOUR_DATASET_NAME";
    createModel(projectId, datasetId, displayName);
  }

  // Create a model
  static void createModel(String projectId, String datasetId, String displayName)
      throws IOException, ExecutionException, InterruptedException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
      // A resource that represents Google Cloud Platform location.
      LocationName projectLocation = LocationName.of(projectId, "us-central1");
      // Set model metadata.
      VideoClassificationModelMetadata metadata =
          VideoClassificationModelMetadata.newBuilder().build();
      Model model =
          Model.newBuilder()
              .setDisplayName(displayName)
              .setDatasetId(datasetId)
              .setVideoClassificationModelMetadata(metadata)
              .build();

      // Create a model with the model metadata in the region.
      OperationFuture<Model, OperationMetadata> future =
          client.createModelAsync(projectLocation, model);
      // OperationFuture.get() will block until the model is created, which may take several hours.
      // You can use OperationFuture.getInitialFuture to get a future representing the initial
      // response to the request, which contains information while the operation is in progress.
      System.out.format("Training operation name: %s%n", future.getInitialFuture().get().getName());
      System.out.println("Training started...");
    }
  }
}

Node.js

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const dataset_id = 'YOUR_DATASET_ID';
// const displayName = 'YOUR_DISPLAY_NAME';

// Imports the Google Cloud AutoML library
const {AutoMlClient} = require('@google-cloud/automl').v1beta1;

// Instantiates a client
const client = new AutoMlClient();

async function createModel() {
  // Construct request
  const request = {
    parent: client.locationPath(projectId, location),
    model: {
      displayName: displayName,
      datasetId: datasetId,
      videoClassificationModelMetadata: {},
    },
  };

  // Don't wait for the LRO
  const [operation] = await client.createModel(request);
  console.log(`Training started... ${operation}`);
  console.log(`Training operation name: ${operation.name}`);
}

createModel();

Python

from google.cloud import automl_v1beta1 as automl


def create_model(
    project_id="YOUR_PROJECT_ID",
    dataset_id="YOUR_DATASET_ID",
    display_name="your_models_display_name",
):
    """Create a automl video classification model."""
    client = automl.AutoMlClient()

    # A resource that represents Google Cloud Platform location.
    project_location = f"projects/{project_id}/locations/us-central1"
    # Leave model unset to use the default base model provided by Google
    metadata = automl.VideoClassificationModelMetadata()
    model = automl.Model(
        display_name=display_name,
        dataset_id=dataset_id,
        video_classification_model_metadata=metadata,
    )

    # Create a model with the model metadata in the region.
    response = client.create_model(parent=project_location, model=model)

    print("Training operation name: {}".format(response.operation.name))
    print("Training started...")

Getting information about a model

When training is complete, you can get information about the newly created model.

The examples in this section return the basic metadata about a model. To get details about a model's accuracy and readiness, see Evaluating models.

Web UI

  1. Navigate to the Models page in the AutoML Video Classification UI.

    Models page with one model shown
  2. Click the name of the model that you want to view.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • model-name: The full name of your model provided by the response when you created the model. The full name has the format: projects/project-number/locations/location-id/models/model-id
  • dataset-id: replace with the dataset identifier for your dataset (not the display name). For example: VCN3940649673949184000

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/model-name/modelEvaluations

Request JSON body:

{
  "displayName": "test_model",
  "dataset_id": "dataset-id",
  "videoClassificationModelMetadata": {}
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Java

import com.google.cloud.automl.v1beta1.AutoMlClient;
import com.google.cloud.automl.v1beta1.Model;
import com.google.cloud.automl.v1beta1.ModelName;
import io.grpc.StatusRuntimeException;
import java.io.IOException;

class GetModel {

  static void getModel() throws IOException, StatusRuntimeException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    String modelId = "YOUR_MODEL_ID";
    getModel(projectId, modelId);
  }

  // Get a model
  static void getModel(String projectId, String modelId)
      throws IOException, StatusRuntimeException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
      // Get the full path of the model.
      ModelName modelFullId = ModelName.of(projectId, "us-central1", modelId);
      Model model = client.getModel(modelFullId);

      // Display the model information.
      System.out.format("Model name: %s%n", model.getName());
      // To get the model id, you have to parse it out of the `name` field. As models Ids are
      // required for other methods.
      // Name Format: `projects/{project_id}/locations/{location_id}/models/{model_id}`
      String[] names = model.getName().split("/");
      String retrievedModelId = names[names.length - 1];
      System.out.format("Model id: %s%n", retrievedModelId);
      System.out.format("Model display name: %s%n", model.getDisplayName());
      System.out.println("Model create time:");
      System.out.format("\tseconds: %s%n", model.getCreateTime().getSeconds());
      System.out.format("\tnanos: %s%n", model.getCreateTime().getNanos());
      System.out.format("Model deployment state: %s%n", model.getDeploymentState());
    }
  }
}

Node.js

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const modelId = 'YOUR_MODEL_ID';

// Imports the Google Cloud AutoML library
const {AutoMlClient} = require('@google-cloud/automl').v1beta1;

// Instantiates a client
const client = new AutoMlClient();

async function getModel() {
  // Construct request
  const request = {
    name: client.modelPath(projectId, location, modelId),
  };

  const [response] = await client.getModel(request);

  console.log(`Model name: ${response.name}`);
  console.log(
    `Model id: ${
      response.name.split('/')[response.name.split('/').length - 1]
    }`
  );
  console.log(`Model display name: ${response.displayName}`);
  console.log('Model create time');
  console.log(`\tseconds ${response.createTime.seconds}`);
  console.log(`\tnanos ${response.createTime.nanos / 1e9}`);
  console.log(`Model deployment state: ${response.deploymentState}`);
}

getModel();

Python

from google.cloud import automl_v1beta1 as automl

# TODO(developer): Uncomment and set the following variables
# project_id = "YOUR_PROJECT_ID"
# model_id = "YOUR_MODEL_ID"

client = automl.AutoMlClient()
# Get the full path of the model.
model_full_id = client.model_path(project_id, "us-central1", model_id)
model = client.get_model(name=model_full_id)

# Retrieve deployment state.
if model.deployment_state == automl.Model.DeploymentState.DEPLOYED:
    deployment_state = "deployed"
else:
    deployment_state = "undeployed"

# Display the model information.
print("Model name: {}".format(model.name))
print("Model id: {}".format(model.name.split("/")[-1]))
print("Model display name: {}".format(model.display_name))
print("Model create time: {}".format(model.create_time))
print("Model deployment state: {}".format(deployment_state))

Listing models

A project can include numerous models. This section describes how to retrieve a list of the available models for a project.

Web UI

Navigate to the Models page in the AutoML Video Classification UI to see the models in your project.

Models page with one model in list

To see the models for a different project, select the project from the drop-down list in the upper right of the title bar.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • model-name: The full name of your model provided by the response when you created the model. The full name has the format: projects/project-number/locations/location-id/models
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Java


import com.google.cloud.automl.v1beta1.AutoMlClient;
import com.google.cloud.automl.v1beta1.AutoMlSettings;
import com.google.cloud.automl.v1beta1.ListModelsRequest;
import com.google.cloud.automl.v1beta1.LocationName;
import com.google.cloud.automl.v1beta1.Model;
import java.io.IOException;
import org.threeten.bp.Duration;

class ListModels {

  static void listModels() throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    listModels(projectId);
  }

  // List the models available in the specified location
  static void listModels(String projectId) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    AutoMlSettings.Builder autoMlSettingsBuilder = AutoMlSettings.newBuilder();

    autoMlSettingsBuilder
        .listModelsSettings()
        .setRetrySettings(
            autoMlSettingsBuilder
                .listModelsSettings()
                .getRetrySettings()
                .toBuilder()
                .setTotalTimeout(Duration.ofSeconds(30))
                .build());
    AutoMlSettings autoMlSettings = autoMlSettingsBuilder.build();

    try (AutoMlClient client = AutoMlClient.create(autoMlSettings)) {
      // A resource that represents Google Cloud Platform location.
      LocationName projectLocation = LocationName.of(projectId, "us-central1");

      // Create list models request.
      ListModelsRequest listModelsRequest =
          ListModelsRequest.newBuilder()
              .setParent(projectLocation.toString())
              .setFilter("")
              .build();

      // List all the models available in the region by applying filter.
      System.out.println("List of models:");
      for (Model model : client.listModels(listModelsRequest).iterateAll()) {
        // Display the model information.
        System.out.format("Model name: %s%n", model.getName());
        // To get the model id, you have to parse it out of the `name` field. As models Ids are
        // required for other methods.
        // Name Format: `projects/{project_id}/locations/{location_id}/models/{model_id}`
        String[] names = model.getName().split("/");
        String retrievedModelId = names[names.length - 1];
        System.out.format("Model id: %s%n", retrievedModelId);
        System.out.format("Model display name: %s%n", model.getDisplayName());
        System.out.println("Model create time:");
        System.out.format("\tseconds: %s%n", model.getCreateTime().getSeconds());
        System.out.format("\tnanos: %s%n", model.getCreateTime().getNanos());
        System.out.format("Model deployment state: %s%n", model.getDeploymentState());
      }
    }
  }
}

Node.js

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';

// Imports the Google Cloud AutoML library
const {AutoMlClient} = require('@google-cloud/automl').v1beta1;

// Instantiates a client
const client = new AutoMlClient();

async function listModels() {
  // Construct request
  const request = {
    parent: client.locationPath(projectId, location),
    filter: 'translation_model_metadata:*',
  };

  const [response] = await client.listModels(request);

  console.log('List of models:');
  for (const model of response) {
    console.log(`Model name: ${model.name}`);
    console.log(`
      Model id: ${model.name.split('/')[model.name.split('/').length - 1]}`);
    console.log(`Model display name: ${model.displayName}`);
    console.log('Model create time');
    console.log(`\tseconds ${model.createTime.seconds}`);
    console.log(`\tnanos ${model.createTime.nanos / 1e9}`);
    console.log(`Model deployment state: ${model.deploymentState}`);
  }
}

listModels();

Python

from google.cloud import automl_v1beta1 as automl

# TODO(developer): Uncomment and set the following variables
# project_id = "YOUR_PROJECT_ID"

client = automl.AutoMlClient()
# A resource that represents Google Cloud Platform location.
project_location = f"projects/{project_id}/locations/us-central1"
request = automl.ListModelsRequest(parent=project_location, filter="")
response = client.list_models(request=request)

print("List of models:")
for model in response:
    # Display the model information.
    if (
        model.deployment_state
        == automl.Model.DeploymentState.DEPLOYED
    ):
        deployment_state = "deployed"
    else:
        deployment_state = "undeployed"

    print("Model name: {}".format(model.name))
    print("Model id: {}".format(model.name.split("/")[-1]))
    print("Model display name: {}".format(model.display_name))
    print("Model create time: {}".format(model.create_time))
    print("Model deployment state: {}".format(deployment_state))

Deleting a model

The following example deletes a model.

Web UI

  1. Navigate to the Models page in the AutoML Video Classification UI. Models page with one model shown
  2. Click the three-dot menu at the far right of the row you want to delete and select Delete.
  3. Click Confirm in the confirmation dialog box.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • model-id: replace with the identifier for your model
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

DELETE https://automl.googleapis.com/v1beta1/projects/project-number/locations/test/models/model-id

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Java

import com.google.cloud.automl.v1beta1.AutoMlClient;
import com.google.cloud.automl.v1beta1.ModelName;
import com.google.protobuf.Empty;
import java.io.IOException;
import java.util.concurrent.ExecutionException;

class DeleteModel {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "YOUR_PROJECT_ID";
    String modelId = "YOUR_MODEL_ID";
    deleteModel(projectId, modelId);
  }

  // Delete a model
  static void deleteModel(String projectId, String modelId)
      throws IOException, ExecutionException, InterruptedException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
      // Get the full path of the model.
      ModelName modelFullId = ModelName.of(projectId, "us-central1", modelId);

      // Delete a model.
      Empty response = client.deleteModelAsync(modelFullId).get();

      System.out.println("Model deletion started...");
      System.out.println(String.format("Model deleted. %s", response));
    }
  }
}

Node.js

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// const projectId = 'YOUR_PROJECT_ID';
// const location = 'us-central1';
// const modelId = 'YOUR_MODEL_ID';

// Imports the Google Cloud AutoML library
const {AutoMlClient} = require('@google-cloud/automl').v1beta1;

// Instantiates a client
const client = new AutoMlClient();

async function deleteModel() {
  // Construct request
  const request = {
    name: client.modelPath(projectId, location, modelId),
  };

  const [response] = await client.deleteModel(request);
  console.log(`Model deleted: ${response}`);
}

deleteModel();

Python

from google.cloud import automl_v1beta1 as automl

# TODO(developer): Uncomment and set the following variables
# project_id = "YOUR_PROJECT_ID"
# model_id = "YOUR_MODEL_ID"

client = automl.AutoMlClient()
# Get the full path of the model.
model_full_id = client.model_path(project_id, "us-central1", model_id)
response = client.delete_model(name=model_full_id)

print("Model deleted. {}".format(response.result()))