Track video objects by using the Google Cloud console

This quickstart walks you through the process of tracking objects in videos using an AutoML model.

Enable the API

Create your dataset

In the Datasets page of the AutoML Video Object Tracking UI, do the following:

  1. Click Create Dataset.
    Create dataset icon.
  2. Specify a name for this dataset, select Video Object Tracking, and then click Create Dataset.
  3. On the Import tab for your dataset, provide the Cloud Storage URI of the CSV file that contains the URIs of your training data. For this quickstart, use the following CSV file. Note that the following URI intentionally doesn't include the gs:// prefix.


  4. Also on the Import tab for your dataset, click Continue to begin importing your data. Page for dataset titled 'my_dataset'

The import process can take a while to complete. How long this process takes depends on the number and length of the videos that you've provided.

Once completed, click the Videos tab for the dataset to browse the list of videos in the dataset:

Videos tab with two videos shown

To view any errors that occurred during the import process, switch to the Import tab to check error messages.

Train your model

  • After your dataset has been created and processed, click the Train tab to start model training.
  • Click Start Training to continue. Train tab with information
  • In the Train new model pane, select a name for your model and click Start Training.

Training is initiated for your model. For this dataset, the training process can take up to 1 hour. The service emails you once training has completed, or if any errors occur.

Once training is complete, the service automatically deploys your model.

Click the Evaluate tab to get more details on F1, Precision, and Recall scores.

Evaluate tab with information

Tracking an object in a video

To make a prediction using your model, that is, to track an object in a video, do the following:

  1. On the Test & Use tab for the model, do the following:
    • Under Test your model, type 'gs://automl-video-demo-data/traffic_videos/traffic_videos_batch_predict.csv'.
    • Also under Test your model, select a directory within your Cloud Storage bucket to receive the object tracking results. Be sure to select a regional bucket for the results.

      You may actually want to create a specific 'results' folder in your Cloud Storage bucket to hold the prediction results.

    • Click Get Predictions.
    Configuring a prediction request for AutoML Video Intelligence

The process for getting predictions can take some time, depending on the number of videos that you want to track objects in.

When the process has completed, the results appear on the page for the model under Recent Predictions. To view the results, do the following:

  1. Under Recent Predictions in the Predictions column, click View for the prediction you want to look at.
  2. Under Video, select the name of the video you want to see the results for.

Results of AutoML Video Intelligence prediction

View the results

In the results, AutoML Video Object Tracking provides three types of information:

  • Labels for the video. You can find this information in the rows below the preview of the video.
  • Time period when the object is present in the video. This is indicated on the timeline of the video.
  • Confidence score for the prediction.

If you want to see more labels or bounding boxes, you can change the threshold score when you request a prediction. AutoML Video Object Tracking only displays the labels that are above the specified threshold.

If the prediction fails, the results in the list show a red icon on the Recent Predictions list.

If only one video in the prediction attempt failed, the prediction shows green in the Recent Predictions list. On the results page for that prediction, you can see the results for the videos that AutoML Video Object Tracking successfully tracked objects.