Quickstart: Using the command line for video object tracking

This quickstart walks you through the process of:

  • Copying a set of videos into Cloud Storage.
  • Creating CSV files that list videos and their labels.
  • Using AutoML Video Object Tracking to create your dataset to train and use your model.

Before you begin

Set up your project

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Enable the AutoML and Cloud Storage APIs.

    Enable the APIs

  5. Install the gcloud command line tool.
  6. Follow the instructions to create a service account and download a key file for that account.
  7. Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to the service account key file that you downloaded when you created the service account.
    export GOOGLE_APPLICATION_CREDENTIALS=key-file
  8. Set the PROJECT_ID environment variable to your Project ID.
    export PROJECT_ID=your-project-id
    The AutoML API calls and resource names include your Project ID in them. The PROJECT_ID environment variable provides a convenient way to specify the ID.
  9. If you are an owner for your project, add your service account to the AutoML Editor IAM role, replacing service-account-name with the name of your new service account. For example, service-account1@myproject.iam.gserviceaccount.com.
    gcloud auth login
    gcloud projects add-iam-policy-binding $PROJECT_ID \
       --member="serviceAccount:service-account-name" \
       --role="roles/automl.editor"
    
  10. Otherwise (if you are not a project owner), ask a project owner to add both your user ID and your service account to the AutoML Editor IAM role.

Create a dataset and import training data

Create a dataset

Decide on a name for your dataset and use the following curl or PowerShell commands to create a new dataset with that name.

REST & CMD LINE

The following shows how to send a POST request. The example uses the access token for a service account set up for the project using the Cloud SDK. For instructions on installing the Cloud SDK, setting up a project with a service account, and obtaining an access token, see the AutoML Video Intelligence Object Tracking Quickstart.

Before using any of the request data below, make the following replacements:

  • dataset-name: the name of your target dataset.
    For example, my_dataset_01
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

Request JSON body:

{
    "displayName": "dataset-name",
    "videoObjectTrackingDatasetMetadata": { }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets" | Select-Object -Expand Content
If the response is successful, the AutoML Video Intelligence Object Tracking API returns the name for your operation. The following is an example of such a response, where project-number is the number of your project and operation-id is the ID of the long-running operation created for the request. For example VOT12345....

Import training data

REST & CMD LINE

For importing your training data, use the importData method. This method requires that you provide two parameters:

  1. the path to the CSV that contains paths to the training,
  2. the test data CSV files. Note: These files are made available in the "automl-video-demo-data" bucket on Cloud Storage.

Before using any of the request data below, make the following replacements:

  • dataset-id: the ID of your dataset. The ID is the last element of the name of your dataset. For example:
    • dataset name: projects/project-number/locations/location-id/datasets/3104518874390609379
    • dataset id: 3104518874390609379
  • bucket-name: replace with the name of the Cloud Storage bucket where you have stored your model training file list CSV file.
  • csv-file-name: replace with the name of your model training file list CSV file.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id:importData

Request JSON body:

{
  "inputConfig": {
    "gcsSource": {
      "inputUris": ["gs://bucket-name/csv-file-name.csv"]
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id:importData

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id:importData" | Select-Object -Expand Content
You should receive an operation ID for your import data operation. The example shows a response that contains the import operation ID VOT7506374678919774208.

Get the status of the import operation

You can query the status of your import data operation by using the following curl or PowerShell commands.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • operation-id: the ID of the long-running operation created for the request and provided in the response when you started the operation, for example VOT12345....
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

PowerShell

Execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id" | Select-Object -Expand Content
The import operation can take some time to complete. When the import task finishes, the operation status shows done: true with no errors listed, as shown in the example above.
  • operation-name: the name of the operation as returned by AutoML Video Intelligence Object Tracking API. The operation name has the format projects/project-number/locations/location-id/operations/operation-id

List all datasets

Use the following curl or PowerShell commands to get a list of your datasets and the number of sample videos that were imported into the dataset.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • project-number: the number of your project
  • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

PowerShell

Execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets " | Select-Object -Expand Content
In the response below, VOT3940649673949184000, is the operation ID of the long-running operation created for the request and provided in the response when you started the operation.

Train your model

Launch a model training operation

After you have created your dataset and imported your training data into your dataset, you can train your custom model. Train your model by using the using the following curl or PowerShell commands.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • dataset-id: the name of your target dataset. For example, my_dataset_01 display name).
  • model-name: replace with a name that you choose for your model.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

Request JSON body:

{
  "datasetId": "dataset-id",
  "displayName": "model-name",
  "videoObjectTrackingModelMetadata": {},
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

PowerShell

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models" | Select-Object -Expand Content
You should receive an operation ID for your model training operation. The above example shows a response that contains the model training operation ID VOT1741767155885539328.

Get the status of the model training operation

You can query the status of your model training operation by using the following curl or PowerShell commands.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • operation-name: the name of the operation as returned by the AutoML Video Intelligence Object Tracking API. The operation name has the format projects/project-number/locations/location-id/operations/operation-id
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/operation-name

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://automl.googleapis.com/v1beta1/operation-name

PowerShell

Execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://automl.googleapis.com/v1beta1/operation-name" | Select-Object -Expand Content
When the operation is complete, operation status shows done: true with no errors listed.

Verify that the model is available

After your model training operation successfully completes, you can verify that your model is available by using the following command to list the models for your project.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • project-number: the number of your project
  • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Make a Prediction

You can request annotations (predictions) for videos by using the batchPredict method. The batchPredict method takes requires two inputs:

  1. a CSV file stored in your Cloud Storage bucket that contains the paths to the videos to annotate,
  2. the start and end times that identify the segment of video to annotate.

For this quickstart, use the CSV file named traffic_video_batch_predict.csv.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • model-id: replace with the identifier for your model
  • input-uri: a Cloud Storage bucket that contains the file you want to annotate, including the file name. Must start with gs://.
    For example: "inputUris": ["gs://automl-video-demo-data/traffic_videos/traffic_video_batch_predict.csv"],
  • output-bucket: replace with a Cloud Storage bucket that will contain the results of your prediction.
  • object-id: replace with an object name that identifies where to store the output of your prediction request in yourCloud Storage bucket. Note: You must have write permissions to the Cloud Storage bucket.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL:

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models/model-id:batchPredict

Request JSON body:

{
  "inputConfig": {
    "gcsSource": {
      "inputUris": ["input-uri"]
    }
  },
  "outputConfig": {
    "gcsDestination": {
      "outputUriPrefix": "gs://output-bucket/object-id"
    }
  }
  "params": {
    "score_threshold": "0.0"
  }
}


To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Get the status of the predict operation

You can query the status of your batch predict operation by using the following curl or PowerShell commands.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

    • project-number: the number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.
    • operation-id: the ID of the long-running operation created for the request and provided in the response when you started the operation, for example VOT12345....

HTTP method and URL:

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

PowerShell

Execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id" | Select-Object -Expand Content
The operation-name is the name of the operation as returned by the AutoML Video Intelligence Object Tracking API. The operation name has the format projects/project-number/locations/location-id/operations/operation-id

When the batch predict task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your command. There is a JSON file for each video segment. The JSON files have a format similar to my-video-01.avi.json, where file extension .json is appended to the original file name.

{
  "inputUris": ["automl-video-demo-data/sample_video.avi"],
  "object_annotations": [ {
    "annotation_spec": {
      "display_name": "Cat",
      "description": "Cat"
    },
    "confidence": 0.43253016
    "frames": [ {
      "frame": {
        "time_offset": {
          "seconds": 4,
          "nanos": 960000000
        },
        "normalized_bounding_box": {
          "x_min": 0.1,
          "y_min": 0.1,
          "x_max": 0.8,
          "y_max": 0.8
        }
      }
    }, {
      "frame": {
        "time_offset": {
          "seconds": 5,
          "nanos": 960000000
        },
        "normalized_bounding_box": {
          "x_min": 0.2,
          "y_min": 0.2,
          "x_max": 0.9,
          "y_max": 0.9
        }
      }
    } ],
    "segment": {
      "start_time_offset": {
          "seconds": 4,
          "nanos": 960000000
      },
      "end_time_offset": {
          "seconds": 5,
          "nanos": 960000000
      }
    }
  } ],
  "error": {
    "details": [ ]
  }
}

Cleanup

If you no longer need your custom model and the related dataset, you can delete them.

Delete a model

You can delete a model by using the following curl or PowerShell commands.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • project-number: the number of your project
  • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.
  • model-id: replace with the identifier for your model.

HTTP method and URL:

DELETE https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models/model-id

To send your request, expand one of these options:

You should receive a successful status code (2xx) and an empty response.

Delete a dataset

You can delete a dataset by using the following curl or PowerShell commands.

REST & CMD LINE

Before using any of the request data below, make the following replacements:

  • project-number: the number of your project
  • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.
  • datase-id: replace with the identifier for your dataset id.

HTTP method and URL:

DELETE https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id

To send your request, expand one of these options:

You should receive a successful status code (2xx) and an empty response.