This legacy version of AutoML Video Intelligence is deprecated and will no longer be available on Google Cloud after January 23, 2024. All the functionality of legacy AutoML Video Intelligence and new features are available on the Vertex AI platform. See Migrate to Vertex AI to learn how to migrate your resources.
Stay organized with collections Save and categorize content based on your preferences.

Label videos by using the command line

This quickstart walks you through the process of:

  • Copying a set of videos into Cloud Storage.
  • Creating CSV files that list videos and their labels.
  • Using AutoML Video to create your dataset, and train and use your model.

Before you begin

Set up your project

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. Install the Google Cloud CLI.

  3. To initialize the gcloud CLI, run the following command: 

    gcloud init

  4. Create or select a Google Cloud project.

    • Create a Cloud project:

      gcloud projects create PROJECT_ID

    • Select the Cloud project that you created:

      gcloud config set project PROJECT_ID

  5. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

  6. Enable the AutoML and Cloud Storage APIs: 

    gcloud services enable storage-component.googleapis.com automl.googleapis.com storage-api.googleapis.com
    7.
  7. Install the Google Cloud CLI.

  8. To initialize the gcloud CLI, run the following command: 

    gcloud init

  9. Create or select a Google Cloud project.

    • Create a Cloud project:

      gcloud projects create PROJECT_ID

    • Select the Cloud project that you created:

      gcloud config set project PROJECT_ID

  10. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

  11. Enable the AutoML and Cloud Storage APIs: 

    gcloud services enable storage-component.googleapis.com automl.googleapis.com storage-api.googleapis.com
    12.
  12. Set the PROJECT_ID environment variable to your Project ID. 
    export PROJECT_ID=PROJECT_ID
    The AutoML API calls and resource names include your Project ID in them. The PROJECT_ID environment variable provides a convenient way to specify the ID.

Create a dataset and import training data

Create a dataset

Decide on a name for your dataset and use the following curl or PowerShell commands to create a new dataset with that name.

REST

Before using any of the request data, make the following replacements:

  • dataset-name: name of the dataset to show in the interface
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

POST  https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

Request JSON body: 

{
  "displayName": "dataset-name",
  "videoClassificationDatasetMetadata": {
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command: 

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    " https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets"

PowerShell

Save the request body in a file called request.json, and execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri " https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets" | Select-Object -Expand Content
If the response is successful, the AutoML Video Intelligence Classification API returns the name for your operation. The following shows an example of such a response, where project-number is the number of your project and operation-id is the ID of the long-running operation created for the request.

Import training data

REST

For importing your training data, use the importData method. This method requires that you provide two parameters:

  1. the path to the CSV that contains paths to the training,
  2. the test data CSV files. Note: These files are made available in the "automl-video-demo-data" bucket on Cloud Storage.

Before using any of the request data, make the following replacements:

  • input-uri: a Cloud Storage bucket that contains the file you want to annotate, including the file name. Must start with gs://. For example:
    "inputUris": ["gs://automl-video-demo-data/hmdb_split1.csv"]
  • dataset-id: replace with the dataset identifier for your dataset (not the display name). For example: VCN4798585402963263488
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

POST  https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id:importData

Request JSON body: 

{
   "inputConfig": {
      "gcsSource": {
         "inputUris": input-uri
      }
   }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command: 

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    " https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id:importData"

PowerShell

Save the request body in a file called request.json, and execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri " https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets/dataset-id:importData" | Select-Object -Expand Content
You should receive an operation ID for your import data operation. The example shows a response that contains the import operation ID VCN7506374678919774208.

Get the status of the import operation

You can query the status of your import data operation by using the following curl or PowerShell commands.

REST

Before using any of the request data, make the following replacements:

  • operation-id: replace with the operation id for your import data operation.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

To send your request, choose one of these options:

curl

Execute the following command: 

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id"

PowerShell

Execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id" | Select-Object -Expand Content
The import operation can take some time to complete. When the import task is finished, you will see done: true in the status of the operation with no errors listed, as shown in the below example.

List all datasets

Use the following curl or PowerShell commands to get a list of your datasets and the number of sample videos that were imported into the dataset.

REST

Before using any of the request data, make the following replacements:

  • project-number: number of your project
  • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region is determined based on video file location.

HTTP method and URL: 

 https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Train your model

Launch a model training operation

After you have created your dataset and imported your training data into your dataset, you can train your model.

Train your model by using the following curl or PowerShell commands.

REST

Before using any of the request data, make the following replacements:

  • dataset-id: The ID is the last element of the name of your dataset. For example, if the name of your dataset is projects/434039606874/locations/us-central1/datasets/VCN3104518874390609379, then the ID of your dataset is VCN3104518874390609379.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

Request JSON body: 

{
  "displayName": "test_model",
  "dataset_id": "dataset-id",
  "videoClassificationModelMetadata": {}
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command: 

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models"

PowerShell

Save the request body in a file called request.json, and execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models" | Select-Object -Expand Content
The operation-id is provided in the response when you started the operation, for example VCN123.... 
{
  "name": "projects/project-number/locations/location-id/operations/operation-id",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.automl.v1beta1.OperationMetadata",
    "progressPercentage": 100,
    "createTime": "2020-02-27T01:56:28.395640Z",
    "updateTime": "2020-02-27T02:04:12.336070Z"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.automl.v1beta1.Model",
    "name": "projects/project-number/locations/location-id/models/operation-id",
    "createTime": "2020-02-27T02:00:22.329970Z",
    "videoClassificationModelMetadata": {
      "trainBudget": "1",
      "trainCost": "1",
      "stopReason": "BUDGET_REACHED"
    },
    "displayName": "a_98487760535e48319dd204e6394670"
  }
}

Get the status of the model training operation

Train your model by using the following curl or PowerShell commands.

REST

Before using any of the request data, make the following replacements:

  • operation-id: replace with the operation id for your training operation.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

To send your request, choose one of these options:

curl

Execute the following command: 

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id"

PowerShell

Execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id" | Select-Object -Expand Content
When the operation is complete, you will see done: true with no errors listed.

Verify that the model is available

After your model training operation successfully completes, you can verify that your model is available by using the following curl or PowerShell commands to list the models for your project.

REST

Before using any of the request data, make the following replacements:

  • dataset-id: replace with the dataset identifier for your dataset (not the display name).
  • model-name: replace with a name that you choose for your model
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

To send your request, choose one of these options:

curl

Execute the following command: 

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id"

PowerShell

Execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id" | Select-Object -Expand Content
You should receive an operation id for your import data operation.
For example: VCN1741767155885539328.

Make a prediction

You can request annotations (predictions) for videos by using the batchPredict command. The batchPredict command takes, as input, a CSV file stored in your Cloud Storage bucket that contains the paths to the videos to annotate, and the start and end times that identify the segment of video to annotate. For this quickstart, this CSV file is named hmdb_split1_test_gs_predict.csv.

Run the following curl or PowerShell commands to make a batch (asynchronous) predict request.

REST

Before using any of the request data, make the following replacements:

  • input-uri: a Cloud Storage bucket that contains the file you want to annotate, including the file name. Must start with gs://. For example:
    "inputUris": ["gs://automl-video-demo-data/hmdb_split1_test_gs_predict.csv"]
  • output-bucket replace with the name of your Cloud Storage bucket. For example: my-project-vcm
  • object-id: replace with the operation id for your import data operation.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

POST https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models/model-id:batchPredict

Request JSON body: 

{
  "inputConfig": {
    "gcsSource": {
      "inputUris": [input-uri]
    }
  },
  "outputConfig": {
    "gcsDestination": {
      "outputUriPrefix": "gs://output-bucket/object-id"
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file called request.json, and execute the following command: 

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models/model-id:batchPredict
"

PowerShell

Save the request body in a file called request.json, and execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models/model-id:batchPredict
" | Select-Object -Expand Content

You should receive an operation id for your batch predict request. For example: VCN926615623331479552

Get status of the predict operation

You can query the status of your batch predict operation by using the following curl or PowerShell commands.

REST

Before using any of the request data, make the following replacements:

  • operation-id: replace with the operation id for your import data operation.
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id

To send your request, choose one of these options:

curl

Execute the following command: 

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: project-number" \
    "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id"

PowerShell

Execute the following command: 

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred"; "x-goog-user-project" = "project-number" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/operations/operation-id" | Select-Object -Expand Content
Depending on the number of videos that you specified in your CSV file, the batch predict task can take some time to complete. When the task is finished, you will see done: true in the status of the operation with no errors listed, as shown in the following example.

When the batch predict task is complete, the output of the prediction is stored in the Cloud Storage bucket that you specified in your command. There is a JSON file for each video segment. For example:
my-video-01.avi.json

{
  "inputUris": ["gs://automl-video-demo-data/sample_video.avi"]
  "segment_classification_annotations": [ {
    "annotation_spec": {
      "display_name": "ApplyLipstick",
      "description": "ApplyLipstick"
    },
    "segments": [ {
      "segment": {
        "start_time_offset": {
        },
        "end_time_offset": {
          "seconds": 4,
          "nanos": 960000000
        }
      },
      "confidence": 0.43253016
    }, {
      "segment": {
        "start_time_offset": {
        },
        "end_time_offset": {
          "seconds": 4,
          "nanos": 960000000
        }
      },
      "confidence": 0.56746984
    } ],
    "frames": [ ]
  } ],
  "shot_classification_annotations": [ {
    "annotation_spec": {
      "display_name": "ApplyLipstick",
      "description": "ApplyLipstick"
    },
    "segments": [ {
      "segment": {
        "start_time_offset": {
        },
        "end_time_offset": {
          "seconds": 5
        }
      },
      "confidence": 0.43253016
    }, {
      "segment": {
        "start_time_offset": {
        },
        "end_time_offset": {
          "seconds": 5
        }
      },
      "confidence": 0.56746984
    } ],
    "frames": [ ]
  } ],
  "one_second_sliding_window_classification_annotations": [ {
    "annotation_spec": {
      "display_name": "ApplyLipstick",
      "description": "ApplyLipstick"
    },
    "segments": [ ],
    "frames": [ {
      "time_offset": {
        "nanos": 800000000
      },
      "confidence": 0.54533803
    }, {
      "time_offset": {
        "nanos": 800000000
      },
      ...
      "confidence": 0.57945728
    }, {
      "time_offset": {
        "seconds": 4,
        "nanos": 300000000
      },
      "confidence": 0.42054281
    } ]
  } ],
  "object_annotations": [ ],
  "error": {
    "details": [ ]
  }
}

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, delete the Cloud project with the resources.

If you no longer need your model and the related dataset, you can delete them.

List models

You can list the models for your project, along with their identifiers, by using the following curl or PowerShell commands:

REST

Before using any of the request data, make the following replacements:

  • model-name: The full name of your model provided by the response when you created the model. The full name has the format: projects/project-number/locations/location-id/models
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

GET https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/models

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Delete a model

You can delete a model by using the following command.

REST

Before using any of the request data, make the following replacements:

  • model-id: replace with the identifier for your model
  • Note:
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region will be determined based on video file location.

HTTP method and URL: 

DELETE https://automl.googleapis.com/v1beta1/projects/project-number/locations/test/models/model-id

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

List datasets

You can list the datasets for your project, along with their identifiers, by using the following curl or PowerShell commands:

REST

Before using any of the request data, make the following replacements:

  • project-number: number of your project
  • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region is determined based on video file location.

HTTP method and URL: 

 https://automl.googleapis.com/v1beta1/projects/project-number/locations/location-id/datasets

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Delete a dataset

You can delete a dataset by using the following curl or PowerShell commands.

REST

Before using any of the request data, make the following replacements:

  • dataset-name: the full name of your dataset, from the response when you created the dataset. The full name has the format:
    projects/project-number/locations/location-id/datasets/dataset-id
    • project-number: number of your project
    • location-id: the Cloud region where annotation should take place. Supported cloud regions are: us-east1, us-west1, europe-west1, asia-east1. If no region is specified, a region is determined based on video file location.
    • dataset-id: the id provided when you created the dataset

HTTP method and URL: 

DELETE  https://automl.googleapis.com/v1beta1/dataset-name

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

What's next