This legacy version of AI Platform Data Labeling is deprecated and will no longer be available on Google Cloud after January 23, 2024. All the functionality of legacy AI Platform Data Labeling and new features are available on the Vertex AI platform. See Migrate to Vertex AI to learn how to migrate your resources.

Submitting video labeling requests

AI Platform Data Labeling Service supports four types of video labeling tasks:

Classification tasks, where labelers assign one or more labels to each video. You specify the number of labelers to label each video. We recommend that the number should be five or less. Data Labeling Service does a majority vote to determine the proper labels. You can also specify whether to apply shot detection on your video or not.
Object detection tasks, where labelers choose a label, then draw one or more bounding boxes to apply to the part(s) of the extracted image from your video. They can choose another label and repeat the process until there are no more interesting objects to identify in the extracted image. You can specify the extraction frame rate to determine how many images to extract from your video.
Object tracking tasks, where labelers choose a label, then draw one or more bounding boxes to indicate the part(s) of the video for certain object(s), and keep tracking each object through the entire video.
Event tasks, where labelers choose a label, then choose the start and end time to apply labels to the segment or point in the video.

Video classification

Web UI

Open the Data Labeling Service UI.
Select Datasets from the left navigation.

The Datasets page shows the status of previously created datasets for the current project.
Click the name of the dataset you want to submit for labeling.

Datasets with status Import complete are available to submit. The Type of data column shows whether the dataset includes images, videos, or text.
On the Dataset detail page, click the Create labeling task button in the title bar.
On the New labeling task page, enter a name and description for the annotated dataset.

The annotated dataset is the version of this dataset after human labelers have labeled it.
From the Objective drop-down, select the type of labeling task you want performed on this dataset.

The drop-down list includes only the objectives available for the type of data in this dataset. If you do not see the objective you want, it probably means you have selected a dataset with a different type of data in it. Close the New labeling task page and select a different dataset.
From the Label set drop-down, choose the label set you want the labelers to apply to data items in this set.

The drop-down list includes all label sets associated with this project. You must choose a set.
From the Instruction drop-down, choose the instructions you want to provide to the labelers working with this dataset.

The drop-down list includes all instructions associated with this project. You must include instructions in the labeling request.
From the labelers per data item drop-down, specify how many labelers you want to have review each item in the dataset.

The default is one, but you can request to have three or five labelers label each item.
Click the check box to confirm that you understand how you will be charged for the labeling.
Click Create.

Command-line

Set the following environment variables:

PROJECT_ID variable to your Google Cloud project ID.
DATASET_ID variable to the ID of your dataset, from the response when you created the dataset. The ID appears at the end of the full dataset name:
```
projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID
```
INSTRUCTION_RESOURCE_NAME to the name of your instruction resource.
ANNOTATION_SPEC_SET_RESOURCE_NAME to the name of your label set resource.

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \
  https://datalabeling.googleapis.com/v1beta1/projects/${PROJECT_ID}/datasets/${DATASET_ID}/video:label \
  -d '{
  "basicConfig": {
    "instruction": "${INSTRUCTION_RESOURCE_NAME}",
    "annotatedDatasetDisplayName": "curl_testing_annotated_dataset",
    "labelGroup": "test_label_group",
    "replica_count": 1
  },
  "feature": "CLASSIFICATION",
  "videoClassificationConfig": {
    "annotationSpecSetConfigs": ["annotationSpecSet": "${ANNOTATION_SPEC_SET_RESOURCE_NAME}"],
  },
}'

You should see output similar to the following. You can use the operation ID to get the status of the task. Getting the status of an operation is an example.

{
  "name": "projects/data-labeling-codelab/operations/5c73dd6b_0000_2b34_a920_883d24fa2064",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.data-labeling.v1beta1.LabelVideoClassificationOperationMetadata",
    "dataset": "projects/data-labeling-codelab/datasets/5c73db3d_0000_23e0_a25b_94eb2c119c4c"
  }
}

Java

Before you can run this code example, you must install the Java Client Libraries.

import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.datalabeling.v1beta1.AnnotatedDataset;
import com.google.cloud.datalabeling.v1beta1.DataLabelingServiceClient;
import com.google.cloud.datalabeling.v1beta1.DataLabelingServiceSettings;
import com.google.cloud.datalabeling.v1beta1.HumanAnnotationConfig;
import com.google.cloud.datalabeling.v1beta1.LabelOperationMetadata;
import com.google.cloud.datalabeling.v1beta1.LabelVideoRequest;
import com.google.cloud.datalabeling.v1beta1.LabelVideoRequest.Feature;
import com.google.cloud.datalabeling.v1beta1.VideoClassificationConfig;
import com.google.cloud.datalabeling.v1beta1.VideoClassificationConfig.AnnotationSpecSetConfig;
import java.io.IOException;
import java.util.concurrent.ExecutionException;

class LabelVideo {

  // Start a Video Labeling Task
  static void labelVideo(
      String formattedInstructionName,
      String formattedAnnotationSpecSetName,
      String formattedDatasetName)
      throws IOException {
    // String formattedInstructionName = DataLabelingServiceClient.formatInstructionName(
    //      "YOUR_PROJECT_ID", "YOUR_INSTRUCTION_UUID");
    // String formattedAnnotationSpecSetName =
    //     DataLabelingServiceClient.formatAnnotationSpecSetName(
    //         "YOUR_PROJECT_ID", "YOUR_ANNOTATION_SPEC_SET_UUID");
    // String formattedDatasetName = DataLabelingServiceClient.formatDatasetName(
    //      "YOUR_PROJECT_ID", "YOUR_DATASET_UUID");


    DataLabelingServiceSettings settings =
        DataLabelingServiceSettings.newBuilder()
            .build();
    try (DataLabelingServiceClient dataLabelingServiceClient =
        DataLabelingServiceClient.create(settings)) {
      HumanAnnotationConfig humanAnnotationConfig =
          HumanAnnotationConfig.newBuilder()
              .setAnnotatedDatasetDisplayName("annotated_displayname")
              .setAnnotatedDatasetDescription("annotated_description")
              .setInstruction(formattedInstructionName)
              .build();

      AnnotationSpecSetConfig annotationSpecSetConfig =
          AnnotationSpecSetConfig.newBuilder()
              .setAnnotationSpecSet(formattedAnnotationSpecSetName)
              .setAllowMultiLabel(true)
              .build();

      VideoClassificationConfig videoClassificationConfig =
          VideoClassificationConfig.newBuilder()
              .setApplyShotDetection(true)
              .addAnnotationSpecSetConfigs(annotationSpecSetConfig)
              .build();

      LabelVideoRequest labelVideoRequest =
          LabelVideoRequest.newBuilder()
              .setParent(formattedDatasetName)
              .setBasicConfig(humanAnnotationConfig)
              .setVideoClassificationConfig(videoClassificationConfig)
              .setFeature(Feature.CLASSIFICATION)
              .build();

      OperationFuture<AnnotatedDataset, LabelOperationMetadata> operation =
          dataLabelingServiceClient.labelVideoAsync(labelVideoRequest);

      // You'll want to save this for later to retrieve your completed operation.
      System.out.format("Operation Name: %s\n", operation.getName());

      // Cancel the operation to avoid charges when testing.
      dataLabelingServiceClient.getOperationsClient().cancelOperation(operation.getName());
    } catch (IOException | InterruptedException | ExecutionException e) {
      e.printStackTrace();
    }
  }
}

Video object detection

Web UI

Open the Data Labeling Service UI.
Select Datasets from the left navigation.

The Datasets page shows the status of previously created datasets for the current project.
Click the name of the dataset you want to submit for labeling.

Datasets with status Import complete are available to submit. The Type of data column shows whether the dataset includes images, videos, or text.
On the Dataset detail page, click the Create labeling task button in the title bar.
On the New labeling task page, enter a name and description for the annotated dataset.

The annotated dataset is the version of this dataset after human labelers have labeled it.
From the Objective drop-down, select the type of labeling task you want performed on this dataset.

The drop-down list includes only the objectives available for the type of data in this dataset. If you do not see the objective you want, it probably means you have selected a dataset with a different type of data in it. Close the New labeling task page and select a different dataset.
From the Label set drop-down, choose the label set you want the labelers to apply to data items in this set.

The drop-down list includes all label sets associated with this project. You must choose a set.
From the Instruction drop-down, choose the instructions you want to provide to the labelers working with this dataset.

The drop-down list includes all instructions associated with this project. You must include instructions in the labeling request.
From the labelers per data item drop-down, specify how many labelers you want to have review each item in the dataset.

The default is one, but you can request to have three or five labelers label each item.
Click the check box to confirm that you understand how you will be charged for the labeling.
Click Create.

Command-line

Set the following environment variables:

PROJECT_ID variable to your Google Cloud project ID.
DATASET_ID variable to the ID of your dataset, from the response when you created the dataset. The ID appears at the end of the full dataset name:
```
projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID
```
INSTRUCTION_RESOURCE_NAME to the name of your instruction resource.
ANNOTATION_SPEC_SET_RESOURCE_NAME to the name of your label set resource.

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \
  https://datalabeling.googleapis.com/v1beta1/projects/${PROJECT_ID}/datasets/${DATASET_ID}/video:label \
  -d '{
  "basicConfig": {
    "instruction": "${INSTRUCTION_RESOURCE_NAME}",
    "annotatedDatasetDisplayName": "curl_testing_annotated_dataset",
    "labelGroup": "test_label_group",
    "replica_count": 1
  },
  "feature": "OBJECT_DETECTION",
  "objectDetectionConfig": {
    "annotationSpecSet": "${ANNOTATION_SPEC_SET_RESOURCE_NAME}",
    "instructionMessage": "Detect an object",
    "extractionFrameRate": 4
  },
}'

You should see output similar to the following. You can use the operation ID to get the status of the task. Getting the status of an operation is an example.

{
  "name": "projects/data-labeling-codelab/operations/5c73dd6b_0000_2b34_a920_883d24fa2064",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.data-labeling.v1beta1.LabelVideoObjectDetectionOperationMetadata",
    "dataset": "projects/data-labeling-codelab/datasets/5c73db3d_0000_23e0_a25b_94eb2c119c4c"
  }
}

Video object tracking

Web UI

Open the Data Labeling Service UI.
Select Datasets from the left navigation.

The Datasets page shows the status of previously created datasets for the current project.
Click the name of the dataset you want to submit for labeling.

Datasets with status Import complete are available to submit. The Type of data column shows whether the dataset includes images, videos, or text.
On the Dataset detail page, click the Create labeling task button in the title bar.
On the New labeling task page, enter a name and description for the annotated dataset.

The annotated dataset is the version of this dataset after human labelers have labeled it.
From the Objective drop-down, select the type of labeling task you want performed on this dataset.

The drop-down list includes only the objectives available for the type of data in this dataset. If you do not see the objective you want, it probably means you have selected a dataset with a different type of data in it. Close the New labeling task page and select a different dataset.
From the Label set drop-down, choose the label set you want the labelers to apply to data items in this set.

The drop-down list includes all label sets associated with this project. You must choose a set.
From the Instruction drop-down, choose the instructions you want to provide to the labelers working with this dataset.

The drop-down list includes all instructions associated with this project. You must include instructions in the labeling request.
From the labelers per data item drop-down, specify how many labelers you want to have review each item in the dataset.

The default is one, but you can request to have three or five labelers label each item.
Click the check box to confirm that you understand how you will be charged for the labeling.
Click Create.

Command-line

Set the following environment variables:

PROJECT_ID variable to your Google Cloud project ID.
DATASET_ID variable to the ID of your dataset, from the response when you created the dataset. The ID appears at the end of the full dataset name:
```
projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID
```
INSTRUCTION_RESOURCE_NAME to the name of your instruction resource.
ANNOTATION_SPEC_SET_RESOURCE_NAME to the name of your label set resource.

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \
  https://datalabeling.googleapis.com/v1beta1/projects/${PROJECT_ID}/datasets/${DATASET_ID}/video:label \
  -d '{
  "basicConfig": {
    "instruction": "${INSTRUCTION_RESOURCE_NAME}",
    "annotatedDatasetDisplayName": "curl_testing_annotated_dataset",
    "labelGroup": "test_label_group",
    "replica_count": 1
  },
  "feature": "OBJECT_TRACKING",
  "objectTrackingConfig": {
    "annotationSpecSet": "${ANNOTATION_SPEC_SET_RESOURCE_NAME}"
  },
}'

You should see output similar to the following. You can use the operation ID to get the status of the task. Getting the status of an operation is an example.

{
  "name": "projects/data-labeling-codelab/operations/5c73dd6b_0000_2b34_a920_883d24fa2064",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.data-labeling.v1beta1.LabelVideoObjectTrackingOperationMetadata",
    "dataset": "projects/data-labeling-codelab/datasets/5c73db3d_0000_23e0_a25b_94eb2c119c4c"
  }
}

Python

Before you can run this code example, you must install the Python Client Libraries.

def label_video(
    dataset_resource_name, instruction_resource_name, annotation_spec_set_resource_name
):
    """Labels a video dataset."""
    from google.cloud import datalabeling_v1beta1 as datalabeling

    client = datalabeling.DataLabelingServiceClient()

    basic_config = datalabeling.HumanAnnotationConfig(
        instruction=instruction_resource_name,
        annotated_dataset_display_name="YOUR_ANNOTATED_DATASET_DISPLAY_NAME",
        label_group="YOUR_LABEL_GROUP",
        replica_count=1,
    )

    feature = datalabeling.LabelVideoRequest.Feature.OBJECT_TRACKING

    config = datalabeling.ObjectTrackingConfig(
        annotation_spec_set=annotation_spec_set_resource_name
    )

    response = client.label_video(
        request={
            "parent": dataset_resource_name,
            "basic_config": basic_config,
            "feature": feature,
            "object_tracking_config": config,
        }
    )

    print(f"Label_video operation name: {response.operation.name}")
    return response

Video event

Web UI

Open the Data Labeling Service UI.
Select Datasets from the left navigation.

The Datasets page shows the status of previously created datasets for the current project.
Click the name of the dataset you want to submit for labeling.

Datasets with status Import complete are available to submit. The Type of data column shows whether the dataset includes images, videos, or text.
On the Dataset detail page, click the Create labeling task button in the title bar.
On the New labeling task page, enter a name and description for the annotated dataset.

The annotated dataset is the version of this dataset after human labelers have labeled it.
From the Objective drop-down, select the type of labeling task you want performed on this dataset.

The drop-down list includes only the objectives available for the type of data in this dataset. If you do not see the objective you want, it probably means you have selected a dataset with a different type of data in it. Close the New labeling task page and select a different dataset.
From the Label set drop-down, choose the label set you want the labelers to apply to data items in this set.

The drop-down list includes all label sets associated with this project. You must choose a set.
From the Instruction drop-down, choose the instructions you want to provide to the labelers working with this dataset.

The drop-down list includes all instructions associated with this project. You must include instructions in the labeling request.
From the labelers per data item drop-down, specify how many labelers you want to have review each item in the dataset.

The default is one, but you can request to have three or five labelers label each item.
Click the check box to confirm that you understand how you will be charged for the labeling.
Click Create.

Command-line

Set the following environment variables:

PROJECT_ID variable to your Google Cloud project ID.
DATASET_ID variable to the ID of your dataset, from the response when you created the dataset. The ID appears at the end of the full dataset name:
```
projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID
```
INSTRUCTION_RESOURCE_NAME to the name of your instruction resource.
ANNOTATION_SPEC_SET_RESOURCE_NAME to the name of your label set resource.

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \
  https://datalabeling.googleapis.com/v1beta1/projects/${PROJECT_ID}/datasets/${DATASET_ID}/video:label \
  -d '{
  "basicConfig": {
    "instruction": "${INSTRUCTION_RESOURCE_NAME}",
    "annotatedDatasetDisplayName": "curl_testing_annotated_dataset",
    "labelGroup": "test_label_group",
    "replica_count": 1
  },
  "feature": "EVENT",
  "eventConfig": {
    "annotationSpecSets": ["${ANNOTATION_SPEC_SET_RESOURCE_NAME}]"
  },
}'

You should see output similar to the following. You can use the operation ID to get the status of the task. Getting the status of an operation is an example.

{
  "name": "projects/data-labeling-codelab/operations/5c73dd6b_0000_2b34_a920_883d24fa2064",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.data-labeling.v1beta1.LabelVideoEventOperationMetadata",
    "dataset": "projects/data-labeling-codelab/datasets/5c73db3d_0000_23e0_a25b_94eb2c119c4c"
  }
}