Submitting image labeling requests

Beta

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

AI Platform Data Labeling Service supports six types of image labeling tasks:

• Classification, where labelers assign one or more labels to each image.
• Bounding boxes, where labelers choose a label, then draw one or more bounding boxes to apply the label to the part(s) of the image.
• Oriented bounding box task, similar to bounding box tasks where the bounding boxes are x-axis aligned while oriented bounding box can be at any direction
• Bounding polygon, which is similar to bounding box, where labelers choose a label and draw closed polygons to apply the label to the part(s) of the image.
• Polyline tasks, similar to bounding box, where labelers choose a label and draw polylines to apply the label to the part(s) of the image. A polyline consists of a list of at least two points connected end-to-end by line segments.
• Segmentation tasks, where labelers generate masks for each label. The output is a color mapping table, which maps label to color.

Image classification tasks

Web UI

1. Open the Data Labeling Service UI.

2. Select Datasets from the left navigation.

   The Datasets page shows the status of previously created datasets for the current project.

3. Click the name of the dataset you want to submit for labeling.

   Datasets with status Import complete are available to submit. The Type of data column shows whether the dataset includes images, videos, or text.

4. On the Dataset detail page, click the Create labeling task button in the title bar.

5. On the New labeling task page, enter a name and description for the annotated dataset.

   The annotated dataset is the version of this dataset after human labelers have labeled it.

6. From the Objective drop-down, select the type of labeling task you want performed on this dataset.

   The drop-down list includes only the objectives available for the type of data in this dataset. If you don't see the objective you want, it probably means you've selected a dataset with a different type of data in it. Close the New labeling task page and select a different dataset.

7. From the Label set drop-down, select the label set you want the labelers to apply to data items in this set.

   The drop-down list includes all label sets associated with this project. You must select a label set.

8. From the Instruction drop-down, choose the instructions you want to provide to the labelers working with this dataset.

   The drop-down list includes all instructions associated with this project. You must include instructions in the labeling request.

9. From the labelers per data item drop-down, specify how many labelers you want to have review each item in the dataset.

   The default is one, but you can request three or five labelers to label each item.

10. Click the check box to confirm that you understand how you will be charged for the labeling.

11. Click Create.

Command-line

Set the following environment variables:

1. PROJECT_ID variable to your Google Cloud project ID.

2. DATASET_ID variable to the ID of your dataset, from the response when you created the dataset. The ID appears at the end of the full dataset name:

   projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID

3. INSTRUCTION_RESOURCE_NAME to the name of your instruction resource.
4. ANNOTATION_SPEC_SET_RESOURCE_NAME to the name of your label set resource.

curl -X POST -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \
  https://datalabeling.googleapis.com/v1beta1/projects/${PROJECT_ID}/datasets/${DATASET_ID}/image:label \
  -d '{
  "basicConfig": {
    "instruction": "${INSTRUCTION_RESOURCE_NAME}",
    "annotatedDatasetDisplayName": "curl_testing_annotated_dataset",
    "labelGroup": "test_label_group",
    "replica_count": 1
  },
  "feature": "TEXT_CLASSIFICATION",
  "imageClassificationConfig": {
    "annotationSpecSet": "${ANNOTATION_SPEC_SET_RESOURCE_NAME}",
  },
}'

You should see output similar to the following. You can use the operation ID to get the status of the task. Getting the status of an operation is an example.

{
  "name": "projects/data-labeling-codelab/operations/5c73dd6b_0000_2b34_a920_883d24fa2064",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.data-labeling.v1beta1.LabelImageClassificationOperationMetadata",
    "dataset": "projects/data-labeling-codelab/datasets/5c73db3d_0000_23e0_a25b_94eb2c119c4c"
  }
}

Python

Before you can run this code example, you must install the Python Client Libraries.

def label_image(
    dataset_resource_name, instruction_resource_name, annotation_spec_set_resource_name
):
    """Labels an image dataset."""
    from google.cloud import datalabeling_v1beta1 as datalabeling

    client = datalabeling.DataLabelingServiceClient()

    basic_config = datalabeling.HumanAnnotationConfig(
        instruction=instruction_resource_name,
        annotated_dataset_display_name="YOUR_ANNOTATED_DATASET_DISPLAY_NAME",
        label_group="YOUR_LABEL_GROUP",
        replica_count=1,
    )

    feature = datalabeling.LabelImageRequest.Feature.CLASSIFICATION

    # annotation_spec_set_resource_name needs to be created beforehand.
    # See the examples in the following:
    # https://cloud.google.com/ai-platform/data-labeling/docs/label-sets
    config = datalabeling.ImageClassificationConfig(
        annotation_spec_set=annotation_spec_set_resource_name,
        allow_multi_label=False,
        answer_aggregation_type=datalabeling.StringAggregationType.MAJORITY_VOTE,
    )

    response = client.label_image(
        request={
            "parent": dataset_resource_name,
            "basic_config": basic_config,
            "feature": feature,
            "image_classification_config": config,
        }
    )

    print(f"Label_image operation name: {response.operation.name}")
    return response

Java

Before you can run this code example, you must install the Java Client Libraries.

import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.datalabeling.v1beta1.AnnotatedDataset;
import com.google.cloud.datalabeling.v1beta1.DataLabelingServiceClient;
import com.google.cloud.datalabeling.v1beta1.DataLabelingServiceSettings;
import com.google.cloud.datalabeling.v1beta1.HumanAnnotationConfig;
import com.google.cloud.datalabeling.v1beta1.ImageClassificationConfig;
import com.google.cloud.datalabeling.v1beta1.LabelImageRequest;
import com.google.cloud.datalabeling.v1beta1.LabelImageRequest.Feature;
import com.google.cloud.datalabeling.v1beta1.LabelOperationMetadata;
import com.google.cloud.datalabeling.v1beta1.StringAggregationType;
import java.io.IOException;
import java.util.concurrent.ExecutionException;

class LabelImage {

  // Start an Image Labeling Task
  static void labelImage(
      String formattedInstructionName,
      String formattedAnnotationSpecSetName,
      String formattedDatasetName)
      throws IOException {
    // String formattedInstructionName = DataLabelingServiceClient.formatInstructionName(
    //      "YOUR_PROJECT_ID", "YOUR_INSTRUCTION_UUID");
    // String formattedAnnotationSpecSetName =
    //     DataLabelingServiceClient.formatAnnotationSpecSetName(
    //         "YOUR_PROJECT_ID", "YOUR_ANNOTATION_SPEC_SET_UUID");
    // String formattedDatasetName = DataLabelingServiceClient.formatDatasetName(
    //      "YOUR_PROJECT_ID", "YOUR_DATASET_UUID");


    DataLabelingServiceSettings settings =
        DataLabelingServiceSettings.newBuilder()
            .build();
    try (DataLabelingServiceClient dataLabelingServiceClient =
        DataLabelingServiceClient.create(settings)) {
      HumanAnnotationConfig humanAnnotationConfig =
          HumanAnnotationConfig.newBuilder()
              .setAnnotatedDatasetDisplayName("annotated_displayname")
              .setAnnotatedDatasetDescription("annotated_description")
              .setInstruction(formattedInstructionName)
              .build();

      ImageClassificationConfig imageClassificationConfig =
          ImageClassificationConfig.newBuilder()
              .setAllowMultiLabel(true)
              .setAnswerAggregationType(StringAggregationType.MAJORITY_VOTE)
              .setAnnotationSpecSet(formattedAnnotationSpecSetName)
              .build();

      LabelImageRequest labelImageRequest =
          LabelImageRequest.newBuilder()
              .setParent(formattedDatasetName)
              .setBasicConfig(humanAnnotationConfig)
              .setImageClassificationConfig(imageClassificationConfig)
              .setFeature(Feature.CLASSIFICATION)
              .build();

      OperationFuture<AnnotatedDataset, LabelOperationMetadata> operation =
          dataLabelingServiceClient.labelImageAsync(labelImageRequest);

      // You'll want to save this for later to retrieve your completed operation.
      System.out.format("Operation Name: %s\n", operation.getName());

      // Cancel the operation to avoid charges when testing.
      dataLabelingServiceClient.getOperationsClient().cancelOperation(operation.getName());

    } catch (IOException | InterruptedException | ExecutionException e) {
      e.printStackTrace();
    }
  }
}

Bounding boxes

Web UI

1. Open the Data Labeling Service UI.

2. Select Datasets from the left navigation.

   The Datasets page shows the status of previously created datasets for the current project.

3. Click the name of the dataset you want to submit for labeling.

   Datasets with status Import complete are available to submit. The Type of data column shows whether the dataset includes images, videos, or text.

4. On the Dataset detail page, click the Create labeling task button in the title bar.

5. On the New labeling task page, enter a name and description for the annotated dataset.

   The annotated dataset is the version of this dataset after human labelers have labeled it.

6. From the Objective drop-down, select the type of labeling task you want performed on this dataset.

   The drop-down list includes only the objectives available for the type of data in this dataset. If you don't see the objective you want, it probably means you've selected a dataset with a different type of data in it. Close the New labeling task page and select a different dataset.

7. From the Label set drop-down, select the label set you want the labelers to apply to data items in this set.

   The drop-down list includes all label sets associated with this project. You must select a label set.

8. From the Instruction drop-down, choose the instructions you want to provide to the labelers working with this dataset.

   The drop-down list includes all instructions associated with this project. You must include instructions in the labeling request.

9. From the labelers per data item drop-down, specify how many labelers you want to have review each item in the dataset.

   The default is one, but you can request three or five labelers to label each item.

10. Click the check box to confirm that you understand how you will be charged for the labeling.

11. Click Create.

Command-line

Set the following environment variables:

1. PROJECT_ID variable to your Google Cloud project ID.

2. DATASET_ID variable to the ID of your dataset, from the response when you created the dataset. The ID appears at the end of the full dataset name:

   projects/PROJECT_ID/locations/us-central1/datasets/DATASET_ID

3. INSTRUCTION_RESOURCE_NAME to the name of your instruction resource.
4. ANNOTATION_SPEC_SET_RESOURCE_NAME to the name of your label set resource.

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type: application/json" \
  https://datalabeling.googleapis.com/v1beta1/projects/${PROJECT_ID}/datasets/${DATASET_ID}/image:label \
  -d '{
  "basicConfig": {
    "instruction": "${INSTRUCTION_RESOURCE_NAME}",
    "annotatedDatasetDisplayName": "curl_testing_annotated_dataset",
    "labelGroup": "test_label_group",
    "replica_count": 1
  },
  "feature": "BOUNDING_BOX",
  "boundingPolyConfig": {
    "annotationSpecSet": "${ANNOTATION_SPEC_SET_RESOURCE_NAME}",
  },
}'

You should see output similar to the following. You can use the operation ID to get the status of the task. Getting the status of an operation is an example.

{
  "name": "projects/data-labeling-codelab/operations/5c73dd6b_0000_2b34_a920_883d24fa2064",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.data-labeling.v1beta1.LabelImageBoundingBoxOperationMetadata",
    "dataset": "projects/data-labeling-codelab/datasets/5c73db3d_0000_23e0_a25b_94eb2c119c4c"
  }
}