Labeling and filtering resources

This page explains how to label your resources and then use the labels to organize and filter your resources.

Overview of labels

You can add labels to your AI Platform Training jobs, then use those labels to organize resources into categories when viewing or monitoring the resources.

For example, you can label jobs by team (such as engineering or research) and development phase (prod or test), then filter the jobs based on the team and phase.

Labels are also available on operations, but these labels are derived from the resource to which the operation applies. You cannot add or update labels on an operation.

A label is a key-value pair, where both the key and the value are custom strings that you supply.

Limits and rules:

  • Maximum 64 labels per resource.
  • Maximum 63 characters per key or value.
  • Keys and values can contain lowercase letters, numeric characters, underscores and dashes.
  • Keys must start with a letter.
  • International characters are allowed.

Examples:

  • Labels indicating team or cost center: team:engineering and team:research.
  • Labels indicating development phase: phase:prod and phase:test.
  • Labels indicating owner or contact: owner:alice and owner:bob.

Adding labels when creating a resource

When creating a resource, you can use the gcloud command or the AI Platform Training and Prediction API in your Python application to add labels to your jobs. Choose a tab below to see examples of each method of adding labels:

console

You must create the resource before adding labels. See how to add and update labels.

gcloud

When creating a new resource, specify the labels field to attach a label to the new resource.

For the examples below, assume you want to use these labels: team:engineering,phase:test, and owner:alice.

For convenience, set up an environment variable containing the labels:

RESOURCE_LABELS="team=engineering,phase=test,owner=alice"

The following code submits a training job with the specified labels:

JOB_NAME="your_job_name"
TRAINER_PACKAGE_PATH="/path/to/your/application/sources"
MAIN_TRAINER_MODULE="trainer.task"
JOB_DIR="gs://your/job/output/path"
TRAIN_DATA="gs://your/training/data/path"
EVAL_DATA="gs://your/eval/data/path"

gcloud ai-platform jobs submit training $JOB_NAME \
        --labels $RESOURCE_LABELS \
        --scale-tier basic \
        --package-path $TRAINER_PACKAGE_PATH \
        --module-name $MAIN_TRAINER_MODULE \
        --job-dir $JOB_DIR \
        --runtime-version 2.11 \
        --region $REGION \
        -- \
        --train-files $TRAIN_DATA \
        --eval-files $EVAL_DATA \
        --train-steps 1000

Python

Set up a dictionary containing the fields for the resource you want to create. Include your labels as a dictionary entry that maps strings to strings.

The following example shows you how to create a job with three labels: team:engineering,phase:test, and owner:alice.

training_inputs = {'scaleTier': 'BASIC',
    'packageUris': ['gs://your/trainer/path/package-0.0.0.tar.gz'],
    'pythonModule': 'trainer.task'
    'args': ['--arg1', 'value1', '--arg2', 'value2'],
    'region': 'us-central1',
    'labels': {
       'team': 'engineering',
       'phase': 'test',
       'owner': 'alice'
    },
    'jobDir': 'gs://your/training/job/directory',
    'runtimeVersion': '2.11'}

job_spec = {'jobId': your_job_name, 'trainingInput': training_inputs}

Send the request:

request = cloudml.projects().jobs().create(body=job_spec,
      parent=project_id)
response = request.execute()

Updating and removing labels

You can update or remove labels on jobs and models, using the gcloud command or in your Python application. It is not possible to update labels on existing model versions.

Choose a tab below to see examples of each method of removing/updating labels:

console

  1. Open the Google Cloud console page for the resource you want to label:

  2. Select the check box next to the name(s) of the resource(s) you want to label.

  3. All label editing occurs in the panel to the right of the resource names.

    • To add labels, enter the key (and optionally, value) for the label. To add multiple labels, click the Add label button.

    • To remove labels, hover over the right of the label and click the Delete icon that displays.

  4. After you edit your labels, click Save to confirm the changes, or Discard changes to start over.

gcloud

You can use the gcloud command to update or remove labels on an existing resource. For example, to adjust the labels on a job, run the gcloud ai-platform jobs update command with the following flags:

  • --update-labels to modify the value of existing labels and add new labels. For example, to change the value of the phase label to production and add a new status label:

    gcloud ai-platform jobs update $JOB_NAME \
        --update-labels phase=production,status=deployed

  • --clear-labels to remove all labels from the job. If you include an --update-labels flag in the same command, the clear command is applied first, followed by the update command. For example, to remove all labels and apply new foo and baz labels:

    gcloud ai-platform jobs update $JOB_NAME \
        --clear-labels \
        --update-labels foo=bar,baz=qux

  • --remove-labels to remove specific labels from the job. If you specify a label that does not exist on the job, the remove command is ignored for that label. For example, to remove the status and visibility labels:

    gcloud ai-platform jobs update $JOB_NAME \
    --remove-labels=status,visibility

Python

To remove or update labels for an existing job:

  • Get the job using a GET request.
  • Modify the labels field.
  • Update the job using a PATCH request.

The following sample shows the READ-MODIFY-WRITE pattern for updating job labels:

existingJob = ml.projects().jobs().get(jobId=jobId).execute()
etag = existingJob['etag']

updatedJob = {
    'labels': {
        'phase': 'prod',
        'team': 'research'
    },
    'etag': etag
}

updateJobRequest = ml.projects().jobs().patch(name=jobId,
      body=updatedJob, updateMask='labels,etag')
updatedJob = updateJobRequest.execute()

Using labels to filter resources

When listing your resources, you can filter the list by label.

console

  1. Open the Google Cloud console page for the resource you want to filter:

  2. Click within the Filter by prefix field, which is located above your list of jobs. Select the Label prefix.

  3. To complete the filter, enter the key and value using the syntax "key:value". For example, "team:engineering" or "owner:alice".

  4. The filter is applied to your resource list, and the name of the filter displays in the filter field. For example: "Labels:owner:alice" or "Labels:team:engineering". You can add multiple filters, if needed.

gcloud

The following example lists all jobs labeled with both the key-value pair team:engineering and the key-value pair owner:alice:

gcloud ai-platform jobs list \
        --filter='labels.team:engineering AND labels.owner=alice'

Python

The following example uses the jobs.list request to retrieve all jobs labeled with both the key-value pair team:engineering and the key-value pair owner:alice:

request = ml.projects().jobs().list(parent=projectID,
      filter='labels.team=engineering AND labels.owner=alice')
results = request.execute()

What's next