ラベルを使用してリソースを整理する

このドキュメントでは、ラベルを使用して Batch リソースを整理する方法について説明します。

ラベルは、リソースに適用され、リソースをグループ化して記述する Key-Value ペアです。Batch には、リソースに自動的に適用される事前定義されたラベルと、ジョブの作成時に定義して適用できるカスタムラベルがあります。

ラベルを使用すると、リソースリストと Cloud Billing レポートの結果をフィルタできます。たとえば、ラベルを使用すると、次のことができます。

  • プロジェクトのジョブのリストを明確にして整理します。

  • ラベルを使用して、指定されたコンテナまたはスクリプトのタイプを記述し、ジョブの実行可能ファイルを区別します。

  • Batch または特定のジョブによって作成されたリソースの Cloud Billing レポートをフィルタリングして、費用を分析します。

ラベルの詳細については、Compute Engine のドキュメントでラベルをご覧ください。

準備

  1. Batch を以前に使用したことがない場合は、Batch を使ってみるを確認し、プロジェクトとユーザーの前提条件を完了して Batch を有効にします。

  2. ジョブの作成に必要な権限を取得するには、次の IAM ロールを付与するよう管理者に依頼してください。

    ロールの付与については、プロジェクト、フォルダ、組織へのアクセスを管理するをご覧ください。

    必要な権限は、カスタムロールや他の事前定義ロールから取得することもできます。

制限事項

Compute Engine のドキュメントで指定されたラベルの要件に加えて、Batch ジョブとそのリソースにラベルを適用するには、次の制限があります。

  • Batch では、Batchを使用して作成され、以下のタイプのリソースのラベルのみがサポートされます：

  • Batch が自動的にジョブに適用する事前定義されたラベルを考慮した後、次の量のカスタムラベルを定義できます。

    • ジョブとそのランナブルに適用するカスタムラベルは最大 63 個定義できます。

    • ジョブ用に作成された各 GPU、永続ディスク、VM に適用するカスタムラベルは最大 61 個定義できます。

  • Batch では、一意の名前を持つカスタムラベルの定義のみがサポートされています。結果として次のようになります。

    • 事前定義済みのラベルをオーバーライドしようとすると、エラーが発生します。

    • 重複するカスタムラベルを定義すると、既存のカスタムラベルがオーバーライドされます。

  • Batch では、ジョブの作成時にのみラベルの定義がサポートされています。

    • ジョブと実行可能ファイルのラベルを追加、更新、削除することはできません。

    • Compute Engine を使用して、ジョブ用に作成された永続ディスクと VM のラベルを追加、更新、削除することはできますが、おすすめしません。ジョブのリソースが存在する期間を正確に予測することはできず、変更が Batch で正しく機能しない可能性があります。

  • ラベルを使用してジョブのリストをフィルタするには、gcloud CLI または Batch API を使用してジョブのリストを表示する必要があります。

定義済みのラベル

各事前定義ラベルには、batch- 接頭辞で始まるキーがあります。デフォルトでは、Batch は次の事前定義ラベルを自動的に適用します。

  • 作成するジョブごとに、次の操作を行います。

    • batch-job-id: このラベルの値は、ジョブの名前に設定されます。

  • ジョブ用に作成された各 GPU、永続ディスク、VM に適用されます。

    • batch-job-id: このラベルの値は、ジョブの名前に設定されます。

    • batch-job-uid: このラベルの値は、ジョブの一意の識別子（UID）に設定されます。

    • batch-node: このラベルの値は null です。ジョブ用に作成されたすべての GPU、永続ディスク、VM をグループ化します。たとえば、Cloud 課金レポートを表示して、Batch によって作成されたすべての GPU、永続ディスク、VM の費用を特定する場合は、このラベルを使用します。

カスタムラベルを定義する

必要に応じて、ジョブの作成時に 1 つ以上のカスタムラベルを定義できます。カスタムラベルは、新しい鍵またはプロジェクトですでに使用している鍵で定義できます。カスタムラベルを定義するには、ラベルの目的に基づいて、このドキュメントの次のいずれかの方法を選択します。

  • ジョブとそのリソースのカスタムラベルを定義する。

    このセクションでは、1 つ以上のカスタムラベルをジョブと、ジョブ用に作成された各 GPU、永続ディスク、VM に適用する方法について説明します。ジョブを作成した後に、これらのラベルを使用して、Cloud 請求レポートとプロジェクトのジョブ、永続ディスク、VM のリストにフィルタを適用できます。

  • ジョブのカスタムラベルを定義する。

    このセクションでは、1 つ以上のカスタムラベルをジョブに適用する方法について説明します。ジョブを作成した後に、これらのラベルを使用してジョブのプロジェクト リストをフィルタリングできます。

  • 実行可能ファイルのカスタムラベルを定義する。

    このセクションでは、ジョブの 1 つ以上のランナブルに 1 つ以上のカスタムラベルを適用する方法について説明します。ジョブを作成した後に、これらのラベルを使用してジョブのプロジェクト リストをフィルタリングできます。

ジョブとそのリソースのカスタムラベルを定義する

ジョブの割り当てポリシーの labels フィールドで定義されたラベルが、ジョブおよびジョブ用に作成された各 GPU（存在する場合）、永続ディスク（すべてのブートディスクと任意の新しいストレージ ボリューム）、および VM に適用されます。

gcloud CLI または Batch API を使用してジョブを作成するときに、ジョブとそのリソースのラベルを定義できます。

gcloud

たとえば、ジョブおよびジョブ用に作成されたリソースに適用される 2 つのカスタムラベルを定義する基本的なコンテナジョブを us-central1 で作成するには、以下の手順に沿って操作します：

  1. ジョブの構成の詳細と allocationPolicy.labels フィールドを指定する JSON ファイルを作成します。

    {
  "allocationPolicy": {
    "instances": [
      {
        "policy": {
          "machineType": "e2-standard-4"
        }
      }
    ],
    "labels": {
      "VM_LABEL_NAME1": "VM_LABEL_VALUE1",
      "VM_LABEL_NAME2": "VM_LABEL_VALUE2"
    }
  },
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello world!"
              ]
            }
          }
        ]
      }
    }
  ]
}

    以下を置き換えます。

    • VM_LABEL_NAME1: ジョブ用に作成された VM に適用する最初のラベルの名前。

    • VM_LABEL_VALUE1: ジョブ用に作成された VM に適用する最初のラベルの値。

    • VM_LABEL_NAME2: ジョブ用に作成された VM に適用する 2 番目のラベルの名前。

    • VM_LABEL_VALUE2: ジョブ用に作成された VM に適用する 2 番目のラベルの値。

  2. gcloud batch jobs submit コマンドを使用して、us-central1 にジョブを作成します。

    gcloud batch jobs submit example-job \
    --config=JSON_CONFIGURATION_FILE \
    --location=us-central1

    JSON_CONFIGURATION_FILE は、前の手順で作成したジョブの構成の詳細を含む JSON ファイルのパスに置き換えます。

API

ジョブおよびジョブ用に作成されたリソースに適用される 2 つのカスタムラベルを定義する基本的なコンテナジョブを us-central1 で作成するには、jobs.create メソッド に POST リクエストを行い、allocationPolicy.labels フィールドを指定します。

POST https://batch.googleapis.com/v1/projects/example-project/locations/us-central1/jobs?job_id=example-job

{
  "allocationPolicy": {
    "instances": [
      {
        "policy": {
          "machineType": "e2-standard-4"
        }
      }
    ],
    "labels": {
      "VM_LABEL_NAME1": "VM_LABEL_VALUE1",
      "VM_LABEL_NAME2": "VM_LABEL_VALUE2"
    }
  },
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello world!"
              ]
            }
          }
        ]
      }
    }
  ]
}

以下を置き換えます。

  • VM_LABEL_NAME1: ジョブ用に作成された VM に適用する最初のラベルの名前。

  • VM_LABEL_VALUE1: ジョブ用に作成された VM に適用する最初のラベルの値。

  • VM_LABEL_NAME2: ジョブ用に作成された VM に適用する 2 番目のラベルの名前。

  • VM_LABEL_VALUE2: ジョブ用に作成された VM に適用する 2 番目のラベルの値。

Java


import com.google.cloud.batch.v1.AllocationPolicy;
import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateBatchAllocationPolicyLabel {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "us-central1";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "example-job";
    // Name of the label1 to be applied for your Job.
    String labelName1 = "VM_LABEL_NAME1";
    // Value for the label1 to be applied for your Job.
    String labelValue1 = "VM_LABEL_VALUE1";
    // Name of the label2 to be applied for your Job.
    String labelName2 = "VM_LABEL_NAME2";
    // Value for the label2 to be applied for your Job.
    String labelValue2 = "VM_LABEL_VALUE2";

    createBatchAllocationPolicyLabel(projectId, region, jobName, labelName1,
        labelValue1, labelName2, labelValue2);
  }

  // This method shows how to create a job with labels defined 
  // in the labels field of a job's allocation policy. These are 
  // applied to the job, as well as to each GPU (if any), persistent disk 
  // (all boot disks and any new storage volumes), and VM created for the job.
  public static Job createBatchAllocationPolicyLabel(String projectId, String region,
                               String jobName, String labelName1,
                               String labelValue1, String labelName2, String labelValue2)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setContainer(
                  Runnable.Container.newBuilder()
                      .setImageUri("gcr.io/google-containers/busybox")
                      .setEntrypoint("/bin/sh")
                      .addCommands("-c")
                      .addCommands(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                              + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      .build())
              .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(2000)
              // In MiB.
              .setMemoryMib(2000)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(1).setTaskSpec(task).build();

      // Policies are used to define on what kind of virtual machines the tasks will run on.
      // In this case, we tell the system to use "e2-standard-4" machine type.
      // Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
      AllocationPolicy.InstancePolicy instancePolicy =
          AllocationPolicy.InstancePolicy.newBuilder().setMachineType("e2-standard-4").build();

      AllocationPolicy allocationPolicy =
          AllocationPolicy.newBuilder()
              .addInstances(AllocationPolicy.InstancePolicyOrTemplate.newBuilder()
                  .setPolicy(instancePolicy)
                  .build())
              // Labels and their value to be applied to the job and its resources
              .putLabels(labelName1, labelValue1)
              .putLabels(labelName2, labelValue2)
              .build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              .setAllocationPolicy(allocationPolicy)
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
                      .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

}

Node.js

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-labels-allocation-job';
// Name of the label1 to be applied for your Job.
const labelName1 = 'vm_label_name_1';
// Value for the label1 to be applied for your Job.
const labelValue1 = 'vmLabelValue1';
// Name of the label2 to be applied for your Job.
const labelName2 = 'vm_label_name_2';
// Value for the label2 to be applied for your Job.
const labelValue2 = 'vmLabelValue2';

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  script: new batch.Runnable.Script({
    commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
  }),
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

// Policies are used to define on what kind of virtual machines the tasks will run on.
// In this case, we tell the system to use "e2-standard-4" machine type.
// Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
const instancePolicy = new batch.AllocationPolicy.InstancePolicy({
  machineType: 'e2-standard-4',
});

const allocationPolicy = new batch.AllocationPolicy({
  instances: [{policy: instancePolicy}],
});
// Labels and their value to be applied to the job and its resources.
allocationPolicy.labels[labelName1] = labelValue1;
allocationPolicy.labels[labelName2] = labelValue2;

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  labels: {env: 'testing', type: 'script'},
  allocationPolicy,
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchLabelsAllocation() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchLabelsAllocation();

Python

from google.cloud import batch_v1


def create_job_with_custom_allocation_policy_labels(
    project_id: str, region: str, job_name: str, labels: dict
) -> batch_v1.Job:
    """
    This method shows the creation of a Batch job with custom labels which describe the allocation policy.
    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
        labels (dict): a dictionary of key-value pairs that will be used as labels
            E.g., {"label_key1": "label_value2", "label_key2": "label_value2"}
    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    runnable = batch_v1.Runnable()
    runnable.container = batch_v1.Runnable.Container()
    runnable.container.image_uri = "gcr.io/google-containers/busybox"
    runnable.container.entrypoint = "/bin/sh"
    runnable.container.commands = [
        "-c",
        "echo Hello world!",
    ]

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 3
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Assign the provided labels to the allocation policy
    allocation_policy.labels = labels

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy

    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

ジョブのカスタムラベルを定義する

ジョブの labels フィールドで定義されたラベルは、ジョブにのみ適用されます。

gcloud CLI または Batch API を使用してジョブを作成するときに、ジョブのラベルを定義できます。

gcloud

たとえば、ジョブ自体に適用される 2 つのカスタムラベルを定義する基本的なコンテナジョブを us-central1 に作成するには、次のようにします。

  1. ジョブの構成の詳細と labels フィールドを指定する JSON ファイルを作成します。

    {
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello World!"
              ]
            }
          }
        ]
      }
    }
  ],
  "labels": {
    "JOB_LABEL_NAME1": "JOB_LABEL_VALUE1",
    "JOB_LABEL_NAME2": "JOB_LABEL_VALUE2"
  }
}

    以下を置き換えます。

    • JOB_LABEL_NAME1: ジョブに適用する最初のラベルの名前。

    • JOB_LABEL_VALUE1: ジョブに適用する最初のラベルの値。

    • JOB_LABEL_NAME2: ジョブに適用する 2 番目のラベルの名前。

    • JOB_LABEL_VALUE2: ジョブに適用する 2 番目のラベルの値。

  2. 次のフラグを指定して gcloud batch jobs submit コマンドを使用して、us-central1 にジョブを作成します。

    gcloud batch jobs submit example-job \
    --config=JSON_CONFIGURATION_FILE \
    --location=us-central1

    JSON_CONFIGURATION_FILE は、前の手順で作成したジョブの構成の詳細を含む JSON ファイルのパスに置き換えます。

API

たとえば、ジョブ自体に適用される 2 つのカスタムラベルを定義するコンテナジョブを us-central1 に作成するには、jobs.create メソッド に POST リクエストを行い、labels フィールドを指定します。

POST https://batch.googleapis.com/v1/projects/example-project/locations/us-central1/jobs?job_id=example-job

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello World!"
              ]
            }
          }
        ]
      }
    }
  ],
  "labels": {
    "JOB_LABEL_NAME1": "JOB_LABEL_VALUE1",
    "JOB_LABEL_NAME2": "JOB_LABEL_VALUE2"
  }
}

以下を置き換えます。

  • JOB_LABEL_NAME1: ジョブに適用する最初のラベルの名前。

  • JOB_LABEL_VALUE1: ジョブに適用する最初のラベルの値。

  • JOB_LABEL_NAME2: ジョブに適用する 2 番目のラベルの名前。

  • JOB_LABEL_VALUE2: ジョブに適用する 2 番目のラベルの値。

Java


import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;


public class CreateBatchLabelJob {

  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "us-central1";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "example-job";
    // Name of the label1 to be applied for your Job.
    String labelName1 = "JOB_LABEL_NAME1";
    // Value for the label1 to be applied for your Job.
    String labelValue1 = "JOB_LABEL_VALUE1";
    // Name of the label2 to be applied for your Job.
    String labelName2 = "JOB_LABEL_NAME2";
    // Value for the label2 to be applied for your Job.
    String labelValue2 = "JOB_LABEL_VALUE2";

    createBatchLabelJob(projectId, region, jobName, labelName1,
        labelValue1, labelName2, labelValue2);
  }

  // Creates a job with labels defined in the labels field.
  public static Job createBatchLabelJob(String projectId, String region, String jobName,
                    String labelName1, String labelValue1, String labelName2, String labelValue2)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setContainer(
                  Runnable.Container.newBuilder()
                      .setImageUri("gcr.io/google-containers/busybox")
                      .setEntrypoint("/bin/sh")
                      .addCommands("-c")
                      .addCommands(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                              + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      .build())
              .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(2000)
              // In MiB.
              .setMemoryMib(2000)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(1).setTaskSpec(task).build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
              .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              // Labels and their value to be applied to the job.
              .putLabels(labelName1, labelValue1)
              .putLabels(labelName2, labelValue2)
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

}

Node.js

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'europe-central2';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'batch-labels-job';
// Name of the label1 to be applied for your Job.
const labelName1 = 'job_label_name_1';
// Value for the label1 to be applied for your Job.
const labelValue1 = 'job_label_value1';
// Name of the label2 to be applied for your Job.
const labelName2 = 'job_label_name_2';
// Value for the label2 to be applied for your Job.
const labelValue2 = 'job_label_value2';

// Define what will be done as part of the job.
const runnable = new batch.Runnable({
  container: new batch.Runnable.Container({
    imageUri: 'gcr.io/google-containers/busybox',
    entrypoint: '/bin/sh',
    commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
  }),
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// Labels and their value to be applied to the job and its resources.
job.labels[labelName1] = labelValue1;
job.labels[labelName2] = labelValue2;

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchLabelsJob() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchLabelsJob();

Python

from google.cloud import batch_v1


def create_job_with_custom_job_labels(
    project_id: str,
    region: str,
    job_name: str,
    labels: dict,
) -> batch_v1.Job:
    """
    This method creates a Batch job with custom labels.
    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
        labels (dict): A dictionary of custom labels to be added to the job.
            E.g., {"label_key1": "label_value2", "label_key2": "label_value2"}
    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    runnable = batch_v1.Runnable()
    runnable.container = batch_v1.Runnable.Container()
    runnable.container.image_uri = "gcr.io/google-containers/busybox"
    runnable.container.entrypoint = "/bin/sh"
    runnable.container.commands = [
        "-c",
        "echo Hello world!",
    ]

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 3
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy

    # Set the labels for the job
    job.labels = labels

    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

実行可能ファイルのカスタムラベルを定義する

実行可能ファイルの labels フィールドで定義されたラベルは、その実行可能ファイルにのみ適用されます。

gcloud CLI または Batch API を使用してジョブを作成するときに、1 つ以上のランナブルのラベルを定義できます。

gcloud

たとえば、2 つのジョブの実行可能ファイルのそれぞれに 1 つずつ、2 つのカスタムラベルを定義するジョブを us-central1 に作成するには、次の操作を行います。

  1. ジョブの構成の詳細と runnables.labels フィールドを指定する JSON ファイルを作成します。

    {
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello from task ${BATCH_TASK_INDEX}!"
              ]
            },
            "labels": {
              "RUNNABLE1_LABEL_NAME1": "RUNNABLE1_LABEL_VALUE1"
            }
          },
          {
            "script": {
              "text": "echo Hello from task ${BATCH_TASK_INDEX}!"
            },
            "labels": {
              "RUNNABLE2_LABEL_NAME1": "RUNNABLE2_LABEL_VALUE1"
            }
          }
        ]
      }
    }
  ]
}

    以下を置き換えます。

    • RUNNABLE1_LABEL_NAME1: ジョブの最初の実行可能ファイルに適用するラベルの名前。

    • RUNNABLE1_LABEL_VALUE1: ジョブの最初の実行可能ファイルに適用するラベルの値。

    • RUNNABLE2_LABEL_NAME1: ジョブの 2 番目の実行可能ファイルに適用するラベルの名前。

    • RUNNABLE2_LABEL_VALUE1: ジョブの 2 番目の実行可能ファイルに適用するラベルの値。

  2. gcloud batch jobs submit コマンドを使用して、us-central1 にジョブを作成します。

    gcloud batch jobs submit example-job \
    --config=JSON_CONFIGURATION_FILE \
    --location=us-central1

    JSON_CONFIGURATION_FILE は、前の手順で作成したジョブの構成の詳細を含む JSON ファイルのパスに置き換えます。

API

たとえば、2 つのジョブの実行可能ファイルのそれぞれに 1 つずつ、2 つのカスタムラベルを定義するジョブを us-central1 に作成するには、jobs.create メソッド に POST リクエストを行い、runnables.labels フィールドを指定します。

POST https://batch.googleapis.com/v1/projects/example-project/locations/us-central1/jobs?job_id=example-job

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "container": {
              "imageUri": "gcr.io/google-containers/busybox",
              "entrypoint": "/bin/sh",
              "commands": [
                "-c",
                "echo Hello from ${BATCH_TASK_INDEX}!"
              ]
            },
            "labels": {
              "RUNNABLE1_LABEL_NAME1": "RUNNABLE1_LABEL_VALUE1"
            }
          },
          {
            "script": {
              "text": "echo Hello from ${BATCH_TASK_INDEX}!"
            },
            "labels": {
              "RUNNABLE2_LABEL_NAME1": "RUNNABLE2_LABEL_VALUE1"
            }
          }
        ]
      }
    }
  ]
}

以下を置き換えます。

  • RUNNABLE1_LABEL_NAME1: 最初のジョブの実行可能ファイルに適用するラベルの名前。

  • RUNNABLE1_LABEL_VALUE1: 最初のジョブの実行可能ファイルに適用するラベルの値。

  • RUNNABLE2_LABEL_NAME1: 2 番目のジョブの実行可能ファイルに適用するラベルの名前。

  • RUNNABLE2_LABEL_VALUE1: 2 番目のジョブの実行可能ファイルに適用するラベルの値。

Java


import com.google.cloud.batch.v1.BatchServiceClient;
import com.google.cloud.batch.v1.ComputeResource;
import com.google.cloud.batch.v1.CreateJobRequest;
import com.google.cloud.batch.v1.Job;
import com.google.cloud.batch.v1.LogsPolicy;
import com.google.cloud.batch.v1.Runnable;
import com.google.cloud.batch.v1.TaskGroup;
import com.google.cloud.batch.v1.TaskSpec;
import com.google.protobuf.Duration;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class CreateBatchRunnableLabel {
  public static void main(String[] args)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // TODO(developer): Replace these variables before running the sample.
    // Project ID or project number of the Google Cloud project you want to use.
    String projectId = "YOUR_PROJECT_ID";
    // Name of the region you want to use to run the job. Regions that are
    // available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
    String region = "us-central1";
    // The name of the job that will be created.
    // It needs to be unique for each project and region pair.
    String jobName = "example-job";
    // Name of the label1 to be applied for your Job.
    String labelName1 = "RUNNABLE_LABEL_NAME1";
    // Value for the label1 to be applied for your Job.
    String labelValue1 = "RUNNABLE_LABEL_VALUE1";
    // Name of the label2 to be applied for your Job.
    String labelName2 = "RUNNABLE_LABEL_NAME2";
    // Value for the label2 to be applied for your Job.
    String labelValue2 = "RUNNABLE_LABEL_VALUE2";

    createBatchRunnableLabel(projectId, region, jobName, labelName1,
        labelValue1, labelName2, labelValue2);
  }

  // Creates a job with labels defined in the labels field
  // for a runnable. The labels are only applied to that runnable.
  // In Batch, a runnable represents a single task or unit of work within a job.
  // It can be a container (like a Docker image) or a script.
  public static Job createBatchRunnableLabel(String projectId, String region, String jobName,
                   String labelName1, String labelValue1, String labelName2, String labelValue2)
      throws IOException, ExecutionException, InterruptedException, TimeoutException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (BatchServiceClient batchServiceClient = BatchServiceClient.create()) {

      // Define what will be done as part of the job.
      Runnable runnable =
          Runnable.newBuilder()
              .setContainer(
                  Runnable.Container.newBuilder()
                      .setImageUri("gcr.io/google-containers/busybox")
                      .setEntrypoint("/bin/sh")
                      .addCommands("-c")
                      .addCommands(
                          "echo Hello world! This is task ${BATCH_TASK_INDEX}. "
                              + "This job has a total of ${BATCH_TASK_COUNT} tasks.")
                      .build())
              // Label and its value to be applied to the container
              // that processes data from a specific region.
              .putLabels(labelName1, labelValue1)
              .setScript(Runnable.Script.newBuilder()
              .setText("echo Hello world! This is task ${BATCH_TASK_INDEX}. ").build())
              // Label and its value to be applied to the script
              // that performs some analysis on the processed data.
              .putLabels(labelName2, labelValue2)
              .build();

      // We can specify what resources are requested by each task.
      ComputeResource computeResource =
          ComputeResource.newBuilder()
              // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
              .setCpuMilli(2000)
              // In MiB.
              .setMemoryMib(2000)
              .build();

      TaskSpec task =
          TaskSpec.newBuilder()
              // Jobs can be divided into tasks. In this case, we have only one task.
              .addRunnables(runnable)
              .setComputeResource(computeResource)
              .setMaxRetryCount(2)
              .setMaxRunDuration(Duration.newBuilder().setSeconds(3600).build())
              .build();

      // Tasks are grouped inside a job using TaskGroups.
      // Currently, it's possible to have only one task group.
      TaskGroup taskGroup = TaskGroup.newBuilder().setTaskCount(1).setTaskSpec(task).build();

      Job job =
          Job.newBuilder()
              .addTaskGroups(taskGroup)
              // We use Cloud Logging as it's an out of the box available option.
              .setLogsPolicy(LogsPolicy.newBuilder()
              .setDestination(LogsPolicy.Destination.CLOUD_LOGGING).build())
              .build();

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              // The job's parent is the region in which the job will run for the specific project.
              .setParent(String.format("projects/%s/locations/%s", projectId, region))
              .setJob(job)
              .setJobId(jobName)
              .build();

      Job result =
          batchServiceClient
              .createJobCallable()
              .futureCall(createJobRequest)
              .get(5, TimeUnit.MINUTES);

      System.out.printf("Successfully created the job: %s", result.getName());

      return result;
    }
  }

}

Node.js

// Imports the Batch library
const batchLib = require('@google-cloud/batch');
const batch = batchLib.protos.google.cloud.batch.v1;

// Instantiates a client
const batchClient = new batchLib.v1.BatchServiceClient();

/**
 * TODO(developer): Update these variables before running the sample.
 */
// Project ID or project number of the Google Cloud project you want to use.
const projectId = await batchClient.getProjectId();
// Name of the region you want to use to run the job. Regions that are
// available for Batch are listed on: https://cloud.google.com/batch/docs/get-started#locations
const region = 'us-central1';
// The name of the job that will be created.
// It needs to be unique for each project and region pair.
const jobName = 'example-job';
// Name of the label1 to be applied for your Job.
const labelName1 = 'RUNNABLE_LABEL_NAME1';
// Value for the label1 to be applied for your Job.
const labelValue1 = 'RUNNABLE_LABEL_VALUE1';
// Name of the label2 to be applied for your Job.
const labelName2 = 'RUNNABLE_LABEL_NAME2';
// Value for the label2 to be applied for your Job.
const labelValue2 = 'RUNNABLE_LABEL_VALUE2';

const container = new batch.Runnable.Container({
  imageUri: 'gcr.io/google-containers/busybox',
  entrypoint: '/bin/sh',
  commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
});

const script = new batch.Runnable.Script({
  commands: ['-c', 'echo Hello world! This is task ${BATCH_TASK_INDEX}.'],
});

const runnable1 = new batch.Runnable({
  container,
  // Label and its value to be applied to the container
  // that processes data from a specific region.
  labels: {
    [labelName1]: labelValue1,
  },
});

const runnable2 = new batch.Runnable({
  script,
  // Label and its value to be applied to the script
  // that performs some analysis on the processed data.
  labels: {
    [labelName2]: labelValue2,
  },
});

// Specify what resources are requested by each task.
const computeResource = new batch.ComputeResource({
  // In milliseconds per cpu-second. This means the task requires 50% of a single CPUs.
  cpuMilli: 500,
  // In MiB.
  memoryMib: 16,
});

const task = new batch.TaskSpec({
  runnables: [runnable1, runnable2],
  computeResource,
  maxRetryCount: 2,
  maxRunDuration: {seconds: 3600},
});

// Tasks are grouped inside a job using TaskGroups.
const group = new batch.TaskGroup({
  taskCount: 3,
  taskSpec: task,
});

const job = new batch.Job({
  name: jobName,
  taskGroups: [group],
  // We use Cloud Logging as it's an option available out of the box
  logsPolicy: new batch.LogsPolicy({
    destination: batch.LogsPolicy.Destination.CLOUD_LOGGING,
  }),
});

// The job's parent is the project and region in which the job will run
const parent = `projects/${projectId}/locations/${region}`;

async function callCreateBatchLabelsRunnable() {
  // Construct request
  const request = {
    parent,
    jobId: jobName,
    job,
  };

  // Run request
  const [response] = await batchClient.createJob(request);
  console.log(JSON.stringify(response));
}

await callCreateBatchLabelsRunnable();

Python

from google.cloud import batch_v1


def create_job_with_custom_runnables_labels(
    project_id: str,
    region: str,
    job_name: str,
    labels: dict,
) -> batch_v1.Job:
    """
    This method creates a Batch job with custom labels for runnable.
    Args:
        project_id (str): project ID or project number of the Cloud project you want to use.
        region (str): name of the region you want to use to run the job. Regions that are
            available for Batch are listed on: https://cloud.google.com/batch/docs/locations
        job_name (str): the name of the job that will be created.
        labels (dict): a dictionary of key-value pairs that will be used as labels
            E.g., {"label_key1": "label_value2"}
    Returns:
        batch_v1.Job: The created Batch job object containing configuration details.
    """
    client = batch_v1.BatchServiceClient()

    runnable = batch_v1.Runnable()
    runnable.display_name = "Script 1"
    runnable.script = batch_v1.Runnable.Script()
    runnable.script.text = "echo Hello world from Script 1 for task ${BATCH_TASK_INDEX}"
    # Add custom labels to the first runnable
    runnable.labels = labels

    # Create a task specification and assign the runnable and volume to it
    task = batch_v1.TaskSpec()
    task.runnables = [runnable]

    # Specify what resources are requested by each task.
    resources = batch_v1.ComputeResource()
    resources.cpu_milli = 2000  # in milliseconds per cpu-second. This means the task requires 2 whole CPUs.
    resources.memory_mib = 16  # in MiB
    task.compute_resource = resources

    task.max_retry_count = 2
    task.max_run_duration = "3600s"

    # Create a task group and assign the task specification to it
    group = batch_v1.TaskGroup()
    group.task_count = 3
    group.task_spec = task

    # Policies are used to define on what kind of virtual machines the tasks will run on.
    # In this case, we tell the system to use "e2-standard-4" machine type.
    # Read more about machine types here: https://cloud.google.com/compute/docs/machine-types
    policy = batch_v1.AllocationPolicy.InstancePolicy()
    policy.machine_type = "e2-standard-4"
    instances = batch_v1.AllocationPolicy.InstancePolicyOrTemplate()
    instances.policy = policy
    allocation_policy = batch_v1.AllocationPolicy()
    allocation_policy.instances = [instances]

    # Create the job and assign the task group and allocation policy to it
    job = batch_v1.Job()
    job.task_groups = [group]
    job.allocation_policy = allocation_policy

    # We use Cloud Logging as it's an out of the box available option
    job.logs_policy = batch_v1.LogsPolicy()
    job.logs_policy.destination = batch_v1.LogsPolicy.Destination.CLOUD_LOGGING

    # Create the job request and set the job and job ID
    create_request = batch_v1.CreateJobRequest()
    create_request.job = job
    create_request.job_id = job_name
    # The job's parent is the region in which the job will run
    create_request.parent = f"projects/{project_id}/locations/{region}"

    return client.create_job(create_request)

次のステップ