REST Resource: projects.locations.jobs

Resource: Job

The Cloud Batch Job description.

JSON representation 
{
  "name": string,
  "uid": string,
  "priority": string,
  "taskGroups": [
    {
      object (TaskGroup)
    }
  ],
  "schedulingPolicy": enum (SchedulingPolicy),
  "dependencies": [
    {
      object (JobDependency)
    }
  ],
  "allocationPolicy": {
    object (AllocationPolicy)
  },
  "labels": {
    string: string,
    ...
  },
  "status": {
    object (JobStatus)
  },
  "notification": {
    object (JobNotification)
  },
  "createTime": string,
  "updateTime": string,
  "logsPolicy": {
    object (LogsPolicy)
  },
  "notifications": [
    {
      object (JobNotification)
    }
  ]
}
Fields
name

string

Output only. Job name. For example: "projects/123456/locations/us-central1/jobs/job01".
uid

string

Output only. A system generated unique ID (in UUID4 format) for the Job.
priority

string (int64 format)

Priority of the Job. The valid value range is [0, 100). Default value is 0. Higher value indicates higher priority. A job with higher priority value is more likely to run earlier if all other requirements are satisfied.
taskGroups[]

object (TaskGroup)

Required. TaskGroups in the Job. Only one TaskGroup is supported now.
schedulingPolicy

enum (SchedulingPolicy)

Scheduling policy for TaskGroups in the job.
dependencies[]

object (JobDependency)

At least one of the dependencies must be satisfied before the Job is scheduled to run. Only one JobDependency is supported now. Not yet implemented.
allocationPolicy

object (AllocationPolicy)

Compute resource allocation for all TaskGroups in the Job.
labels

map (key: string, value: string)

Labels for the Job. Labels could be user provided or system generated. For example, "labels": { "department": "finance", "environment": "test" } You can assign up to 64 labels. Google Compute Engine label restrictions apply. Label names that start with "goog-" or "google-" are reserved.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
status

object (JobStatus)

Output only. Job status. It is read only for users.
notification
(deprecated)

object (JobNotification)

Deprecated: please use notifications instead.
createTime

string (Timestamp format)

Output only. When the Job was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. The last time the Job was updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
logsPolicy

object (LogsPolicy)

Log preservation policy for the Job.
notifications[]

object (JobNotification)

Notification configurations.

TaskGroup

A TaskGroup defines one or more Tasks that all share the same TaskSpec.

JSON representation 
{
  "name": string,
  "taskSpec": {
    object (TaskSpec)
  },
  "taskCount": string,
  "parallelism": string,
  "schedulingPolicy": enum (SchedulingPolicy),
  "allocationPolicy": {
    object (AllocationPolicy)
  },
  "labels": {
    string: string,
    ...
  },
  "taskEnvironments": [
    {
      object (Environment)
    }
  ],
  "taskCountPerNode": string,
  "requireHostsFile": boolean,
  "permissiveSsh": boolean
}
Fields
name

string

Output only. TaskGroup name. The system generates this field based on parent Job name. For example: "projects/123456/locations/us-west1/jobs/job01/taskGroups/group01".
taskSpec

object (TaskSpec)

Required. Tasks in the group share the same task spec.
taskCount

string (int64 format)

Number of Tasks in the TaskGroup. Default is 1.
parallelism

string (int64 format)

Max number of tasks that can run in parallel. Default to min(taskCount, 1000). Field parallelism must be 1 if the schedulingPolicy is IN_ORDER.
schedulingPolicy

enum (SchedulingPolicy)

Scheduling policy for Tasks in the TaskGroup. The default value is AS_SOON_AS_POSSIBLE.
allocationPolicy

object (AllocationPolicy)

Compute resource allocation for the TaskGroup. If specified, it overrides resources in Job.
labels

map (key: string, value: string)

Labels for the TaskGroup. Labels could be user provided or system generated. You can assign up to 64 labels. Google Compute Engine label restrictions apply. Label names that start with "goog-" or "google-" are reserved.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
taskEnvironments[]

object (Environment)

An array of environment variable mappings, which are passed to Tasks with matching indices. If taskEnvironments is used then taskCount should not be specified in the request (and will be ignored). Task count will be the length of taskEnvironments.

Tasks get a BATCH_TASK_INDEX and BATCH_TASK_COUNT environment variable, in addition to any environment variables set in taskEnvironments, specifying the number of Tasks in the Task's parent TaskGroup, and the specific Task's index in the TaskGroup (0 through BATCH_TASK_COUNT - 1).
taskCountPerNode

string (int64 format)

Max number of tasks that can be run on a VM at the same time. If not specified, the system will decide a value based on available compute resources on a VM and task requirements.
requireHostsFile

boolean

When true, Batch will populate a file with a list of all VMs assigned to the TaskGroup and set the BATCH_HOSTS_FILE environment variable to the path of that file. Defaults to false.
permissiveSsh

boolean

When true, Batch will configure SSH to allow passwordless login between VMs running the Batch tasks in the same TaskGroup.

TaskSpec

Spec of a task

JSON representation 
{
  "runnables": [
    {
      object (Runnable)
    }
  ],
  "computeResource": {
    object (ComputeResource)
  },
  "maxRunDuration": string,
  "maxRetryCount": integer,
  "lifecyclePolicies": [
    {
      object (LifecyclePolicy)
    }
  ],
  "environments": {
    string: string,
    ...
  },
  "volumes": [
    {
      object (Volume)
    }
  ],
  "environment": {
    object (Environment)
  }
}
Fields
runnables[]

object (Runnable)

The sequence of scripts or containers to run for this Task. Each Task using this TaskSpec executes its list of runnables in order. The Task succeeds if all of its runnables either exit with a zero status or any that exit with a non-zero status have the ignoreExitStatus flag.

Background runnables are killed automatically (if they have not already exited) a short time after all foreground runnables have completed. Even though this is likely to result in a non-zero exit status for the background runnable, these automatic kills are not treated as Task failures.
computeResource

object (ComputeResource)

ComputeResource requirements.
maxRunDuration

string (Duration format)

Maximum duration the task should run. The task will be killed and marked as FAILED if over this limit.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
maxRetryCount

integer

Maximum number of retries on failures. The default, 0, which means never retry. The valid value range is [0, 10].
lifecyclePolicies[]

object (LifecyclePolicy)

Lifecycle management schema when any task in a task group is failed. Currently we only support one lifecycle policy. When the lifecycle policy condition is met, the action in the policy will execute. If task execution result does not meet with the defined lifecycle policy, we consider it as the default policy. Default policy means if the exit code is 0, exit task. If task ends with non-zero exit code, retry the task with maxRetryCount.
environments
(deprecated)

map (key: string, value: string)

Deprecated: please use environment(non-plural) instead.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
volumes[]

object (Volume)

Volumes to mount before running Tasks using this TaskSpec.
environment

object (Environment)

Environment variables to set before running the Task.

Runnable

Runnable describes instructions for executing a specific script or container as part of a Task.

JSON representation 
{
  "displayName": string,
  "ignoreExitStatus": boolean,
  "background": boolean,
  "alwaysRun": boolean,
  "environment": {
    object (Environment)
  },
  "timeout": string,
  "labels": {
    string: string,
    ...
  },

  // Union field executable can be only one of the following:
  "container": {
    object (Container)
  },
  "script": {
    object (Script)
  },
  "barrier": {
    object (Barrier)
  }
  // End of list of possible types for union field executable.
}
Fields
displayName

string

Optional. DisplayName is an optional field that can be provided by the caller. If provided, it will be used in logs and other outputs to identify the script, making it easier for users to understand the logs. If not provided the index of the runnable will be used for outputs.
ignoreExitStatus

boolean

Normally, a non-zero exit status causes the Task to fail. This flag allows execution of other Runnables to continue instead.
background

boolean

This flag allows a Runnable to continue running in the background while the Task executes subsequent Runnables. This is useful to provide services to other Runnables (or to provide debugging support tools like SSH servers).
alwaysRun

boolean

By default, after a Runnable fails, no further Runnable are executed. This flag indicates that this Runnable must be run even if the Task has already failed. This is useful for Runnables that copy output files off of the VM or for debugging.

The alwaysRun flag does not override the Task's overall maxRunDuration. If the maxRunDuration has expired then no further Runnables will execute, not even alwaysRun Runnables.
environment

object (Environment)

Environment variables for this Runnable (overrides variables set for the whole Task or TaskGroup).
timeout

string (Duration format)

Timeout for this Runnable.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
labels

map (key: string, value: string)

Labels for this Runnable.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
Union field executable. The script or container to run. executable can be only one of the following:
container

object (Container)

Container runnable.
script

object (Script)

Script runnable.
barrier

object (Barrier)

Barrier runnable.

Container

Container runnable.

JSON representation 
{
  "imageUri": string,
  "commands": [
    string
  ],
  "entrypoint": string,
  "volumes": [
    string
  ],
  "options": string,
  "blockExternalNetwork": boolean,
  "username": string,
  "password": string,
  "enableImageStreaming": boolean
}
Fields
imageUri

string

The URI to pull the container image from.
commands[]

string

Overrides the CMD specified in the container. If there is an ENTRYPOINT (either in the container image or with the entrypoint field below) then commands are appended as arguments to the ENTRYPOINT.
entrypoint

string

Overrides the ENTRYPOINT specified in the container.
volumes[]

string

Volumes to mount (bind mount) from the host machine files or directories into the container, formatted to match docker run's --volume option, e.g. /foo:/bar, or /foo:/bar:ro

If the TaskSpec.Volumes field is specified but this field is not, Batch will mount each volume from the host machine to the container with the same mount path by default. In this case, the default mount option for containers will be read-only (ro) for existing persistent disks and read-write (rw) for other volume types, regardless of the original mount options specified in TaskSpec.Volumes. If you need different mount settings, you can explicitly configure them in this field.
options

string

Arbitrary additional options to include in the "docker run" command when running this container, e.g. "--network host".
blockExternalNetwork

boolean

If set to true, external network access to and from container will be blocked, containers that are with blockExternalNetwork as true can still communicate with each other, network cannot be specified in the container.options field.
username

string

Optional username for logging in to a docker registry. If username matches projects/*/secrets/*/versions/* then Batch will read the username from the Secret Manager.
password

string

Optional password for logging in to a docker registry. If password matches projects/*/secrets/*/versions/* then Batch will read the password from the Secret Manager;
enableImageStreaming

boolean

Optional. Not yet implemented. If set to true, container will run with Image streaming. The container runtime will be changed to containerd instead of docker. Currently, only imageUri, commands, entrypoint and volumes are supported and any other fields will be ignored.

Script

Script runnable.

JSON representation 
{

  // Union field command can be only one of the following:
  "path": string,
  "text": string
  // End of list of possible types for union field command.
}
Fields

Union field command.

command can be only one of the following:
path

string

Script file path on the host VM.

To specify an interpreter, please add a #!<interpreter>(also known as [shebang line](https://en.wikipedia.org/wiki/Shebang_(Unix))) as the first line of the file.(For example, to execute the script using bash, #!/bin/bash should be the first line of the file. To execute the script usingPython3, #!/usr/bin/env python3 should be the first line of the file.) Otherwise, the file will by default be excuted by /bin/sh.
text

string

Shell script text.

To specify an interpreter, please add a #!<interpreter>\n at the beginning of the text.(For example, to execute the script using bash, #!/bin/bash\n should be added. To execute the script usingPython3, #!/usr/bin/env python3\n should be added.) Otherwise, the script will by default be excuted by /bin/sh.

Barrier

Barrier runnable blocks until all tasks in a taskgroup reach it.

JSON representation 
{
  "name": string
}
Fields
name

string

Barriers are identified by their index in runnable list. Names are not required, but if present should be an identifier.

Environment

An Environment describes a collection of environment variables to set when executing Tasks.

JSON representation 
{
  "variables": {
    string: string,
    ...
  },
  "secretVariables": {
    string: string,
    ...
  },
  "encryptedVariables": {
    object (KMSEnvMap)
  }
}
Fields
variables

map (key: string, value: string)

A map of environment variable names to values.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
secretVariables

map (key: string, value: string)

A map of environment variable names to Secret Manager secret names. The VM will access the named secrets to set the value of each environment variable.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
encryptedVariables

object (KMSEnvMap)

An encrypted JSON dictionary where the key/value pairs correspond to environment variable names and their values.

KMSEnvMap

JSON representation 
{
  "keyName": string,
  "cipherText": string
}
Fields
keyName

string

The name of the KMS key that will be used to decrypt the cipher text.
cipherText

string

The value of the cipherText response from the encrypt method.

ComputeResource

Compute resource requirements.

ComputeResource defines the amount of resources required for each task. Make sure your tasks have enough resources to successfully run. If you also define the types of resources for a job to use with the InstancePolicyOrTemplate field, make sure both fields are compatible with each other.

JSON representation 
{
  "cpuMilli": string,
  "memoryMib": string,
  "gpuCount": string,
  "bootDiskMib": string
}
Fields
cpuMilli

string (int64 format)

The milliCPU count.

cpuMilli defines the amount of CPU resources per task in milliCPU units. For example, 1000 corresponds to 1 vCPU per task. If undefined, the default value is 2000.

If you also define the VM's machine type using the machineType in InstancePolicy field or inside the instanceTemplate in the InstancePolicyOrTemplate field, make sure the CPU resources for both fields are compatible with each other and with how many tasks you want to allow to run on the same VM at the same time.

For example, if you specify the n2-standard-2 machine type, which has 2 vCPUs each, you are recommended to set cpuMilli no more than 2000, or you are recommended to run two tasks on the same VM if you set cpuMilli to 1000 or less.
memoryMib

string (int64 format)

Memory in MiB.

memoryMib defines the amount of memory per task in MiB units. If undefined, the default value is 2000. If you also define the VM's machine type using the machineType in InstancePolicy field or inside the instanceTemplate in the InstancePolicyOrTemplate field, make sure the memory resources for both fields are compatible with each other and with how many tasks you want to allow to run on the same VM at the same time.

For example, if you specify the n2-standard-2 machine type, which has 8 GiB each, you are recommended to set memoryMib to no more than 8192, or you are recommended to run two tasks on the same VM if you set memoryMib to 4096 or less.
gpuCount

string (int64 format)

The GPU count.

Not yet implemented.
bootDiskMib

string (int64 format)

Extra boot disk size in MiB for each task.

LifecyclePolicy

LifecyclePolicy describes how to deal with task failures based on different conditions.

JSON representation 
{
  "action": enum (Action),
  "actionCondition": {
    object (ActionCondition)
  }
}
Fields
action

enum (Action)

Action to execute when ActionCondition is true. When RETRY_TASK is specified, we will retry failed tasks if we notice any exit code match and fail tasks if no match is found. Likewise, when FAIL_TASK is specified, we will fail tasks if we notice any exit code match and retry tasks if no match is found.
actionCondition

object (ActionCondition)

Conditions that decide why a task failure is dealt with a specific action.

Action

Action on task failures based on different conditions.

Enums
ACTION_UNSPECIFIED Action unspecified.
RETRY_TASK Action that tasks in the group will be scheduled to re-execute.
FAIL_TASK Action that tasks in the group will be stopped immediately.

ActionCondition

Conditions for actions to deal with task failures.

JSON representation 
{
  "exitCodes": [
    integer
  ]
}
Fields
exitCodes[]

integer

Exit codes of a task execution. If there are more than 1 exit codes, when task executes with any of the exit code in the list, the condition is met and the action will be executed.

Volume

Volume describes a volume and parameters for it to be mounted to a VM.

JSON representation 
{
  "mountPath": string,
  "mountOptions": [
    string
  ],

  // Union field source can be only one of the following:
  "nfs": {
    object (NFS)
  },
  "pd": {
    object (PD)
  },
  "gcs": {
    object (GCS)
  },
  "deviceName": string
  // End of list of possible types for union field source.
}
Fields
mountPath

string

The mount path for the volume, e.g. /mnt/disks/share.
mountOptions[]

string

For Google Cloud Storage (GCS), mount options are the options supported by the gcsfuse tool (https://github.com/GoogleCloudPlatform/gcsfuse). For existing persistent disks, mount options provided by the mount command (https://man7.org/linux/man-pages/man8/mount.8.html) except writing are supported. This is due to restrictions of multi-writer mode (https://cloud.google.com/compute/docs/disks/sharing-disks-between-vms). For other attached disks and Network File System (NFS), mount options are these supported by the mount command (https://man7.org/linux/man-pages/man8/mount.8.html).
Union field source. The source for the volume. source can be only one of the following:
nfs

object (NFS)

A Network File System (NFS) volume. For example, a Filestore file share.
pd
(deprecated)

object (PD)

Deprecated: please use deviceName instead.
gcs

object (GCS)

A Google Cloud Storage (GCS) volume.
deviceName

string

Device name of an attached disk volume, which should align with a deviceName specified by job.allocation_policy.instances[0].policy.disks[i].device_name or defined by the given instance template in job.allocation_policy.instances[0].instance_template.

NFS

Represents an NFS volume.

JSON representation 
{
  "server": string,
  "remotePath": string
}
Fields
server

string

The IP address of the NFS.
remotePath

string

Remote source path exported from the NFS, e.g., "/share".

PD

Deprecated: please use deviceName instead.

JSON representation 
{
  "disk": string,
  "device": string,
  "existing": boolean
}
Fields
disk

string

PD disk name, e.g. pd-1.
device

string

PD device name, e.g. persistent-disk-1.
existing
(deprecated)

boolean

Whether this is an existing PD. Default is false. If false, i.e., new PD, we will format it into ext4 and mount to the given path. If true, i.e., existing PD, it should be in ext4 format and we will mount it to the given path.

GCS

Represents a Google Cloud Storage volume.

JSON representation 
{
  "remotePath": string
}
Fields
remotePath

string

Remote path, either a bucket name or a subdirectory of a bucket, e.g.: bucket_name, bucket_name/subdirectory/

SchedulingPolicy

How Tasks in the TaskGroup should be scheduled relative to each other.

Enums
SCHEDULING_POLICY_UNSPECIFIED Unspecified.
AS_SOON_AS_POSSIBLE

Run Tasks as soon as resources are available.

Tasks might be executed in parallel depending on parallelism and taskCount values.
IN_ORDER Run Tasks sequentially with increased task index.

AllocationPolicy

A Job's resource allocation policy describes when, where, and how compute resources should be allocated for the Job.

JSON representation 
{
  "location": {
    object (LocationPolicy)
  },
  "instance": {
    object (InstancePolicy)
  },
  "instances": [
    {
      object (InstancePolicyOrTemplate)
    }
  ],
  "instanceTemplates": [
    string
  ],
  "provisioningModels": [
    enum (ProvisioningModel)
  ],
  "serviceAccountEmail": string,
  "serviceAccount": {
    object (ServiceAccount)
  },
  "labels": {
    string: string,
    ...
  },
  "network": {
    object (NetworkPolicy)
  },
  "placement": {
    object (PlacementPolicy)
  }
}
Fields
location

object (LocationPolicy)

Location where compute resources should be allocated for the Job.
instance
(deprecated)

object (InstancePolicy)

Deprecated: please use instances[0].policy instead.
instances[]

object (InstancePolicyOrTemplate)

Describe instances that can be created by this AllocationPolicy. Only instances[0] is supported now.
instanceTemplates[]
(deprecated)

string

Deprecated: please use instances[0].template instead.
provisioningModels[]
(deprecated)

enum (ProvisioningModel)

Deprecated: please use instances[0].policy.provisioning_model instead.
serviceAccountEmail
(deprecated)

string

Deprecated: please use serviceAccount instead.
serviceAccount

object (ServiceAccount)

Service account that VMs will run as.
labels

map (key: string, value: string)

Labels applied to all VM instances and other resources created by AllocationPolicy. Labels could be user provided or system generated. You can assign up to 64 labels. Google Compute Engine label restrictions apply. Label names that start with "goog-" or "google-" are reserved.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
network

object (NetworkPolicy)

The network policy.
placement

object (PlacementPolicy)

The placement policy.

LocationPolicy

JSON representation 
{
  "allowedLocations": [
    string
  ],
  "deniedLocations": [
    string
  ]
}
Fields
allowedLocations[]

string

A list of allowed location names represented by internal URLs.

Each location can be a region or a zone. Only one region or multiple zones in one region is supported now. For example, ["regions/us-central1"] allow VMs in any zones in region us-central1. ["zones/us-central1-a", "zones/us-central1-c"] only allow VMs in zones us-central1-a and us-central1-c.

All locations end up in different regions would cause errors. For example, ["regions/us-central1", "zones/us-central1-a", "zones/us-central1-b", "zones/us-west1-a"] contains 2 regions "us-central1" and "us-west1". An error is expected in this case.
deniedLocations[]

string

A list of denied location names.

Not yet implemented.

InstancePolicy

InstancePolicy describes an instance type and resources attached to each VM created by this InstancePolicy.

JSON representation 
{
  "allowedMachineTypes": [
    string
  ],
  "machineType": string,
  "minCpuPlatform": string,
  "provisioningModel": enum (ProvisioningModel),
  "accelerators": [
    {
      object (Accelerator)
    }
  ],
  "bootDisk": {
    object (Disk)
  },
  "disks": [
    {
      object (AttachedDisk)
    }
  ],
  "reservation": string
}
Fields
allowedMachineTypes[]
(deprecated)

string

Deprecated: please use machineType instead.
machineType

string

The Compute Engine machine type.
minCpuPlatform

string

The minimum CPU platform. See https://cloud.google.com/compute/docs/instances/specify-min-cpu-platform.
provisioningModel

enum (ProvisioningModel)

The provisioning model.
accelerators[]

object (Accelerator)

The accelerators attached to each VM instance.
bootDisk

object (Disk)

Boot disk to be created and attached to each VM by this InstancePolicy. Boot disk will be deleted when the VM is deleted. Batch API now only supports booting from image.
disks[]

object (AttachedDisk)

Non-boot disks to be attached for each VM created by this InstancePolicy. New disks will be deleted when the VM is deleted. A non-boot disk is a disk that can be of a device with a file system or a raw storage drive that is not ready for data storage and accessing.
reservation

string

Optional. If specified, VMs will consume only the specified reservation. If not specified (default), VMs will consume any applicable reservation.

ProvisioningModel

Compute Engine VM instance provisioning model.

Enums
PROVISIONING_MODEL_UNSPECIFIED Unspecified.
STANDARD Standard VM.
SPOT SPOT VM.
PREEMPTIBLE

Preemptible VM (PVM).

Above SPOT VM is the preferable model for preemptible VM instances: the old preemptible VM model (indicated by this field) is the older model, and has been migrated to use the SPOT model as the underlying technology. This old model will still be supported.

Accelerator

Accelerator describes Compute Engine accelerators to be attached to the VM.

JSON representation 
{
  "type": string,
  "count": string,
  "installGpuDrivers": boolean,
  "driverVersion": string
}
Fields
type

string

The accelerator type. For example, "nvidia-tesla-t4". See gcloud compute accelerator-types list.
count

string (int64 format)

The number of accelerators of this type.
installGpuDrivers
(deprecated)

boolean

Deprecated: please use instances[0].install_gpu_drivers instead.
driverVersion

string

Optional. The NVIDIA GPU driver version that should be installed for this type.

You can define the specific driver version such as "470.103.01", following the driver version requirements in https://cloud.google.com/compute/docs/gpus/install-drivers-gpu#minimum-driver. Batch will install the specific accelerator driver if qualified.

Disk

A new persistent disk or a local ssd. A VM can only have one local SSD setting but multiple local SSD partitions. See https://cloud.google.com/compute/docs/disks#pdspecs and https://cloud.google.com/compute/docs/disks#localssds.

JSON representation 
{
  "type": string,
  "sizeGb": string,
  "diskInterface": string,

  // Union field data_source can be only one of the following:
  "image": string,
  "snapshot": string
  // End of list of possible types for union field data_source.
}
Fields
type

string

Disk type as shown in gcloud compute disk-types list. For example, local SSD uses type "local-ssd". Persistent disks and boot disks use "pd-balanced", "pd-extreme", "pd-ssd" or "pd-standard".
sizeGb

string (int64 format)

Disk size in GB.

Non-Boot Disk: If the type specifies a persistent disk, this field is ignored if data_source is set as image or snapshot. If the type specifies a local SSD, this field should be a multiple of 375 GB, otherwise, the final size will be the next greater multiple of 375 GB.

Boot Disk: Batch will calculate the boot disk size based on source image and task requirements if you do not speicify the size. If both this field and the bootDiskMib field in task spec's computeResource are defined, Batch will only honor this field. Also, this field should be no smaller than the source disk's size when the data_source is set as snapshot or image. For example, if you set an image as the data_source field and the image's default disk size 30 GB, you can only use this field to make the disk larger or equal to 30 GB.
diskInterface

string

Local SSDs are available through both "SCSI" and "NVMe" interfaces. If not indicated, "NVMe" will be the default one for local ssds. This field is ignored for persistent disks as the interface is chosen automatically. See https://cloud.google.com/compute/docs/disks/persistent-disks#choose_an_interface.
Union field data_source. A data source from which a PD will be created. data_source can be only one of the following:
image

string

URL for a VM image to use as the data source for this disk. For example, the following are all valid URLs:

  • Specify the image by its family name: projects/{project}/global/images/family/{image_family}
  • Specify the image version: projects/{project}/global/images/{imageVersion}

You can also use Batch customized image in short names. The following image values are supported for a boot disk:

  • batch-debian: use Batch Debian images.
  • batch-centos: use Batch CentOS images.
  • batch-cos: use Batch Container-Optimized images.
  • batch-hpc-centos: use Batch HPC CentOS images.
  • batch-hpc-rocky: use Batch HPC Rocky Linux images.
snapshot

string

Name of a snapshot used as the data source. Snapshot is not supported as boot disk now.

AttachedDisk

A new or an existing persistent disk (PD) or a local ssd attached to a VM instance.

JSON representation 
{
  "deviceName": string,

  // Union field attached can be only one of the following:
  "newDisk": {
    object (Disk)
  },
  "existingDisk": string
  // End of list of possible types for union field attached.
}
Fields
deviceName

string

Device name that the guest operating system will see. It is used by Runnable.volumes field to mount disks. So please specify the deviceName if you want Batch to help mount the disk, and it should match the deviceName field in volumes.

Union field attached.

attached can be only one of the following:
newDisk

object (Disk)
existingDisk

string

Name of an existing PD.

InstancePolicyOrTemplate

InstancePolicyOrTemplate lets you define the type of resources to use for this job either with an InstancePolicy or an instance template. If undefined, Batch picks the type of VM to use and doesn't include optional VM resources such as GPUs and extra disks.

JSON representation 
{
  "installGpuDrivers": boolean,

  // Union field policy_template can be only one of the following:
  "policy": {
    object (InstancePolicy)
  },
  "instanceTemplate": string
  // End of list of possible types for union field policy_template.
}
Fields
installGpuDrivers

boolean

Set this field true if users want Batch to help fetch drivers from a third party location and install them for GPUs specified in policy.accelerators or instanceTemplate on their behalf. Default is false.

For Container-Optimized Image cases, Batch will install the accelerator driver following milestones of https://cloud.google.com/container-optimized-os/docs/release-notes. For non Container-Optimized Image cases, following https://github.com/GoogleCloudPlatform/compute-gpu-installation/blob/main/linux/install_gpu_driver.py.

Union field policy_template.

policy_template can be only one of the following:
policy

object (InstancePolicy)

InstancePolicy.
instanceTemplate

string

Name of an instance template used to create VMs. Named the field as 'instanceTemplate' instead of 'template' to avoid c++ keyword conflict.

ServiceAccount

Carries information about a Google Cloud service account.

JSON representation 
{
  "email": string,
  "scopes": [
    string
  ]
}
Fields
email

string

Email address of the service account. If not specified, the default Compute Engine service account for the project will be used. If instance template is being used, the service account has to be specified in the instance template and it has to match the email field here.
scopes[]

string

List of scopes to be enabled for this service account on the VM, in addition to the cloud-platform API scope that will be added by default.

NetworkPolicy

NetworkPolicy describes VM instance network configurations.

JSON representation 
{
  "networkInterfaces": [
    {
      object (NetworkInterface)
    }
  ]
}
Fields
networkInterfaces[]

object (NetworkInterface)

Network configurations.

NetworkInterface

A network interface.

JSON representation 
{
  "network": string,
  "subnetwork": string,
  "noExternalIpAddress": boolean
}
Fields
network

string

The URL of an existing network resource. You can specify the network as a full or partial URL.

For example, the following are all valid URLs:
subnetwork

string

The URL of an existing subnetwork resource in the network. You can specify the subnetwork as a full or partial URL.

For example, the following are all valid URLs:
noExternalIpAddress

boolean

Default is false (with an external IP address). Required if no external public IP address is attached to the VM. If no external public IP address, additional configuration is required to allow the VM to access Google Services. See https://cloud.google.com/vpc/docs/configure-private-google-access and https://cloud.google.com/nat/docs/gce-example#create-nat for more information.

PlacementPolicy

PlacementPolicy describes a group placement policy for the VMs controlled by this AllocationPolicy.

JSON representation 
{
  "collocation": string,
  "maxDistance": string
}
Fields
collocation

string

UNSPECIFIED vs. COLLOCATED (default UNSPECIFIED). Use COLLOCATED when you want VMs to be located close to each other for low network latency between the VMs. No placement policy will be generated when collocation is UNSPECIFIED.
maxDistance

string (int64 format)

When specified, causes the job to fail if more than maxDistance logical switches are required between VMs. Batch uses the most compact possible placement of VMs even when maxDistance is not specified. An explicit maxDistance makes that level of compactness a strict requirement. Not yet implemented

SchedulingPolicy

The order that TaskGroups are scheduled relative to each other.

Not yet implemented.

Enums
SCHEDULING_POLICY_UNSPECIFIED Unspecified.
AS_SOON_AS_POSSIBLE Run all TaskGroups as soon as possible.

JobDependency

JobDependency describes the state of other Jobs that the start of this Job depends on. All dependent Jobs must have been submitted in the same region.

JSON representation 
{
  "items": {
    string: enum (Type),
    ...
  }
}
Fields
items

map (key: string, value: enum (Type))

Each item maps a Job name to a Type. All items must be satisfied for the JobDependency to be satisfied (the AND operation). Once a condition for one item becomes true, it won't go back to false even the dependent Job state changes again.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

Type

Dependency type.

Enums
TYPE_UNSPECIFIED Unspecified.
SUCCEEDED The dependent Job has succeeded.
FAILED The dependent Job has failed.
FINISHED SUCCEEDED or FAILED.

JobStatus

Job status.

JSON representation 
{
  "state": enum (State),
  "statusEvents": [
    {
      object (StatusEvent)
    }
  ],
  "taskGroups": {
    string: {
      object (TaskGroupStatus)
    },
    ...
  },
  "runDuration": string,
  "resourceUsage": {
    object (ResourceUsage)
  }
}
Fields
state

enum (State)

Job state
statusEvents[]

object (StatusEvent)

Job status events
taskGroups

map (key: string, value: object (TaskGroupStatus))

Aggregated task status for each TaskGroup in the Job. The map key is TaskGroup ID.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
runDuration

string (Duration format)

The duration of time that the Job spent in status RUNNING.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
resourceUsage

object (ResourceUsage)

The resource usage of the job.

State

Valid Job states.

Enums
STATE_UNSPECIFIED Job state unspecified.
QUEUED Job is admitted (validated and persisted) and waiting for resources.
SCHEDULED Job is scheduled to run as soon as resource allocation is ready. The resource allocation may happen at a later time but with a high chance to succeed.
RUNNING Resource allocation has been successful. At least one Task in the Job is RUNNING.
SUCCEEDED All Tasks in the Job have finished successfully.
FAILED At least one Task in the Job has failed.
DELETION_IN_PROGRESS The Job will be deleted, but has not been deleted yet. Typically this is because resources used by the Job are still being cleaned up.

TaskGroupStatus

Aggregated task status for a TaskGroup.

JSON representation 
{
  "counts": {
    string: string,
    ...
  },
  "instances": [
    {
      object (InstanceStatus)
    }
  ]
}
Fields
counts

map (key: string, value: string (int64 format))

Count of task in each state in the TaskGroup. The map key is task state name.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
instances[]

object (InstanceStatus)

Status of instances allocated for the TaskGroup.

InstanceStatus

VM instance status.

JSON representation 
{
  "machineType": string,
  "provisioningModel": enum (ProvisioningModel),
  "taskPack": string,
  "bootDisk": {
    object (Disk)
  }
}
Fields
machineType

string

The Compute Engine machine type.
provisioningModel

enum (ProvisioningModel)

The VM instance provisioning model.
taskPack

string (int64 format)

The max number of tasks can be assigned to this instance type.
bootDisk

object (Disk)

The VM boot disk.

ResourceUsage

ResourceUsage describes the resource usage of the job.

JSON representation 
{
  "coreHours": number
}
Fields
coreHours

number

The CPU core hours that the job consumes.

JobNotification

Notification configurations.

JSON representation 
{
  "pubsubTopic": string,
  "message": {
    object (Message)
  }
}
Fields
pubsubTopic

string

The Pub/Sub topic where notifications like the job state changes will be published. The topic must exist in the same project as the job and billings will be charged to this project. If not specified, no Pub/Sub messages will be sent. Topic format: projects/{project}/topics/{topic}.
message

object (Message)

The attribute requirements of messages to be sent to this Pub/Sub topic. Without this field, no message will be sent.

Message

Message details. Describe the conditions under which messages will be sent. If no attribute is defined, no message will be sent by default. One message should specify either the job or the task level attributes, but not both. For example, job level: JOB_STATE_CHANGED and/or a specified newJobState; task level: TASK_STATE_CHANGED and/or a specified newTaskState.

JSON representation 
{
  "type": enum (Type),
  "newJobState": enum (State),
  "newTaskState": enum (State)
}
Fields
type

enum (Type)

The message type.
newJobState

enum (State)

The new job state.
newTaskState

enum (State)

The new task state.

Type

The message type.

Enums
TYPE_UNSPECIFIED Unspecified.
JOB_STATE_CHANGED Notify users that the job state has changed.
TASK_STATE_CHANGED Notify users that the task state has changed.

LogsPolicy

LogsPolicy describes how outputs from a Job's Tasks (stdout/stderr) will be preserved.

JSON representation 
{
  "destination": enum (Destination),
  "logsPath": string
}
Fields
destination

enum (Destination)

Where logs should be saved.
logsPath

string

The path to which logs are saved when the destination = PATH. This can be a local file path on the VM, or under the mount point of a Persistent Disk or Filestore, or a Cloud Storage path.

Destination

The destination (if any) for logs.

Enums
DESTINATION_UNSPECIFIED Logs are not preserved.
CLOUD_LOGGING Logs are streamed to Cloud Logging.
PATH Logs are saved to a file path.

Methods

create

 Create a Job.

delete

 Delete a Job.

get

 Get a Job specified by its resource name.

list

 List all Jobs for a project within a region.