REST Resource: projects.locations.lakes.tasks

Stay organized with collections Save and categorize content based on your preferences.

Resource: Task

A task represents a user-visible job.

JSON representation
{
  "name": string,
  "uid": string,
  "createTime": string,
  "updateTime": string,
  "description": string,
  "displayName": string,
  "state": enum (State),
  "labels": {
    string: string,
    ...
  },
  "triggerSpec": {
    object (TriggerSpec)
  },
  "executionSpec": {
    object (ExecutionSpec)
  },
  "executionStatus": {
    object (ExecutionStatus)
  },
  "spark": {
    object (SparkTaskConfig)
  }
}
Fields
name

string

Output only. The relative resource name of the task, of the form: projects/{project_number}/locations/{locationId}/lakes/{lakeId}/ tasks/{taskId}.

uid

string

Output only. System generated globally unique ID for the task. This ID will be different if the task is deleted and re-created with the same name.

createTime

string (Timestamp format)

Output only. The time when the task was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

updateTime

string (Timestamp format)

Output only. The time when the task was last updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

description

string

Optional. Description of the task.

displayName

string

Optional. User friendly display name.

state

enum (State)

Output only. Current state of the task.

labels

map (key: string, value: string)

Optional. User-defined labels for the task.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

triggerSpec

object (TriggerSpec)

Required. Spec related to how often and when a task should be triggered.

executionSpec

object (ExecutionSpec)

Required. Spec related to how a task is executed.

executionStatus

object (ExecutionStatus)

Output only. Status of the latest task executions.

spark

object (SparkTaskConfig)

Config related to running custom Spark tasks.

TriggerSpec

Task scheduling and trigger settings.

JSON representation
{
  "type": enum (Type),
  "startTime": string,
  "disabled": boolean,
  "maxRetries": integer,
  "schedule": string
}
Fields
type

enum (Type)

Required. Immutable. Trigger type of the user-specified Task.

startTime

string (Timestamp format)

Optional. The first run of the task will be after this time. If not specified, the task will run shortly after being submitted if ON_DEMAND and based on the schedule if RECURRING.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

disabled

boolean

Optional. Prevent the task from executing. This does not cancel already running tasks. It is intended to temporarily disable RECURRING tasks.

maxRetries

integer

Optional. Number of retry attempts before aborting. Set to zero to never attempt to retry a failed task.

schedule

string

Optional. Cron schedule (https://en.wikipedia.org/wiki/Cron) for running tasks periodically. To explicitly set a timezone to the cron tab, apply a prefix in the cron tab: "CRON_TZ=${IANA_TIME_ZONE}" or "TZ=${IANA_TIME_ZONE}". The ${IANA_TIME_ZONE} may only be a valid string from IANA time zone database. For example, "CRON_TZ=America/New_York 1 * * * *", or "TZ=America/New_York 1 * * * *". This field is required for RECURRING tasks.

Type

Determines how often and when the job will run.

Enums
TYPE_UNSPECIFIED Unspecified trigger type.
ON_DEMAND The task runs one-time shortly after Task Creation.
RECURRING The task is scheduled to run periodically.

ExecutionSpec

Execution related settings, like retry and serviceAccount.

JSON representation
{
  "args": {
    string: string,
    ...
  },
  "serviceAccount": string,
  "project": string,
  "maxJobExecutionLifetime": string,
  "kmsKey": string
}
Fields
args

map (key: string, value: string)

Optional. The arguments to pass to the task. The args can use placeholders of the format ${placeholder} as part of key/value string. These will be interpolated before passing the args to the driver. Currently supported placeholders: - ${taskId} - ${job_time} To pass positional args, set the key as TASK_ARGS. The value should be a comma-separated string of all the positional arguments. To use a delimiter other than comma, refer to https://cloud.google.com/sdk/gcloud/reference/topic/escaping. In case of other keys being present in the args, then TASK_ARGS will be passed as the last argument.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

serviceAccount

string

Required. Service account to use to execute a task. If not provided, the default Compute service account for the project is used.

project

string

Optional. The project in which jobs are run. By default, the project containing the Lake is used. If a project is provided, the ExecutionSpec.service_account must belong to this project.

maxJobExecutionLifetime

string (Duration format)

Optional. The maximum duration after which the job execution is expired.

A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

kmsKey

string

Optional. The Cloud KMS key to use for encryption, of the form: projects/{project_number}/locations/{locationId}/keyRings/{key-ring-name}/cryptoKeys/{key-name}.

ExecutionStatus

Status of the task execution (e.g. Jobs).

JSON representation
{
  "updateTime": string,
  "latestJob": {
    object (Job)
  }
}
Fields
updateTime

string (Timestamp format)

Output only. Last update time of the status.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

latestJob

object (Job)

Output only. latest job execution

SparkTaskConfig

User-specified config for running a Spark task.

JSON representation
{
  "fileUris": [
    string
  ],
  "archiveUris": [
    string
  ],
  "infrastructureSpec": {
    object (InfrastructureSpec)
  },

  // Union field driver can be only one of the following:
  "mainJarFileUri": string,
  "mainClass": string,
  "pythonScriptFile": string,
  "sqlScriptFile": string,
  "sqlScript": string
  // End of list of possible types for union field driver.
}
Fields
fileUris[]

string

Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.

archiveUris[]

string

Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.

infrastructureSpec

object (InfrastructureSpec)

Optional. Infrastructure specification for the execution.

Union field driver. Required. The specification of the main method to call to drive the job. Specify either the jar file that contains the main class or the main class name. driver can be only one of the following:
mainJarFileUri

string

The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments (--key=value).

mainClass

string

The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris. The execution args are passed in as a sequence of named process arguments (--key=value).

pythonScriptFile

string

The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments (--key=value).

sqlScriptFile

string

A reference to a query file. This can be the Cloud Storage URI of the query file or it can the path to a SqlScript Content. The execution args are used to declare a set of script variables (set key="value";).

sqlScript

string

The query text. The execution args are used to declare a set of script variables (set key="value";).

InfrastructureSpec

Configuration for the underlying infrastructure used to run workloads.

JSON representation
{
  "batch": {
    object (BatchComputeResources)
  },
  "containerImage": {
    object (ContainerImageRuntime)
  },
  "vpcNetwork": {
    object (VpcNetwork)
  }
}
Fields
batch

object (BatchComputeResources)

Compute resources needed for a Task when using Dataproc Serverless.

containerImage

object (ContainerImageRuntime)

Container Image Runtime Configuration.

vpcNetwork

object (VpcNetwork)

Vpc network.

BatchComputeResources

Batch compute resources associated with the task.

JSON representation
{
  "executorsCount": integer,
  "maxExecutorsCount": integer
}
Fields
executorsCount

integer

Optional. Total number of job executors. Executor Count should be between 2 and 100. [Default=2]

maxExecutorsCount

integer

Optional. Max configurable executors. If maxExecutorsCount > executorsCount, then auto-scaling is enabled. Max Executor Count should be between 2 and 1000. [Default=1000]

ContainerImageRuntime

Container Image Runtime Configuration used with Batch execution.

JSON representation
{
  "image": string,
  "javaJars": [
    string
  ],
  "pythonPackages": [
    string
  ],
  "properties": {
    string: string,
    ...
  }
}
Fields
image

string

Optional. Container image to use.

javaJars[]

string

Optional. A list of Java JARS to add to the classpath. Valid input includes Cloud Storage URIs to Jar binaries. For example, gs://bucket-name/my/path/to/file.jar

pythonPackages[]

string

Optional. A list of python packages to be installed. Valid formats include Cloud Storage URI to a PIP installable library. For example, gs://bucket-name/my/path/to/lib.tar.gz

properties

map (key: string, value: string)

Optional. Override to common configuration of open source components installed on the Dataproc cluster. The properties to set on daemon config files. Property keys are specified in prefix:property format, for example core:hadoop.tmp.dir. For more information, see Cluster properties.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

VpcNetwork

Cloud VPC Network used to run the infrastructure.

JSON representation
{
  "networkTags": [
    string
  ],

  // Union field network_name can be only one of the following:
  "network": string,
  "subNetwork": string
  // End of list of possible types for union field network_name.
}
Fields
networkTags[]

string

Optional. List of network tags to apply to the job.

Union field network_name. The Cloud VPC network identifier. network_name can be only one of the following:
network

string

Optional. The Cloud VPC network in which the job is run. By default, the Cloud VPC network named Default within the project is used.

subNetwork

string

Optional. The Cloud VPC sub-network in which the job is run.

Methods

create

Creates a task resource within a lake.

delete

Delete the task resource.

get

Get task resource.

getIamPolicy

Gets the access control policy for a resource.

list

Lists tasks under the given lake.

patch

Update the task resource.

setIamPolicy

Sets the access control policy on the specified resource.

testIamPermissions

Returns permissions that a caller has on the specified resource.