REST Resource: projects.locations.pipelineJobs

Resource: PipelineJob

An instance of a machine learning PipelineJob.

JSON representation 
{
  "name": string,
  "displayName": string,
  "createTime": string,
  "startTime": string,
  "endTime": string,
  "updateTime": string,
  "pipelineSpec": {
    object
  },
  "state": enum (PipelineState),
  "jobDetail": {
    object (PipelineJobDetail)
  },
  "error": {
    object (Status)
  },
  "labels": {
    string: string,
    ...
  },
  "runtimeConfig": {
    object (RuntimeConfig)
  },
  "encryptionSpec": {
    object (EncryptionSpec)
  },
  "serviceAccount": string,
  "network": string
}
Fields
name

string

Output only. The resource name of the PipelineJob.
displayName

string

The display name of the Pipeline. The name can be up to 128 characters long and can be consist of any UTF-8 characters.
createTime

string (Timestamp format)

Output only. Pipeline creation time.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
startTime

string (Timestamp format)

Output only. Pipeline start time.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

Output only. Pipeline end time.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. Timestamp when this PipelineJob was most recently updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
pipelineSpec

object (Struct format)

Required. The spec of the pipeline.
state

enum (PipelineState)

Output only. The detailed state of the job.
jobDetail

object (PipelineJobDetail)

Output only. The details of pipeline run. Not available in the list view.
error

object (Status)

Output only. The error that occurred during pipeline execution. Only populated when the pipeline's state is FAILED or CANCELLED.
labels

map (key: string, value: string)

The labels with user-defined metadata to organize PipelineJob.

Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed.

See https://goo.gl/xmQnxf for more information and examples of labels.
runtimeConfig

object (RuntimeConfig)

Runtime config of the pipeline.
encryptionSpec

object (EncryptionSpec)

Customer-managed encryption key spec for a pipelineJob. If set, this PipelineJob and all of its sub-resources will be secured by this key.
serviceAccount

string

The service account that the pipeline workload runs as. If not specified, the Compute Engine default service account in the project will be used. See https://cloud.google.com/compute/docs/access/service-accounts#default_service_account

Users starting the pipeline must have the iam.serviceAccounts.actAs permission on this service account.
network

string

The full name of the Compute Engine network to which the Pipeline Job's workload should be peered. For example, projects/12345/global/networks/myVPC. Format is of the form projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. Pipeline job will apply the network configuration to the GCP resources being launched, if applied, such as Vertex AI Training or Dataflow job. If left unspecified, the workload is not peered with any network.

PipelineJobDetail

The runtime detail of PipelineJob.

JSON representation 
{
  "pipelineContext": {
    object (Context)
  },
  "pipelineRunContext": {
    object (Context)
  },
  "taskDetails": [
    {
      object (PipelineTaskDetail)
    }
  ]
}
Fields
pipelineContext

object (Context)

Output only. The context of the pipeline.
pipelineRunContext

object (Context)

Output only. The context of the current pipeline run.
taskDetails[]

object (PipelineTaskDetail)

Output only. The runtime details of the tasks under the pipeline.

Context

Instance of a general context.

JSON representation 
{
  "name": string,
  "displayName": string,
  "etag": string,
  "labels": {
    string: string,
    ...
  },
  "createTime": string,
  "updateTime": string,
  "parentContexts": [
    string
  ],
  "schemaTitle": string,
  "schemaVersion": string,
  "metadata": {
    object
  },
  "description": string
}
Fields
name

string

Output only. The resource name of the Context.
displayName

string

User provided display name of the Context. May be up to 128 Unicode characters.
etag

string

An eTag used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
labels

map (key: string, value: string)

The labels with user-defined metadata to organize your Contexts.

Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Context (System labels are excluded).
createTime

string (Timestamp format)

Output only. Timestamp when this Context was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. Timestamp when this Context was last updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
parentContexts[]

string

Output only. A list of resource names of Contexts that are parents of this Context. A Context may have at most 10 parentContexts.
schemaTitle

string

The title of the schema describing the metadata.

Schema title and version is expected to be registered in earlier Create Schema calls. And both are used together as unique identifiers to identify schemas within the local metadata store.
schemaVersion

string

The version of the schema in schema_name to use.

Schema title and version is expected to be registered in earlier Create Schema calls. And both are used together as unique identifiers to identify schemas within the local metadata store.
metadata

object (Struct format)

Properties of the Context. The size of this field should not exceed 200KB.
description

string

Description of the Context

PipelineTaskDetail

The runtime detail of a task execution.

JSON representation 
{
  "taskId": string,
  "parentTaskId": string,
  "taskName": string,
  "createTime": string,
  "startTime": string,
  "endTime": string,
  "executorDetail": {
    object (PipelineTaskExecutorDetail)
  },
  "state": enum (State),
  "execution": {
    object (Execution)
  },
  "error": {
    object (Status)
  },
  "inputs": {
    string: {
      object (ArtifactList)
    },
    ...
  },
  "outputs": {
    string: {
      object (ArtifactList)
    },
    ...
  }
}
Fields
taskId

string (int64 format)

Output only. The system generated ID of the task.
parentTaskId

string (int64 format)

Output only. The id of the parent task if the task is within a component scope. Empty if the task is at the root level.
taskName

string

Output only. The user specified name of the task that is defined in [PipelineJob.spec][].
createTime

string (Timestamp format)

Output only. Task create time.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
startTime

string (Timestamp format)

Output only. Task start time.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

Output only. Task end time.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
executorDetail

object (PipelineTaskExecutorDetail)

Output only. The detailed execution info.
state

enum (State)

Output only. State of the task.
execution

object (Execution)

Output only. The execution metadata of the task.
error

object (Status)

Output only. The error that occurred during task execution. Only populated when the task's state is FAILED or CANCELLED.
inputs

map (key: string, value: object (ArtifactList))

Output only. The runtime input artifacts of the task.
outputs

map (key: string, value: object (ArtifactList))

Output only. The runtime output artifacts of the task.

PipelineTaskExecutorDetail

The runtime detail of a pipeline executor.

JSON representation 
{

  // Union field details can be only one of the following:
  "containerDetail": {
    object (ContainerDetail)
  },
  "customJobDetail": {
    object (CustomJobDetail)
  }
  // End of list of possible types for union field details.
}
Fields

Union field details.

details can be only one of the following:
containerDetail

object (ContainerDetail)

Output only. The detailed info for a container executor.
customJobDetail
(deprecated)

object (CustomJobDetail)

Output only. The detailed info for a custom job executor.

ContainerDetail

The detail of a container execution. It contains the job names of the lifecycle of a container execution.

JSON representation 
{
  "mainJob": string,
  "preCachingCheckJob": string
}
Fields
mainJob

string

Output only. The name of the CustomJob for the main container execution.
preCachingCheckJob

string

Output only. The name of the CustomJob for the pre-caching-check container execution. This job will be available if the PipelineJob.pipeline_spec specifies the pre_caching_check hook in the lifecycle events.

CustomJobDetail

The detailed info for a custom job executor.

JSON representation 
{
  "job": string
}
Fields
job

string

Output only. The name of the CustomJob.

State

Specifies state of TaskExecution

Enums
STATE_UNSPECIFIED Unspecified.
PENDING Specifies pending state for the task.
RUNNING Specifies task is being executed.
SUCCEEDED Specifies task completed successfully.
CANCEL_PENDING Specifies Task cancel is in pending state.
CANCELLING Specifies task is being cancelled.
CANCELLED Specifies task was cancelled.
FAILED Specifies task failed.
SKIPPED Specifies task was skipped due to cache hit.
NOT_TRIGGERED Specifies that the task was not triggered because the task's trigger policy is not satisfied. The trigger policy is specified in the condition field of PipelineJob.pipeline_spec.

Execution

Instance of a general execution.

JSON representation 
{
  "name": string,
  "displayName": string,
  "state": enum (State),
  "etag": string,
  "labels": {
    string: string,
    ...
  },
  "createTime": string,
  "updateTime": string,
  "schemaTitle": string,
  "schemaVersion": string,
  "metadata": {
    object
  },
  "description": string
}
Fields
name

string

Output only. The resource name of the Execution.
displayName

string

User provided display name of the Execution. May be up to 128 Unicode characters.
state

enum (State)

The state of this Execution. This is a property of the Execution, and does not imply or capture any ongoing process. This property is managed by clients (such as Vertex Pipelines) and the system does not prescribe or check the validity of state transitions.
etag

string

An eTag used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
labels

map (key: string, value: string)

The labels with user-defined metadata to organize your Executions.

Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Execution (System labels are excluded).
createTime

string (Timestamp format)

Output only. Timestamp when this Execution was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. Timestamp when this Execution was last updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
schemaTitle

string

The title of the schema describing the metadata.

Schema title and version is expected to be registered in earlier Create Schema calls. And both are used together as unique identifiers to identify schemas within the local metadata store.
schemaVersion

string

The version of the schema in schemaTitle to use.

Schema title and version is expected to be registered in earlier Create Schema calls. And both are used together as unique identifiers to identify schemas within the local metadata store.
metadata

object (Struct format)

Properties of the Execution. The size of this field should not exceed 200KB.
description

string

Description of the Execution

State

Describes the state of the Execution.

Enums
STATE_UNSPECIFIED Unspecified Execution state
NEW The Execution is new
RUNNING The Execution is running
COMPLETE The Execution has finished running
FAILED The Execution has failed
CACHED The Execution completed through Cache hit.
CANCELLED The Execution was cancelled.

ArtifactList

A list of artifact metadata.

JSON representation 
{
  "artifacts": [
    {
      object (Artifact)
    }
  ]
}
Fields
artifacts[]

object (Artifact)

Output only. A list of artifact metadata.

Artifact

Instance of a general artifact.

JSON representation 
{
  "name": string,
  "displayName": string,
  "uri": string,
  "etag": string,
  "labels": {
    string: string,
    ...
  },
  "createTime": string,
  "updateTime": string,
  "state": enum (State),
  "schemaTitle": string,
  "schemaVersion": string,
  "metadata": {
    object
  },
  "description": string
}
Fields
name

string

Output only. The resource name of the Artifact.
displayName

string

User provided display name of the Artifact. May be up to 128 Unicode characters.
uri

string

The uniform resource identifier of the artifact file. May be empty if there is no actual artifact file.
etag

string

An eTag used to perform consistent read-modify-write updates. If not set, a blind "overwrite" update happens.
labels

map (key: string, value: string)

The labels with user-defined metadata to organize your Artifacts.

Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Artifact (System labels are excluded).
createTime

string (Timestamp format)

Output only. Timestamp when this Artifact was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. Timestamp when this Artifact was last updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
state

enum (State)

The state of this Artifact. This is a property of the Artifact, and does not imply or capture any ongoing process. This property is managed by clients (such as Vertex Pipelines), and the system does not prescribe or check the validity of state transitions.
schemaTitle

string

The title of the schema describing the metadata.

Schema title and version is expected to be registered in earlier Create Schema calls. And both are used together as unique identifiers to identify schemas within the local metadata store.
schemaVersion

string

The version of the schema in schema_name to use.

Schema title and version is expected to be registered in earlier Create Schema calls. And both are used together as unique identifiers to identify schemas within the local metadata store.
metadata

object (Struct format)

Properties of the Artifact. The size of this field should not exceed 200KB.
description

string

Description of the Artifact

State

Describes the state of the Artifact.

Enums
STATE_UNSPECIFIED Unspecified state for the Artifact.
PENDING A state used by systems like Vertex Pipelines to indicate that the underlying data item represented by this Artifact is being created.
LIVE A state indicating that the Artifact should exist, unless something external to the system deletes it.

RuntimeConfig

The runtime config of a PipelineJob.

JSON representation 
{
  "parameters": {
    string: {
      object (Value)
    },
    ...
  },
  "gcsOutputDirectory": string
}
Fields
parameters

map (key: string, value: object (Value))

The runtime parameters of the PipelineJob. The parameters will be passed into PipelineJob.pipeline_spec to replace the placeholders at runtime.
gcsOutputDirectory

string

Required. A path in a Cloud Storage bucket, which will be treated as the root output directory of the pipeline. It is used by the system to generate the paths of output artifacts. The artifact paths are generated with a sub-path pattern {job_id}/{taskId}/{output_key} under the specified output directory. The service account specified in this pipeline must have the storage.objects.get and storage.objects.create permissions for this bucket.

Value

Value is the value of the field.

JSON representation 
{

  // Union field value can be only one of the following:
  "intValue": string,
  "doubleValue": number,
  "stringValue": string
  // End of list of possible types for union field value.
}
Fields

Union field value.

value can be only one of the following:
intValue

string (int64 format)

An integer value.
doubleValue

number

A double value.
stringValue

string

A string value.

Methods

cancel

 Cancels a PipelineJob.

create

 Creates a PipelineJob.

delete

 Deletes a PipelineJob.

get

 Gets a PipelineJob.

list

 Lists PipelineJobs in a Location.