Class OrderedJob (5.14.0)

OrderedJob(mapping=None, *, ignore_unknown_fields=False, **kwargs)

A job executed by the workflow.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Attributes

Name Description
step_id str
Required. The step id. The id must be unique among all jobs within the template. The step id is used as prefix for job id, as job goog-dataproc-workflow-step-id label, and in prerequisiteStepIds field from other steps. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.
hadoop_job google.cloud.dataproc_v1.types.HadoopJob
Optional. Job is a Hadoop job. This field is a member of oneof_ job_type.
spark_job google.cloud.dataproc_v1.types.SparkJob
Optional. Job is a Spark job. This field is a member of oneof_ job_type.
pyspark_job google.cloud.dataproc_v1.types.PySparkJob
Optional. Job is a PySpark job. This field is a member of oneof_ job_type.
hive_job google.cloud.dataproc_v1.types.HiveJob
Optional. Job is a Hive job. This field is a member of oneof_ job_type.
pig_job google.cloud.dataproc_v1.types.PigJob
Optional. Job is a Pig job. This field is a member of oneof_ job_type.
spark_r_job google.cloud.dataproc_v1.types.SparkRJob
Optional. Job is a SparkR job. This field is a member of oneof_ job_type.
spark_sql_job google.cloud.dataproc_v1.types.SparkSqlJob
Optional. Job is a SparkSql job. This field is a member of oneof_ job_type.
presto_job google.cloud.dataproc_v1.types.PrestoJob
Optional. Job is a Presto job. This field is a member of oneof_ job_type.
trino_job google.cloud.dataproc_v1.types.TrinoJob
Optional. Job is a Trino job. This field is a member of oneof_ job_type.
flink_job google.cloud.dataproc_v1.types.FlinkJob
Optional. Job is a Flink job. This field is a member of oneof_ job_type.
labels MutableMapping[str, str]
Optional. The labels to associate with this job. Label keys must be between 1 and 63 characters long, and must conform to the following regular expression: [\p{Ll}\p{Lo}][\p{Ll}\p{Lo}\p{N}_-]{0,62} Label values must be between 1 and 63 characters long, and must conform to the following regular expression: [\p{Ll}\p{Lo}\p{N}_-]{0,63} No more than 32 labels can be associated with a given job.
scheduling google.cloud.dataproc_v1.types.JobScheduling
Optional. Job scheduling configuration.
prerequisite_step_ids MutableSequence[str]
Optional. The optional list of prerequisite job step_ids. If not specified, the job will start at the beginning of workflow.

Classes

LabelsEntry

LabelsEntry(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The abstract base class for a message.

Parameters
Name Description
kwargs dict

Keys and values corresponding to the fields of the message.

mapping Union[dict, .Message]

A dictionary or message to be used to determine the values for this message.

ignore_unknown_fields Optional(bool)

If True, do not raise errors for unknown fields. Only applied if mapping is a mapping type or there are keyword parameters.