Defines a particular step within a Cloud Dataflow job.
A job consists of multiple steps, each of which performs some
specific operation as part of the overall job. Data is typically
passed from one step to another as part of the job.
Here's an example of a sequence of steps which together implement a
Map-Reduce job:
Read a collection of data from some source, parsing the
collection's elements.
Validate the elements.
Apply a user-defined function to map each element to some value
and extract an element-specific key value.
Group elements with the same key into a single element with that
key, transforming a multiply-keyed collection into a
uniquely-keyed collection.
Write the elements out to some data sink.
Note that the Cloud Dataflow service may be used to run many
different types of jobs, not just Map-Reduce.
Attributes
Name
Description
kind
str
The kind of step in the Cloud Dataflow job.
name
str
The name that identifies the step. This must
be unique for each step with respect to all
other steps in the Cloud Dataflow job.
properties
google.protobuf.struct_pb2.Struct
Named properties associated with the step. Each kind of
predefined step has its own required set of properties. Must
be provided on Create. Only retrieved with JOB_VIEW_ALL.
