Classes
AutoscalingSettings
Settings for WorkerPool autoscaling.
Protobuf type google.events.cloud.dataflow.v1beta3.AutoscalingSettings
AutoscalingSettings.Builder
Settings for WorkerPool autoscaling.
Protobuf type google.events.cloud.dataflow.v1beta3.AutoscalingSettings
BigQueryIODetails
Metadata for a BigQuery connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.BigQueryIODetails
BigQueryIODetails.Builder
Metadata for a BigQuery connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.BigQueryIODetails
BigTableIODetails
Metadata for a Cloud Bigtable connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.BigTableIODetails
BigTableIODetails.Builder
Metadata for a Cloud Bigtable connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.BigTableIODetails
Data
DatastoreIODetails
Metadata for a Datastore connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.DatastoreIODetails
DatastoreIODetails.Builder
Metadata for a Datastore connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.DatastoreIODetails
DebugOptions
Describes any options that have an effect on the debugging of pipelines.
Protobuf type google.events.cloud.dataflow.v1beta3.DebugOptions
DebugOptions.Builder
Describes any options that have an effect on the debugging of pipelines.
Protobuf type google.events.cloud.dataflow.v1beta3.DebugOptions
Environment
Describes the environment in which a Dataflow Job runs.
Protobuf type google.events.cloud.dataflow.v1beta3.Environment
Environment.Builder
Describes the environment in which a Dataflow Job runs.
Protobuf type google.events.cloud.dataflow.v1beta3.Environment
ExecutionStageState
A message describing the state of a particular execution stage.
Protobuf type google.events.cloud.dataflow.v1beta3.ExecutionStageState
ExecutionStageState.Builder
A message describing the state of a particular execution stage.
Protobuf type google.events.cloud.dataflow.v1beta3.ExecutionStageState
FileIODetails
Metadata for a File connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.FileIODetails
FileIODetails.Builder
Metadata for a File connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.FileIODetails
Job
Defines a job to be run by the Cloud Dataflow service. Do not enter confidential information when you supply string values using the API. Fields stripped from source Job proto:
- steps
- pipeline_description
- transform_name_mapping
Protobuf type google.events.cloud.dataflow.v1beta3.Job
Job.Builder
Defines a job to be run by the Cloud Dataflow service. Do not enter confidential information when you supply string values using the API. Fields stripped from source Job proto:
- steps
- pipeline_description
- transform_name_mapping
Protobuf type google.events.cloud.dataflow.v1beta3.Job
JobEventData
The data within all Job events.
Protobuf type google.events.cloud.dataflow.v1beta3.JobEventData
JobEventData.Builder
The data within all Job events.
Protobuf type google.events.cloud.dataflow.v1beta3.JobEventData
JobExecutionInfo
Additional information about how a Cloud Dataflow job will be executed that isn't contained in the submitted job.
Protobuf type google.events.cloud.dataflow.v1beta3.JobExecutionInfo
JobExecutionInfo.Builder
Additional information about how a Cloud Dataflow job will be executed that isn't contained in the submitted job.
Protobuf type google.events.cloud.dataflow.v1beta3.JobExecutionInfo
JobExecutionStageInfo
Contains information about how a particular google.dataflow.v1beta3.Step will be executed.
Protobuf type google.events.cloud.dataflow.v1beta3.JobExecutionStageInfo
JobExecutionStageInfo.Builder
Contains information about how a particular google.dataflow.v1beta3.Step will be executed.
Protobuf type google.events.cloud.dataflow.v1beta3.JobExecutionStageInfo
JobMetadata
Metadata available primarily for filtering jobs. Will be included in the ListJob response and Job SUMMARY view.
Protobuf type google.events.cloud.dataflow.v1beta3.JobMetadata
JobMetadata.Builder
Metadata available primarily for filtering jobs. Will be included in the ListJob response and Job SUMMARY view.
Protobuf type google.events.cloud.dataflow.v1beta3.JobMetadata
Package
The packages that must be installed in order for a worker to run the steps of the Cloud Dataflow job that will be assigned to its worker pool. This is the mechanism by which the Cloud Dataflow SDK causes code to be loaded onto the workers. For example, the Cloud Dataflow Java SDK might use this to install jars containing the user's code and all of the various dependencies (libraries, data files, etc.) required in order for that code to run.
Protobuf type google.events.cloud.dataflow.v1beta3.Package
Package.Builder
The packages that must be installed in order for a worker to run the steps of the Cloud Dataflow job that will be assigned to its worker pool. This is the mechanism by which the Cloud Dataflow SDK causes code to be loaded onto the workers. For example, the Cloud Dataflow Java SDK might use this to install jars containing the user's code and all of the various dependencies (libraries, data files, etc.) required in order for that code to run.
Protobuf type google.events.cloud.dataflow.v1beta3.Package
PubSubIODetails
Metadata for a Pub/Sub connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.PubSubIODetails
PubSubIODetails.Builder
Metadata for a Pub/Sub connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.PubSubIODetails
SdkHarnessContainerImage
Defines an SDK harness container for executing Dataflow pipelines.
Protobuf type google.events.cloud.dataflow.v1beta3.SdkHarnessContainerImage
SdkHarnessContainerImage.Builder
Defines an SDK harness container for executing Dataflow pipelines.
Protobuf type google.events.cloud.dataflow.v1beta3.SdkHarnessContainerImage
SdkVersion
The version of the SDK used to run the job.
Protobuf type google.events.cloud.dataflow.v1beta3.SdkVersion
SdkVersion.Builder
The version of the SDK used to run the job.
Protobuf type google.events.cloud.dataflow.v1beta3.SdkVersion
SpannerIODetails
Metadata for a Spanner connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.SpannerIODetails
SpannerIODetails.Builder
Metadata for a Spanner connector used by the job.
Protobuf type google.events.cloud.dataflow.v1beta3.SpannerIODetails
WorkerPool
Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.
Protobuf type google.events.cloud.dataflow.v1beta3.WorkerPool
WorkerPool.Builder
Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.
Protobuf type google.events.cloud.dataflow.v1beta3.WorkerPool
Interfaces
AutoscalingSettingsOrBuilder
BigQueryIODetailsOrBuilder
BigTableIODetailsOrBuilder
DatastoreIODetailsOrBuilder
DebugOptionsOrBuilder
EnvironmentOrBuilder
ExecutionStageStateOrBuilder
FileIODetailsOrBuilder
JobEventDataOrBuilder
JobExecutionInfoOrBuilder
JobExecutionStageInfoOrBuilder
JobMetadataOrBuilder
JobOrBuilder
PackageOrBuilder
PubSubIODetailsOrBuilder
SdkHarnessContainerImageOrBuilder
SdkVersionOrBuilder
SpannerIODetailsOrBuilder
WorkerPoolOrBuilder
Enums
AutoscalingAlgorithm
Specifies the algorithm used to determine the number of worker processes to run at any given point in time, based on the amount of data left to process, the number of workers, and how quickly existing workers are processing data.
Protobuf enum google.events.cloud.dataflow.v1beta3.AutoscalingAlgorithm
DefaultPackageSet
The default set of packages to be staged on a pool of workers.
Protobuf enum google.events.cloud.dataflow.v1beta3.DefaultPackageSet
FlexResourceSchedulingGoal
Specifies the resource to optimize for in Flexible Resource Scheduling.
Protobuf enum google.events.cloud.dataflow.v1beta3.FlexResourceSchedulingGoal
JobState
Describes the overall state of a google.dataflow.v1beta3.Job.
Protobuf enum google.events.cloud.dataflow.v1beta3.JobState
JobType
Specifies the processing model used by a [google.dataflow.v1beta3.Job], which determines the way the Job is managed by the Cloud Dataflow service (how workers are scheduled, how inputs are sharded, etc).
Protobuf enum google.events.cloud.dataflow.v1beta3.JobType
SdkVersion.SdkSupportStatus
The support status of the SDK used to run the job.
Protobuf enum google.events.cloud.dataflow.v1beta3.SdkVersion.SdkSupportStatus
ShuffleMode
Specifies the shuffle mode used by a [google.dataflow.v1beta3.Job], which determines the approach data is shuffled during processing. More details in: https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#dataflow-shuffle
Protobuf enum google.events.cloud.dataflow.v1beta3.ShuffleMode
TeardownPolicy
Specifies what happens to a resource when a Cloud Dataflow google.dataflow.v1beta3.Job has completed.
Protobuf enum google.events.cloud.dataflow.v1beta3.TeardownPolicy
WorkerIPAddressConfiguration
Specifies how IP addresses should be allocated to the worker machines.
Protobuf enum google.events.cloud.dataflow.v1beta3.WorkerIPAddressConfiguration