Dataflow V1beta3 API - Class Google::Cloud::Dataflow::V1beta3::Environment (v0.3.0)

Reference documentation and code samples for the Dataflow V1beta3 API class Google::Cloud::Dataflow::V1beta3::Environment.

Describes the environment in which a Dataflow Job runs.

Inherits

Object

Extended By

Google::Protobuf::MessageExts::ClassMethods

Includes

Google::Protobuf::MessageExts

Methods

#cluster_manager_api_service

def cluster_manager_api_service() -> ::String

Returns

(::String) — The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

#cluster_manager_api_service=

def cluster_manager_api_service=(value) -> ::String

Parameter

value (::String) — The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Returns

(::String) — The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

#dataset

def dataset() -> ::String

Returns

(::String) — The dataset for the current project where various workflow related tables are stored.

The supported resource type is:

Google BigQuery: bigquery.googleapis.com/{dataset}

#dataset=

def dataset=(value) -> ::String

Parameter

value (::String) — The dataset for the current project where various workflow related tables are stored.

The supported resource type is:

Google BigQuery: bigquery.googleapis.com/{dataset}

Returns

(::String) — The dataset for the current project where various workflow related tables are stored.

The supported resource type is:

Google BigQuery: bigquery.googleapis.com/{dataset}

#debug_options

def debug_options() -> ::Google::Cloud::Dataflow::V1beta3::DebugOptions

Returns

(::Google::Cloud::Dataflow::V1beta3::DebugOptions) — Any debugging options to be supplied to the job.

#debug_options=

def debug_options=(value) -> ::Google::Cloud::Dataflow::V1beta3::DebugOptions

Parameter

value (::Google::Cloud::Dataflow::V1beta3::DebugOptions) — Any debugging options to be supplied to the job.

Returns

(::Google::Cloud::Dataflow::V1beta3::DebugOptions) — Any debugging options to be supplied to the job.

#experiments

def experiments() -> ::Array<::String>

Returns

(::Array<::String>) — The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

#experiments=

def experiments=(value) -> ::Array<::String>

Parameter

value (::Array<::String>) — The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Returns

(::Array<::String>) — The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

#flex_resource_scheduling_goal

def flex_resource_scheduling_goal() -> ::Google::Cloud::Dataflow::V1beta3::FlexResourceSchedulingGoal

Returns

(::Google::Cloud::Dataflow::V1beta3::FlexResourceSchedulingGoal) — Which Flexible Resource Scheduling mode to run in.

#flex_resource_scheduling_goal=

def flex_resource_scheduling_goal=(value) -> ::Google::Cloud::Dataflow::V1beta3::FlexResourceSchedulingGoal

Parameter

value (::Google::Cloud::Dataflow::V1beta3::FlexResourceSchedulingGoal) — Which Flexible Resource Scheduling mode to run in.

Returns

(::Google::Cloud::Dataflow::V1beta3::FlexResourceSchedulingGoal) — Which Flexible Resource Scheduling mode to run in.

#internal_experiments

def internal_experiments() -> ::Google::Protobuf::Any

Returns

(::Google::Protobuf::Any) — Experimental settings.

#internal_experiments=

def internal_experiments=(value) -> ::Google::Protobuf::Any

Parameter

value (::Google::Protobuf::Any) — Experimental settings.

Returns

(::Google::Protobuf::Any) — Experimental settings.

#sdk_pipeline_options

def sdk_pipeline_options() -> ::Google::Protobuf::Struct

Returns

(::Google::Protobuf::Struct) — The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

#sdk_pipeline_options=

def sdk_pipeline_options=(value) -> ::Google::Protobuf::Struct

Parameter

value (::Google::Protobuf::Struct) — The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Returns

(::Google::Protobuf::Struct) — The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

#service_account_email

def service_account_email() -> ::String

Returns

(::String) — Identity to run virtual machines as. Defaults to the default account.

#service_account_email=

def service_account_email=(value) -> ::String

Parameter

value (::String) — Identity to run virtual machines as. Defaults to the default account.

Returns

(::String) — Identity to run virtual machines as. Defaults to the default account.

#service_kms_key_name

def service_kms_key_name() -> ::String

Returns

(::String) — If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

#service_kms_key_name=

def service_kms_key_name=(value) -> ::String

Parameter

value (::String) — If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Returns

(::String) — If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

#service_options

def service_options() -> ::Array<::String>

Returns

(::Array<::String>) — The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

#service_options=

def service_options=(value) -> ::Array<::String>

Parameter

value (::Array<::String>) — The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Returns

(::Array<::String>) — The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

#shuffle_mode

def shuffle_mode() -> ::Google::Cloud::Dataflow::V1beta3::ShuffleMode

Returns

(::Google::Cloud::Dataflow::V1beta3::ShuffleMode) — Output only. The shuffle mode used for the job.

#temp_storage_prefix

def temp_storage_prefix() -> ::String

Returns

(::String) — The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

Google Cloud Storage:

storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

#temp_storage_prefix=

def temp_storage_prefix=(value) -> ::String

Parameter

value (::String) — The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

Google Cloud Storage:

storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Returns

(::String) — The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

Google Cloud Storage:

storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

#user_agent

def user_agent() -> ::Google::Protobuf::Struct

Returns

(::Google::Protobuf::Struct) — A description of the process that generated the request.

#user_agent=

def user_agent=(value) -> ::Google::Protobuf::Struct

Parameter

value (::Google::Protobuf::Struct) — A description of the process that generated the request.

Returns

(::Google::Protobuf::Struct) — A description of the process that generated the request.

#version

def version() -> ::Google::Protobuf::Struct

Returns

(::Google::Protobuf::Struct) — A structure describing which components and their versions of the service are required in order to run the job.

#version=

def version=(value) -> ::Google::Protobuf::Struct

Parameter

value (::Google::Protobuf::Struct) — A structure describing which components and their versions of the service are required in order to run the job.

Returns

(::Google::Protobuf::Struct) — A structure describing which components and their versions of the service are required in order to run the job.

#worker_pools

def worker_pools() -> ::Array<::Google::Cloud::Dataflow::V1beta3::WorkerPool>

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::WorkerPool>) — The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

#worker_pools=

def worker_pools=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::WorkerPool>

Parameter

value (::Array<::Google::Cloud::Dataflow::V1beta3::WorkerPool>) — The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Returns

(::Array<::Google::Cloud::Dataflow::V1beta3::WorkerPool>) — The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

#worker_region

def worker_region() -> ::String

Returns

(::String) — The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

#worker_region=

def worker_region=(value) -> ::String

Parameter

value (::String) — The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Returns

(::String) — The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

#worker_zone

def worker_zone() -> ::String

Returns

(::String) — The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

#worker_zone=

def worker_zone=(value) -> ::String

Parameter

value (::String) — The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Returns

(::String) — The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.