Class FlexTemplateRuntimeEnvironment (0.6.0)

FlexTemplateRuntimeEnvironment(
    mapping=None, *, ignore_unknown_fields=False, **kwargs
)

The environment values to be set at runtime for flex template.

Attributes

Name	Description
num_workers	`int` The initial number of Google Compute Engine instances for the job.
max_workers	`int` The maximum number of Google Compute Engine instances to be made available to your pipeline during execution, from 1 to 1000.
zone	`str` The Compute Engine `availability zone
service_account_email	`str` The email address of the service account to run the job as.
temp_location	`str` The Cloud Storage path to use for temporary files. Must be a valid Cloud Storage URL, beginning with ``gs://``.
machine_type	`str` The machine type to use for the job. Defaults to the value from the template if not specified.
additional_experiments	`Sequence[str]` Additional experiment flags for the job.
network	`str` Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
subnetwork	`str` Subnetwork to which VMs will be assigned, if desired. You can specify a subnetwork using either a complete URL or an abbreviated path. Expected to be of the form "https://www.googleapis.com/compute/v1/projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNETWORK" or "regions/REGION/subnetworks/SUBNETWORK". If the subnetwork is located in a Shared VPC network, you must use the complete URL.
additional_user_labels	`Mapping[str, str]` Additional user labels to be specified for the job. Keys and values must follow the restrictions specified in the `labeling restrictions
kms_key_name	`str` Name for the Cloud KMS key for the job. Key format is: projects/
ip_configuration	`google.cloud.dataflow_v1beta3.types.WorkerIPAddressConfiguration` Configuration for VM IPs.
worker_region	`str` The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.
worker_zone	`str` The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity. If both ``worker_zone`` and ``zone`` are set, ``worker_zone`` takes precedence.
enable_streaming_engine	`bool` Whether to enable Streaming Engine for the job.
flexrs_goal	`google.cloud.dataflow_v1beta3.types.FlexResourceSchedulingGoal` Set FlexRS goal for the job. https://cloud.google.com/dataflow/docs/guides/flexrs
staging_location	`str` The Cloud Storage path for staging local files. Must be a valid Cloud Storage URL, beginning with ``gs://``.
sdk_container_image	`str` Docker registry location of container image to use for the 'worker harness. Default is the container for the version of the SDK. Note this field is only valid for portable pipelines.
disk_size_gb	`int` Worker disk size, in gigabytes.
autoscaling_algorithm	`google.cloud.dataflow_v1beta3.types.AutoscalingAlgorithm` The algorithm to use for autoscaling
dump_heap_on_oom	`bool` If true, save a heap dump before killing a thread or process which is GC thrashing or out of memory. The location of the heap file will either be echoed back to the user, or the user will be given the opportunity to download the heap file.
save_heap_dumps_to_gcs_path	`str` Cloud Storage bucket (directory) to upload heap dumps to the given location. Enabling this implies that heap dumps should be generated on OOM (dump_heap_on_oom is set to true).
launcher_machine_type	`str` The machine type to use for launching the job. The default is n1-standard-1.

Inheritance

builtins.object > proto.message.Message > FlexTemplateRuntimeEnvironment

Classes

AdditionalUserLabelsEntry

AdditionalUserLabelsEntry(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The abstract base class for a message.

Parameters

Name	Description
kwargs	`dict` Keys and values corresponding to the fields of the message.
mapping	Union[dict, `.Message`] A dictionary or message to be used to determine the values for this message.
ignore_unknown_fields	`Optional(bool)` If True, do not raise errors for unknown fields. Only applied if `mapping` is a mapping type or there are keyword parameters.