Class FlexTemplateRuntimeEnvironment (0.7.0)

Version 0.7.0

• 0.8.15 (latest)
• 0.8.14
• 0.7.0
• 0.6.2
• 0.5.5
• 0.4.1
• 0.3.1
• 0.2.0
• 0.1.5

FlexTemplateRuntimeEnvironment(
    mapping=None, *, ignore_unknown_fields=False, **kwargs
)

The environment values to be set at runtime for flex template.

Attributes
Name	Description
num_workers	int The initial number of Google Compute Engine instances for the job.
max_workers	int The maximum number of Google Compute Engine instances to be made available to your pipeline during execution, from 1 to 1000.
zone	str The Compute Engine `availability zone
service_account_email	str The email address of the service account to run the job as.
temp_location	str The Cloud Storage path to use for temporary files. Must be a valid Cloud Storage URL, beginning with gs://.
machine_type	str The machine type to use for the job. Defaults to the value from the template if not specified.
additional_experiments	MutableSequence[str] Additional experiment flags for the job.
network	str Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
subnetwork	str Subnetwork to which VMs will be assigned, if desired. You can specify a subnetwork using either a complete URL or an abbreviated path. Expected to be of the form "https://www.googleapis.com/compute/v1/projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNETWORK" or "regions/REGION/subnetworks/SUBNETWORK". If the subnetwork is located in a Shared VPC network, you must use the complete URL.
additional_user_labels	MutableMapping[str, str] Additional user labels to be specified for the job. Keys and values must follow the restrictions specified in the `labeling restrictions
kms_key_name	str Name for the Cloud KMS key for the job. Key format is: projects/
ip_configuration	google.cloud.dataflow_v1beta3.types.WorkerIPAddressConfiguration Configuration for VM IPs.
worker_region	str The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.
worker_zone	str The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity. If both worker_zone and zone are set, worker_zone takes precedence.
enable_streaming_engine	bool Whether to enable Streaming Engine for the job.
flexrs_goal	google.cloud.dataflow_v1beta3.types.FlexResourceSchedulingGoal Set FlexRS goal for the job. https://cloud.google.com/dataflow/docs/guides/flexrs
staging_location	str The Cloud Storage path for staging local files. Must be a valid Cloud Storage URL, beginning with gs://.
sdk_container_image	str Docker registry location of container image to use for the 'worker harness. Default is the container for the version of the SDK. Note this field is only valid for portable pipelines.
disk_size_gb	int Worker disk size, in gigabytes.
autoscaling_algorithm	google.cloud.dataflow_v1beta3.types.AutoscalingAlgorithm The algorithm to use for autoscaling
dump_heap_on_oom	bool If true, save a heap dump before killing a thread or process which is GC thrashing or out of memory. The location of the heap file will either be echoed back to the user, or the user will be given the opportunity to download the heap file.
save_heap_dumps_to_gcs_path	str Cloud Storage bucket (directory) to upload heap dumps to the given location. Enabling this implies that heap dumps should be generated on OOM (dump_heap_on_oom is set to true).
launcher_machine_type	str The machine type to use for launching the job. The default is n1-standard-1.

Classes

AdditionalUserLabelsEntry

AdditionalUserLabelsEntry(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The abstract base class for a message.

Parameters
Name	Description
kwargs	dict Keys and values corresponding to the fields of the message.
mapping	Union[dict, .Message] A dictionary or message to be used to determine the values for this message.
ignore_unknown_fields	Optional(bool) If True, do not raise errors for unknown fields. Only applied if mapping is a mapping type or there are keyword parameters.