RuntimeEnvironment(mapping=None, *, ignore_unknown_fields=False, **kwargs)The environment values to set at runtime.
Attributes | |
|---|---|
| Name | Description |
num_workers |
int
The initial number of Google Compute Engine instnaces for the job. |
max_workers |
int
The maximum number of Google Compute Engine instances to be made available to your pipeline during execution, from 1 to 1000. |
zone |
str
The Compute Engine `availability zone |
service_account_email |
str
The email address of the service account to run the job as. |
temp_location |
str
The Cloud Storage path to use for temporary files. Must be a valid Cloud Storage URL, beginning with gs://.
|
bypass_temp_dir_validation |
bool
Whether to bypass the safety checks for the job's temporary directory. Use with caution. |
machine_type |
str
The machine type to use for the job. Defaults to the value from the template if not specified. |
additional_experiments |
MutableSequence[str]
Additional experiment flags for the job, specified with the --experiments option.
|
network |
str
Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default". |
subnetwork |
str
Subnetwork to which VMs will be assigned, if desired. You can specify a subnetwork using either a complete URL or an abbreviated path. Expected to be of the form "https://www.googleapis.com/compute/v1/projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNETWORK" or "regions/REGION/subnetworks/SUBNETWORK". If the subnetwork is located in a Shared VPC network, you must use the complete URL. |
additional_user_labels |
MutableMapping[str, str]
Additional user labels to be specified for the job. Keys and values should follow the restrictions specified in the `labeling restrictions |
kms_key_name |
str
Name for the Cloud KMS key for the job. Key format is: projects/ |
ip_configuration |
google.cloud.dataflow_v1beta3.types.WorkerIPAddressConfiguration
Configuration for VM IPs. |
worker_region |
str
The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region. |
worker_zone |
str
The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity. If both worker_zone and zone are set,
worker_zone takes precedence.
|
enable_streaming_engine |
bool
Whether to enable Streaming Engine for the job. |
Classes
AdditionalUserLabelsEntry
AdditionalUserLabelsEntry(mapping=None, *, ignore_unknown_fields=False, **kwargs)The abstract base class for a message.
| Parameters | |
|---|---|
| Name | Description |
kwargs |
dict
Keys and values corresponding to the fields of the message. |
mapping |
Union[dict,
A dictionary or message to be used to determine the values for this message. |
ignore_unknown_fields |
Optional(bool)
If True, do not raise errors for unknown fields. Only applied if |