Reference documentation and code samples for the Google Cloud Dataflow V1beta3 Client class WorkerPool.
Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.
Generated from protobuf message google.dataflow.v1beta3.WorkerPool
Namespace
Google \ Cloud \ Dataflow \ V1beta3Methods
__construct
Constructor.
| Parameters | |
|---|---|
| Name | Description |
data |
array
Optional. Data for populating the Message object. |
↳ kind |
string
The kind of the worker pool; currently only |
↳ num_workers |
int
Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default. |
↳ packages |
array<Package>
Packages to be installed on workers. |
↳ default_package_set |
int
The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language. |
↳ machine_type |
string
Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default. |
↳ teardown_policy |
int
Sets the policy for determining when to turndown worker pool. Allowed values are: |
↳ disk_size_gb |
int
Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default. |
↳ disk_type |
string
Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default. |
↳ disk_source_image |
string
Fully qualified source image for disks. |
↳ zone |
string
Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default. |
↳ taskrunner_settings |
TaskRunnerSettings
Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field. |
↳ on_host_maintenance |
string
The action to take on host maintenance, as defined by the Google Compute Engine API. |
↳ data_disks |
array<Disk>
Data disks that are used by a VM in this workflow. |
↳ metadata |
array|Google\Protobuf\Internal\MapField
Metadata to set on the Google Compute Engine VMs. |
↳ autoscaling_settings |
AutoscalingSettings
Settings for autoscaling of this WorkerPool. |
↳ pool_args |
Google\Protobuf\Any
Extra arguments for this worker pool. |
↳ network |
string
Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default". |
↳ subnetwork |
string
Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK". |
↳ worker_harness_container_image |
string
Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry. Deprecated for the Fn API path. Use sdk_harness_container_images instead. |
↳ num_threads_per_worker |
int
The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming). |
↳ ip_configuration |
int
Configuration for VM IPs. |
↳ sdk_harness_container_images |
array<SdkHarnessContainerImage>
Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries. |
getKind
The kind of the worker pool; currently only harness and shuffle
are supported.
| Returns | |
|---|---|
| Type | Description |
string |
|
setKind
The kind of the worker pool; currently only harness and shuffle
are supported.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getNumWorkers
Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.
| Returns | |
|---|---|
| Type | Description |
int |
|
setNumWorkers
Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getPackages
Packages to be installed on workers.
| Returns | |
|---|---|
| Type | Description |
Google\Protobuf\Internal\RepeatedField |
|
setPackages
Packages to be installed on workers.
| Parameter | |
|---|---|
| Name | Description |
var |
array<Package>
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getDefaultPackageSet
The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.
| Returns | |
|---|---|
| Type | Description |
int |
|
setDefaultPackageSet
The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getMachineType
Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.
| Returns | |
|---|---|
| Type | Description |
string |
|
setMachineType
Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getTeardownPolicy
Sets the policy for determining when to turndown worker pool.
Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and
TEARDOWN_NEVER.
TEARDOWN_ALWAYS means workers are always torn down regardless of whether
the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down
if the job succeeds. TEARDOWN_NEVER means the workers are never torn
down.
If the workers are not torn down by the service, they will
continue to run and use Google Compute Engine VM resources in the
user's project until they are explicitly terminated by the user.
Because of this, Google recommends using the TEARDOWN_ALWAYS
policy except for small, manually supervised test jobs.
If unknown or unspecified, the service will attempt to choose a reasonable
default.
| Returns | |
|---|---|
| Type | Description |
int |
|
setTeardownPolicy
Sets the policy for determining when to turndown worker pool.
Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and
TEARDOWN_NEVER.
TEARDOWN_ALWAYS means workers are always torn down regardless of whether
the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down
if the job succeeds. TEARDOWN_NEVER means the workers are never torn
down.
If the workers are not torn down by the service, they will
continue to run and use Google Compute Engine VM resources in the
user's project until they are explicitly terminated by the user.
Because of this, Google recommends using the TEARDOWN_ALWAYS
policy except for small, manually supervised test jobs.
If unknown or unspecified, the service will attempt to choose a reasonable
default.
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getDiskSizeGb
Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.
| Returns | |
|---|---|
| Type | Description |
int |
|
setDiskSizeGb
Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getDiskType
Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.
| Returns | |
|---|---|
| Type | Description |
string |
|
setDiskType
Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getDiskSourceImage
Fully qualified source image for disks.
| Returns | |
|---|---|
| Type | Description |
string |
|
setDiskSourceImage
Fully qualified source image for disks.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getZone
Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.
| Returns | |
|---|---|
| Type | Description |
string |
|
setZone
Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getTaskrunnerSettings
Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.
| Returns | |
|---|---|
| Type | Description |
TaskRunnerSettings|null |
|
hasTaskrunnerSettings
clearTaskrunnerSettings
setTaskrunnerSettings
Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.
| Parameter | |
|---|---|
| Name | Description |
var |
TaskRunnerSettings
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getOnHostMaintenance
The action to take on host maintenance, as defined by the Google Compute Engine API.
| Returns | |
|---|---|
| Type | Description |
string |
|
setOnHostMaintenance
The action to take on host maintenance, as defined by the Google Compute Engine API.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getDataDisks
Data disks that are used by a VM in this workflow.
| Returns | |
|---|---|
| Type | Description |
Google\Protobuf\Internal\RepeatedField |
|
setDataDisks
Data disks that are used by a VM in this workflow.
| Parameter | |
|---|---|
| Name | Description |
var |
array<Disk>
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getMetadata
Metadata to set on the Google Compute Engine VMs.
| Returns | |
|---|---|
| Type | Description |
Google\Protobuf\Internal\MapField |
|
setMetadata
Metadata to set on the Google Compute Engine VMs.
| Parameter | |
|---|---|
| Name | Description |
var |
array|Google\Protobuf\Internal\MapField
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getAutoscalingSettings
Settings for autoscaling of this WorkerPool.
| Returns | |
|---|---|
| Type | Description |
AutoscalingSettings|null |
|
hasAutoscalingSettings
clearAutoscalingSettings
setAutoscalingSettings
Settings for autoscaling of this WorkerPool.
| Parameter | |
|---|---|
| Name | Description |
var |
AutoscalingSettings
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getPoolArgs
Extra arguments for this worker pool.
| Returns | |
|---|---|
| Type | Description |
Google\Protobuf\Any|null |
|
hasPoolArgs
clearPoolArgs
setPoolArgs
Extra arguments for this worker pool.
| Parameter | |
|---|---|
| Name | Description |
var |
Google\Protobuf\Any
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getNetwork
Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
| Returns | |
|---|---|
| Type | Description |
string |
|
setNetwork
Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getSubnetwork
Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".
| Returns | |
|---|---|
| Type | Description |
string |
|
setSubnetwork
Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getWorkerHarnessContainerImage
Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.
Deprecated for the Fn API path. Use sdk_harness_container_images instead.
| Returns | |
|---|---|
| Type | Description |
string |
|
setWorkerHarnessContainerImage
Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.
Deprecated for the Fn API path. Use sdk_harness_container_images instead.
| Parameter | |
|---|---|
| Name | Description |
var |
string
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getNumThreadsPerWorker
The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).
| Returns | |
|---|---|
| Type | Description |
int |
|
setNumThreadsPerWorker
The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getIpConfiguration
Configuration for VM IPs.
| Returns | |
|---|---|
| Type | Description |
int |
|
setIpConfiguration
Configuration for VM IPs.
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getSdkHarnessContainerImages
Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.
| Returns | |
|---|---|
| Type | Description |
Google\Protobuf\Internal\RepeatedField |
|
setSdkHarnessContainerImages
Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.
| Parameter | |
|---|---|
| Name | Description |
var |
array<SdkHarnessContainerImage>
|
| Returns | |
|---|---|
| Type | Description |
$this |
|