Google Cloud Dataflow V1beta3 Client - Class WorkerPool (0.6.4)

Reference documentation and code samples for the Google Cloud Dataflow V1beta3 Client class WorkerPool.

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Generated from protobuf message google.dataflow.v1beta3.WorkerPool

Namespace

Google \ Cloud \ Dataflow \ V1beta3

Methods

__construct

Constructor.

Parameters
Name Description
data array

Optional. Data for populating the Message object.
↳ kind string

The kind of the worker pool; currently only harness and shuffle are supported.
↳ num_workers int

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.
↳ packages array<Google\Cloud\Dataflow\V1beta3\Package>

Packages to be installed on workers.
↳ default_package_set int

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.
↳ machine_type string

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.
↳ teardown_policy int

Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.
↳ disk_size_gb int

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.
↳ disk_type string

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.
↳ disk_source_image string

Fully qualified source image for disks.
↳ zone string

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.
↳ taskrunner_settings Google\Cloud\Dataflow\V1beta3\TaskRunnerSettings

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.
↳ on_host_maintenance string

The action to take on host maintenance, as defined by the Google Compute Engine API.
↳ data_disks array<Google\Cloud\Dataflow\V1beta3\Disk>

Data disks that are used by a VM in this workflow.
↳ metadata array|Google\Protobuf\Internal\MapField

Metadata to set on the Google Compute Engine VMs.
↳ autoscaling_settings Google\Cloud\Dataflow\V1beta3\AutoscalingSettings

Settings for autoscaling of this WorkerPool.
↳ pool_args Google\Protobuf\Any

Extra arguments for this worker pool.
↳ network string

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
↳ subnetwork string

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".
↳ worker_harness_container_image string

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry. Deprecated for the Fn API path. Use sdk_harness_container_images instead.
↳ num_threads_per_worker int

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).
↳ ip_configuration int

Configuration for VM IPs.
↳ sdk_harness_container_images array<Google\Cloud\Dataflow\V1beta3\SdkHarnessContainerImage>

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

getKind

The kind of the worker pool; currently only harness and shuffle are supported.

Returns
Type Description
string

setKind

The kind of the worker pool; currently only harness and shuffle are supported.

Parameter
Name Description
var string
Returns
Type Description
$this

getNumWorkers

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

Returns
Type Description
int

setNumWorkers

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name Description
var int
Returns
Type Description
$this

getPackages

Packages to be installed on workers.

Returns
Type Description
Google\Protobuf\Internal\RepeatedField

setPackages

Packages to be installed on workers.

Parameter
Name Description
var array<Google\Cloud\Dataflow\V1beta3\Package>
Returns
Type Description
$this

getDefaultPackageSet

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

Returns
Type Description
int

setDefaultPackageSet

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

Parameter
Name Description
var int
Returns
Type Description
$this

getMachineType

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

Returns
Type Description
string

setMachineType

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name Description
var string
Returns
Type Description
$this

getTeardownPolicy

Sets the policy for determining when to turndown worker pool.

Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.

Returns
Type Description
int

setTeardownPolicy

Sets the policy for determining when to turndown worker pool.

Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name Description
var int
Returns
Type Description
$this

getDiskSizeGb

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

Returns
Type Description
int

setDiskSizeGb

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name Description
var int
Returns
Type Description
$this

getDiskType

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

Returns
Type Description
string

setDiskType

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name Description
var string
Returns
Type Description
$this

getDiskSourceImage

Fully qualified source image for disks.

Returns
Type Description
string

setDiskSourceImage

Fully qualified source image for disks.

Parameter
Name Description
var string
Returns
Type Description
$this

getZone

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

Returns
Type Description
string

setZone

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name Description
var string
Returns
Type Description
$this

getTaskrunnerSettings

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

Returns
Type Description
Google\Cloud\Dataflow\V1beta3\TaskRunnerSettings|null

hasTaskrunnerSettings

clearTaskrunnerSettings

setTaskrunnerSettings

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

Parameter
Name Description
var Google\Cloud\Dataflow\V1beta3\TaskRunnerSettings
Returns
Type Description
$this

getOnHostMaintenance

The action to take on host maintenance, as defined by the Google Compute Engine API.

Returns
Type Description
string

setOnHostMaintenance

The action to take on host maintenance, as defined by the Google Compute Engine API.

Parameter
Name Description
var string
Returns
Type Description
$this

getDataDisks

Data disks that are used by a VM in this workflow.

Returns
Type Description
Google\Protobuf\Internal\RepeatedField

setDataDisks

Data disks that are used by a VM in this workflow.

Parameter
Name Description
var array<Google\Cloud\Dataflow\V1beta3\Disk>
Returns
Type Description
$this

getMetadata

Metadata to set on the Google Compute Engine VMs.

Returns
Type Description
Google\Protobuf\Internal\MapField

setMetadata

Metadata to set on the Google Compute Engine VMs.

Parameter
Name Description
var array|Google\Protobuf\Internal\MapField
Returns
Type Description
$this

getAutoscalingSettings

Settings for autoscaling of this WorkerPool.

Returns
Type Description
Google\Cloud\Dataflow\V1beta3\AutoscalingSettings|null

hasAutoscalingSettings

clearAutoscalingSettings

setAutoscalingSettings

Settings for autoscaling of this WorkerPool.

Parameter
Name Description
var Google\Cloud\Dataflow\V1beta3\AutoscalingSettings
Returns
Type Description
$this

getPoolArgs

Extra arguments for this worker pool.

Returns
Type Description
Google\Protobuf\Any|null

hasPoolArgs

clearPoolArgs

setPoolArgs

Extra arguments for this worker pool.

Parameter
Name Description
var Google\Protobuf\Any
Returns
Type Description
$this

getNetwork

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

Returns
Type Description
string

setNetwork

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

Parameter
Name Description
var string
Returns
Type Description
$this

getSubnetwork

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

Returns
Type Description
string

setSubnetwork

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

Parameter
Name Description
var string
Returns
Type Description
$this

getWorkerHarnessContainerImage

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Returns
Type Description
string

setWorkerHarnessContainerImage

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Parameter
Name Description
var string
Returns
Type Description
$this

getNumThreadsPerWorker

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

Returns
Type Description
int

setNumThreadsPerWorker

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

Parameter
Name Description
var int
Returns
Type Description
$this

getIpConfiguration

Configuration for VM IPs.

Returns
Type Description
int

setIpConfiguration

Configuration for VM IPs.

Parameter
Name Description
var int
Returns
Type Description
$this

getSdkHarnessContainerImages

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Returns
Type Description
Google\Protobuf\Internal\RepeatedField

setSdkHarnessContainerImages

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Parameter
Name Description
var array<Google\Cloud\Dataflow\V1beta3\SdkHarnessContainerImage>
Returns
Type Description
$this