Google Cloud Dataflow V1beta3 Client - Class WorkerPool (0.5.1)

Version 0.5.1

• 0.6.6 (latest)
• 0.6.5
• 0.5.1
• 0.4.1
• 0.3.10

Beta

This library is covered by the Pre-GA Offerings Terms of the Terms of Service. Pre-GA libraries might have limited support, and changes to pre-GA libraries might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

Reference documentation and code samples for the Google Cloud Dataflow V1beta3 Client class WorkerPool.

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Generated from protobuf message google.dataflow.v1beta3.WorkerPool

Namespace

Google \ Cloud \ Dataflow \ V1beta3

Methods

__construct

Constructor.

Parameters
Name	Description
data	array Optional. Data for populating the Message object.
↳ kind	string The kind of the worker pool; currently only harness and shuffle are supported.
↳ num_workers	int Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.
↳ packages	array<Google\Cloud\Dataflow\V1beta3\Package> Packages to be installed on workers.
↳ default_package_set	int The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.
↳ machine_type	string Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.
↳ teardown_policy	int Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.
↳ disk_size_gb	int Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.
↳ disk_type	string Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.
↳ disk_source_image	string Fully qualified source image for disks.
↳ zone	string Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.
↳ taskrunner_settings	Google\Cloud\Dataflow\V1beta3\TaskRunnerSettings Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.
↳ on_host_maintenance	string The action to take on host maintenance, as defined by the Google Compute Engine API.
↳ data_disks	array<Google\Cloud\Dataflow\V1beta3\Disk> Data disks that are used by a VM in this workflow.
↳ metadata	array|Google\Protobuf\Internal\MapField Metadata to set on the Google Compute Engine VMs.
↳ autoscaling_settings	Google\Cloud\Dataflow\V1beta3\AutoscalingSettings Settings for autoscaling of this WorkerPool.
↳ pool_args	Google\Protobuf\Any Extra arguments for this worker pool.
↳ network	string Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
↳ subnetwork	string Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".
↳ worker_harness_container_image	string Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry. Deprecated for the Fn API path. Use sdk_harness_container_images instead.
↳ num_threads_per_worker	int The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).
↳ ip_configuration	int Configuration for VM IPs.
↳ sdk_harness_container_images	array<Google\Cloud\Dataflow\V1beta3\SdkHarnessContainerImage> Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

getKind

The kind of the worker pool; currently only harness and shuffle are supported.

Returns
Type	Description
string

setKind

The kind of the worker pool; currently only harness and shuffle are supported.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getNumWorkers

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

Returns
Type	Description
int

setNumWorkers

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getPackages

Packages to be installed on workers.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setPackages

Packages to be installed on workers.

Parameter
Name	Description
var	array<Google\Cloud\Dataflow\V1beta3\Package>

Returns
Type	Description
$this

getDefaultPackageSet

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

Returns
Type	Description
int

setDefaultPackageSet

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getMachineType

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

Returns
Type	Description
string

setMachineType

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getTeardownPolicy

Sets the policy for determining when to turndown worker pool.

Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.

Returns
Type	Description
int

setTeardownPolicy

Sets the policy for determining when to turndown worker pool.

Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getDiskSizeGb

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

Returns
Type	Description
int

setDiskSizeGb

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getDiskType

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

Returns
Type	Description
string

setDiskType

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getDiskSourceImage

Fully qualified source image for disks.

Returns
Type	Description
string

setDiskSourceImage

Fully qualified source image for disks.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getZone

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

Returns
Type	Description
string

setZone

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getTaskrunnerSettings

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

Returns
Type	Description
Google\Cloud\Dataflow\V1beta3\TaskRunnerSettings|null

hasTaskrunnerSettings

clearTaskrunnerSettings

setTaskrunnerSettings

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

Parameter
Name	Description
var	Google\Cloud\Dataflow\V1beta3\TaskRunnerSettings

Returns
Type	Description
$this

getOnHostMaintenance

The action to take on host maintenance, as defined by the Google Compute Engine API.

Returns
Type	Description
string

setOnHostMaintenance

The action to take on host maintenance, as defined by the Google Compute Engine API.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getDataDisks

Data disks that are used by a VM in this workflow.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setDataDisks

Data disks that are used by a VM in this workflow.

Parameter
Name	Description
var	array<Google\Cloud\Dataflow\V1beta3\Disk>

Returns
Type	Description
$this

getMetadata

Metadata to set on the Google Compute Engine VMs.

Returns
Type	Description
Google\Protobuf\Internal\MapField

setMetadata

Metadata to set on the Google Compute Engine VMs.

Parameter
Name	Description
var	array|Google\Protobuf\Internal\MapField

Returns
Type	Description
$this

getAutoscalingSettings

Settings for autoscaling of this WorkerPool.

Returns
Type	Description
Google\Cloud\Dataflow\V1beta3\AutoscalingSettings|null

hasAutoscalingSettings

clearAutoscalingSettings

setAutoscalingSettings

Settings for autoscaling of this WorkerPool.

Parameter
Name	Description
var	Google\Cloud\Dataflow\V1beta3\AutoscalingSettings

Returns
Type	Description
$this

getPoolArgs

Extra arguments for this worker pool.

Returns
Type	Description
Google\Protobuf\Any|null

hasPoolArgs

clearPoolArgs

setPoolArgs

Extra arguments for this worker pool.

Parameter
Name	Description
var	Google\Protobuf\Any

Returns
Type	Description
$this

getNetwork

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

Returns
Type	Description
string

setNetwork

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getSubnetwork

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

Returns
Type	Description
string

setSubnetwork

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getWorkerHarnessContainerImage

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Returns
Type	Description
string

setWorkerHarnessContainerImage

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getNumThreadsPerWorker

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

Returns
Type	Description
int

setNumThreadsPerWorker

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getIpConfiguration

Configuration for VM IPs.

Returns
Type	Description
int

setIpConfiguration

Configuration for VM IPs.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getSdkHarnessContainerImages

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setSdkHarnessContainerImages

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Parameter
Name	Description
var	array<Google\Cloud\Dataflow\V1beta3\SdkHarnessContainerImage>

Returns
Type	Description
$this