Google Cloud Dataflow V1beta3 Client - Class Environment (0.6.6)

Version latest

• 0.6.6 (latest)
• 0.6.5
• 0.5.1
• 0.4.1
• 0.3.10

Beta

This library is covered by the Pre-GA Offerings Terms of the Terms of Service. Pre-GA libraries might have limited support, and changes to pre-GA libraries might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

Reference documentation and code samples for the Google Cloud Dataflow V1beta3 Client class Environment.

Describes the environment in which a Dataflow Job runs.

Generated from protobuf message google.dataflow.v1beta3.Environment

Namespace

Google \ Cloud \ Dataflow \ V1beta3

Methods

__construct

Constructor.

Parameters
Name	Description
data	array Optional. Data for populating the Message object.
↳ temp_storage_prefix	string The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
↳ cluster_manager_api_service	string The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".
↳ experiments	array The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.
↳ service_options	array The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).
↳ service_kms_key_name	string If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK). Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY
↳ worker_pools	array<WorkerPool> The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.
↳ user_agent	Google\Protobuf\Struct A description of the process that generated the request.
↳ version	Google\Protobuf\Struct A structure describing which components and their versions of the service are required in order to run the job.
↳ dataset	string The dataset for the current project where various workflow related tables are stored. The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}
↳ sdk_pipeline_options	Google\Protobuf\Struct The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.
↳ internal_experiments	Google\Protobuf\Any Experimental settings.
↳ service_account_email	string Identity to run virtual machines as. Defaults to the default account.
↳ flex_resource_scheduling_goal	int Which Flexible Resource Scheduling mode to run in.
↳ worker_region	string The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.
↳ worker_zone	string The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.
↳ shuffle_mode	int Output only. The shuffle mode used for the job.
↳ debug_options	DebugOptions Any debugging options to be supplied to the job.

getTempStoragePrefix

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings.

The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Returns
Type	Description
string

setTempStoragePrefix

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings.

The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getClusterManagerApiService

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Returns
Type	Description
string

setClusterManagerApiService

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getExperiments

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setExperiments

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Parameter
Name	Description
var	string[]

Returns
Type	Description
$this

getServiceOptions

The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setServiceOptions

The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Parameter
Name	Description
var	string[]

Returns
Type	Description
$this

getServiceKmsKeyName

If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Returns
Type	Description
string

setServiceKmsKeyName

If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getWorkerPools

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setWorkerPools

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Parameter
Name	Description
var	array<WorkerPool>

Returns
Type	Description
$this

getUserAgent

A description of the process that generated the request.

Returns
Type	Description
Google\Protobuf\Struct|null

hasUserAgent

clearUserAgent

setUserAgent

A description of the process that generated the request.

Parameter
Name	Description
var	Google\Protobuf\Struct

Returns
Type	Description
$this

getVersion

A structure describing which components and their versions of the service are required in order to run the job.

Returns
Type	Description
Google\Protobuf\Struct|null

hasVersion

clearVersion

setVersion

A structure describing which components and their versions of the service are required in order to run the job.

Parameter
Name	Description
var	Google\Protobuf\Struct

Returns
Type	Description
$this

getDataset

The dataset for the current project where various workflow related tables are stored.

The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}

Returns
Type	Description
string

setDataset

The dataset for the current project where various workflow related tables are stored.

The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getSdkPipelineOptions

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Returns
Type	Description
Google\Protobuf\Struct|null

hasSdkPipelineOptions

clearSdkPipelineOptions

setSdkPipelineOptions

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Parameter
Name	Description
var	Google\Protobuf\Struct

Returns
Type	Description
$this

getInternalExperiments

Experimental settings.

Returns
Type	Description
Google\Protobuf\Any|null

hasInternalExperiments

clearInternalExperiments

setInternalExperiments

Experimental settings.

Parameter
Name	Description
var	Google\Protobuf\Any

Returns
Type	Description
$this

getServiceAccountEmail

Identity to run virtual machines as. Defaults to the default account.

Returns
Type	Description
string

setServiceAccountEmail

Identity to run virtual machines as. Defaults to the default account.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getFlexResourceSchedulingGoal

Which Flexible Resource Scheduling mode to run in.

Returns
Type	Description
int

setFlexResourceSchedulingGoal

Which Flexible Resource Scheduling mode to run in.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getWorkerRegion

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Returns
Type	Description
string

setWorkerRegion

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getWorkerZone

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Returns
Type	Description
string

setWorkerZone

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getShuffleMode

Output only. The shuffle mode used for the job.

Returns
Type	Description
int

setShuffleMode

Output only. The shuffle mode used for the job.

Parameter
Name	Description
var	int

Returns
Type	Description
$this

getDebugOptions

Any debugging options to be supplied to the job.

Returns
Type	Description
DebugOptions|null

hasDebugOptions

clearDebugOptions

setDebugOptions

Any debugging options to be supplied to the job.

Parameter
Name	Description
var	DebugOptions

Returns
Type	Description
$this