Dataflow v1beta3 API - Class Environment (2.0.0-beta06)

public sealed class Environment : IMessage<Environment>, IEquatable<Environment>, IDeepCloneable<Environment>, IBufferMessage, IMessage

Reference documentation and code samples for the Dataflow v1beta3 API class Environment.

Describes the environment in which a Dataflow Job runs.

Inheritance

object > Environment

Namespace

Google.Cloud.Dataflow.V1Beta3

Assembly

Google.Cloud.Dataflow.V1Beta3.dll

Constructors

Environment()

public Environment()

Environment(Environment)

public Environment(Environment other)
Parameter
Name Description
other Environment

Properties

ClusterManagerApiService

public string ClusterManagerApiService { get; set; }

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Property Value
Type Description
string

Dataset

public string Dataset { get; set; }

The dataset for the current project where various workflow related tables are stored.

The supported resource type is:

Google BigQuery: bigquery.googleapis.com/{dataset}

Property Value
Type Description
string

DebugOptions

public DebugOptions DebugOptions { get; set; }

Any debugging options to be supplied to the job.

Property Value
Type Description
DebugOptions

Experiments

public RepeatedField<string> Experiments { get; }

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Property Value
Type Description
RepeatedFieldstring

FlexResourceSchedulingGoal

public FlexResourceSchedulingGoal FlexResourceSchedulingGoal { get; set; }

Which Flexible Resource Scheduling mode to run in.

Property Value
Type Description
FlexResourceSchedulingGoal

InternalExperiments

public Any InternalExperiments { get; set; }

Experimental settings.

Property Value
Type Description
Any

SdkPipelineOptions

public Struct SdkPipelineOptions { get; set; }

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Property Value
Type Description
Struct

ServiceAccountEmail

public string ServiceAccountEmail { get; set; }

Identity to run virtual machines as. Defaults to the default account.

Property Value
Type Description
string

ServiceKmsKeyName

public string ServiceKmsKeyName { get; set; }

If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Property Value
Type Description
string

ServiceOptions

public RepeatedField<string> ServiceOptions { get; }

The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Property Value
Type Description
RepeatedFieldstring

ShuffleMode

public ShuffleMode ShuffleMode { get; set; }

Output only. The shuffle mode used for the job.

Property Value
Type Description
ShuffleMode

TempStoragePrefix

public string TempStoragePrefix { get; set; }

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

Google Cloud Storage:

storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Property Value
Type Description
string

UserAgent

public Struct UserAgent { get; set; }

A description of the process that generated the request.

Property Value
Type Description
Struct

Version

public Struct Version { get; set; }

A structure describing which components and their versions of the service are required in order to run the job.

Property Value
Type Description
Struct

WorkerPools

public RepeatedField<WorkerPool> WorkerPools { get; }

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Property Value
Type Description
RepeatedFieldWorkerPool

WorkerRegion

public string WorkerRegion { get; set; }

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Property Value
Type Description
string

WorkerZone

public string WorkerZone { get; set; }

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Property Value
Type Description
string