Google Cloud Dataproc V1 Client - Class ClusterConfig (3.7.1)

Version 3.7.1

• 3.14.0 (latest)
• 3.13.4
• 3.12.0
• 3.11.0
• 3.10.1
• 3.9.0
• 3.8.1
• 3.7.1
• 3.6.1
• 3.5.1
• 3.4.0
• 3.3.0
• 3.2.2
• 2.2.0
• 2.1.0
• 2.0.0

Reference documentation and code samples for the Google Cloud Dataproc V1 Client class ClusterConfig.

The cluster config.

Generated from protobuf message google.cloud.dataproc.v1.ClusterConfig

Namespace

Google \ Cloud \ Dataproc \ V1

Methods

__construct

Constructor.

Parameters
Name	Description
data	array Optional. Data for populating the Message object.
↳ config_bucket	string Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
↳ temp_bucket	string Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.
↳ gce_cluster_config	Google\Cloud\Dataproc\V1\GceClusterConfig Optional. The shared Compute Engine config settings for all instances in a cluster.
↳ master_config	Google\Cloud\Dataproc\V1\InstanceGroupConfig Optional. The Compute Engine config settings for the cluster's master instance.
↳ worker_config	Google\Cloud\Dataproc\V1\InstanceGroupConfig Optional. The Compute Engine config settings for the cluster's worker instances.
↳ secondary_worker_config	Google\Cloud\Dataproc\V1\InstanceGroupConfig Optional. The Compute Engine config settings for a cluster's secondary worker instances
↳ software_config	Google\Cloud\Dataproc\V1\SoftwareConfig Optional. The config settings for cluster software.
↳ initialization_actions	array<Google\Cloud\Dataproc\V1\NodeInitializationAction> Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor:Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "${ROLE}" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi
↳ encryption_config	Google\Cloud\Dataproc\V1\EncryptionConfig Optional. Encryption settings for the cluster.
↳ autoscaling_config	Google\Cloud\Dataproc\V1\AutoscalingConfig Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.
↳ security_config	Google\Cloud\Dataproc\V1\SecurityConfig Optional. Security settings for the cluster.
↳ lifecycle_config	Google\Cloud\Dataproc\V1\LifecycleConfig Optional. Lifecycle setting for the cluster.
↳ endpoint_config	Google\Cloud\Dataproc\V1\EndpointConfig Optional. Port/endpoint configuration for this cluster
↳ metastore_config	Google\Cloud\Dataproc\V1\MetastoreConfig Optional. Metastore configuration.
↳ dataproc_metric_config	Google\Cloud\Dataproc\V1\DataprocMetricConfig Optional. The config for Dataproc metrics.
↳ auxiliary_node_groups	array<Google\Cloud\Dataproc\V1\AuxiliaryNodeGroup> Optional. The node group settings.

getConfigBucket

Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output.

If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Returns
Type	Description
string

setConfigBucket

Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output.

If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getTempBucket

Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets).

This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Returns
Type	Description
string

setTempBucket

Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets).

This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Parameter
Name	Description
var	string

Returns
Type	Description
$this

getGceClusterConfig

Optional. The shared Compute Engine config settings for all instances in a cluster.

Returns
Type	Description
Google\Cloud\Dataproc\V1\GceClusterConfig|null

hasGceClusterConfig

clearGceClusterConfig

setGceClusterConfig

Optional. The shared Compute Engine config settings for all instances in a cluster.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\GceClusterConfig

Returns
Type	Description
$this

getMasterConfig

Optional. The Compute Engine config settings for the cluster's master instance.

Returns
Type	Description
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null

hasMasterConfig

clearMasterConfig

setMasterConfig

Optional. The Compute Engine config settings for the cluster's master instance.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\InstanceGroupConfig

Returns
Type	Description
$this

getWorkerConfig

Optional. The Compute Engine config settings for the cluster's worker instances.

Returns
Type	Description
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null

hasWorkerConfig

clearWorkerConfig

setWorkerConfig

Optional. The Compute Engine config settings for the cluster's worker instances.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\InstanceGroupConfig

Returns
Type	Description
$this

getSecondaryWorkerConfig

Optional. The Compute Engine config settings for a cluster's secondary worker instances

Returns
Type	Description
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null

hasSecondaryWorkerConfig

clearSecondaryWorkerConfig

setSecondaryWorkerConfig

Optional. The Compute Engine config settings for a cluster's secondary worker instances

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\InstanceGroupConfig

Returns
Type	Description
$this

getSoftwareConfig

Optional. The config settings for cluster software.

Returns
Type	Description
Google\Cloud\Dataproc\V1\SoftwareConfig|null

hasSoftwareConfig

clearSoftwareConfig

setSoftwareConfig

Optional. The config settings for cluster software.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\SoftwareConfig

Returns
Type	Description
$this

getInitializationActions

Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes.

You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor:Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "${ROLE}" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setInitializationActions

Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes.

You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor:Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "${ROLE}" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi

Parameter
Name	Description
var	array<Google\Cloud\Dataproc\V1\NodeInitializationAction>

Returns
Type	Description
$this

getEncryptionConfig

Optional. Encryption settings for the cluster.

Returns
Type	Description
Google\Cloud\Dataproc\V1\EncryptionConfig|null

hasEncryptionConfig

clearEncryptionConfig

setEncryptionConfig

Optional. Encryption settings for the cluster.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\EncryptionConfig

Returns
Type	Description
$this

getAutoscalingConfig

Optional. Autoscaling config for the policy associated with the cluster.

Cluster does not autoscale if this field is unset.

Returns
Type	Description
Google\Cloud\Dataproc\V1\AutoscalingConfig|null

hasAutoscalingConfig

clearAutoscalingConfig

setAutoscalingConfig

Optional. Autoscaling config for the policy associated with the cluster.

Cluster does not autoscale if this field is unset.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\AutoscalingConfig

Returns
Type	Description
$this

getSecurityConfig

Optional. Security settings for the cluster.

Returns
Type	Description
Google\Cloud\Dataproc\V1\SecurityConfig|null

hasSecurityConfig

clearSecurityConfig

setSecurityConfig

Optional. Security settings for the cluster.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\SecurityConfig

Returns
Type	Description
$this

getLifecycleConfig

Optional. Lifecycle setting for the cluster.

Returns
Type	Description
Google\Cloud\Dataproc\V1\LifecycleConfig|null

hasLifecycleConfig

clearLifecycleConfig

setLifecycleConfig

Optional. Lifecycle setting for the cluster.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\LifecycleConfig

Returns
Type	Description
$this

getEndpointConfig

Optional. Port/endpoint configuration for this cluster

Returns
Type	Description
Google\Cloud\Dataproc\V1\EndpointConfig|null

hasEndpointConfig

clearEndpointConfig

setEndpointConfig

Optional. Port/endpoint configuration for this cluster

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\EndpointConfig

Returns
Type	Description
$this

getMetastoreConfig

Optional. Metastore configuration.

Returns
Type	Description
Google\Cloud\Dataproc\V1\MetastoreConfig|null

hasMetastoreConfig

clearMetastoreConfig

setMetastoreConfig

Optional. Metastore configuration.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\MetastoreConfig

Returns
Type	Description
$this

getDataprocMetricConfig

Optional. The config for Dataproc metrics.

Returns
Type	Description
Google\Cloud\Dataproc\V1\DataprocMetricConfig|null

hasDataprocMetricConfig

clearDataprocMetricConfig

setDataprocMetricConfig

Optional. The config for Dataproc metrics.

Parameter
Name	Description
var	Google\Cloud\Dataproc\V1\DataprocMetricConfig

Returns
Type	Description
$this

getAuxiliaryNodeGroups

Optional. The node group settings.

Returns
Type	Description
Google\Protobuf\Internal\RepeatedField

setAuxiliaryNodeGroups

Optional. The node group settings.

Parameter
Name	Description
var	array<Google\Cloud\Dataproc\V1\AuxiliaryNodeGroup>

Returns
Type	Description
$this