Google Cloud Dataproc V1 Client - Class ClusterConfig (3.4.0)

Reference documentation and code samples for the Google Cloud Dataproc V1 Client class ClusterConfig.

The cluster config.

Generated from protobuf message google.cloud.dataproc.v1.ClusterConfig

Methods

__construct

Constructor.

Parameters
NameDescription
data array

Optional. Data for populating the Message object.

↳ config_bucket string

Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

↳ temp_bucket string

Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

↳ gce_cluster_config Google\Cloud\Dataproc\V1\GceClusterConfig

Optional. The shared Compute Engine config settings for all instances in a cluster.

↳ master_config Google\Cloud\Dataproc\V1\InstanceGroupConfig

Optional. The Compute Engine config settings for the cluster's master instance.

↳ worker_config Google\Cloud\Dataproc\V1\InstanceGroupConfig

Optional. The Compute Engine config settings for the cluster's worker instances.

↳ secondary_worker_config Google\Cloud\Dataproc\V1\InstanceGroupConfig

Optional. The Compute Engine config settings for a cluster's secondary worker instances

↳ software_config Google\Cloud\Dataproc\V1\SoftwareConfig

Optional. The config settings for cluster software.

↳ initialization_actions array<Google\Cloud\Dataproc\V1\NodeInitializationAction>

Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor:Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "${ROLE}" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi

↳ encryption_config Google\Cloud\Dataproc\V1\EncryptionConfig

Optional. Encryption settings for the cluster.

↳ autoscaling_config Google\Cloud\Dataproc\V1\AutoscalingConfig

Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.

↳ security_config Google\Cloud\Dataproc\V1\SecurityConfig

Optional. Security settings for the cluster.

↳ lifecycle_config Google\Cloud\Dataproc\V1\LifecycleConfig

Optional. Lifecycle setting for the cluster.

↳ endpoint_config Google\Cloud\Dataproc\V1\EndpointConfig

Optional. Port/endpoint configuration for this cluster

↳ metastore_config Google\Cloud\Dataproc\V1\MetastoreConfig

Optional. Metastore configuration.

↳ dataproc_metric_config Google\Cloud\Dataproc\V1\DataprocMetricConfig

Optional. The config for Dataproc metrics.

↳ auxiliary_node_groups array<Google\Cloud\Dataproc\V1\AuxiliaryNodeGroup>

Optional. The node group settings.

getConfigBucket

Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output.

If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Returns
TypeDescription
string

setConfigBucket

Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output.

If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getTempBucket

Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets).

This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Returns
TypeDescription
string

setTempBucket

Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets).

This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.

Parameter
NameDescription
var string
Returns
TypeDescription
$this

getGceClusterConfig

Optional. The shared Compute Engine config settings for all instances in a cluster.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\GceClusterConfig|null

hasGceClusterConfig

clearGceClusterConfig

setGceClusterConfig

Optional. The shared Compute Engine config settings for all instances in a cluster.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\GceClusterConfig
Returns
TypeDescription
$this

getMasterConfig

Optional. The Compute Engine config settings for the cluster's master instance.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null

hasMasterConfig

clearMasterConfig

setMasterConfig

Optional. The Compute Engine config settings for the cluster's master instance.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\InstanceGroupConfig
Returns
TypeDescription
$this

getWorkerConfig

Optional. The Compute Engine config settings for the cluster's worker instances.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null

hasWorkerConfig

clearWorkerConfig

setWorkerConfig

Optional. The Compute Engine config settings for the cluster's worker instances.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\InstanceGroupConfig
Returns
TypeDescription
$this

getSecondaryWorkerConfig

Optional. The Compute Engine config settings for a cluster's secondary worker instances

Returns
TypeDescription
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null

hasSecondaryWorkerConfig

clearSecondaryWorkerConfig

setSecondaryWorkerConfig

Optional. The Compute Engine config settings for a cluster's secondary worker instances

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\InstanceGroupConfig
Returns
TypeDescription
$this

getSoftwareConfig

Optional. The config settings for cluster software.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\SoftwareConfig|null

hasSoftwareConfig

clearSoftwareConfig

setSoftwareConfig

Optional. The config settings for cluster software.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\SoftwareConfig
Returns
TypeDescription
$this

getInitializationActions

Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes.

You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor:Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "${ROLE}" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi

Returns
TypeDescription
Google\Protobuf\Internal\RepeatedField

setInitializationActions

Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes.

You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor:Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "${ROLE}" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi

Parameter
NameDescription
var array<Google\Cloud\Dataproc\V1\NodeInitializationAction>
Returns
TypeDescription
$this

getEncryptionConfig

Optional. Encryption settings for the cluster.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\EncryptionConfig|null

hasEncryptionConfig

clearEncryptionConfig

setEncryptionConfig

Optional. Encryption settings for the cluster.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\EncryptionConfig
Returns
TypeDescription
$this

getAutoscalingConfig

Optional. Autoscaling config for the policy associated with the cluster.

Cluster does not autoscale if this field is unset.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\AutoscalingConfig|null

hasAutoscalingConfig

clearAutoscalingConfig

setAutoscalingConfig

Optional. Autoscaling config for the policy associated with the cluster.

Cluster does not autoscale if this field is unset.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\AutoscalingConfig
Returns
TypeDescription
$this

getSecurityConfig

Optional. Security settings for the cluster.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\SecurityConfig|null

hasSecurityConfig

clearSecurityConfig

setSecurityConfig

Optional. Security settings for the cluster.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\SecurityConfig
Returns
TypeDescription
$this

getLifecycleConfig

Optional. Lifecycle setting for the cluster.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\LifecycleConfig|null

hasLifecycleConfig

clearLifecycleConfig

setLifecycleConfig

Optional. Lifecycle setting for the cluster.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\LifecycleConfig
Returns
TypeDescription
$this

getEndpointConfig

Optional. Port/endpoint configuration for this cluster

Returns
TypeDescription
Google\Cloud\Dataproc\V1\EndpointConfig|null

hasEndpointConfig

clearEndpointConfig

setEndpointConfig

Optional. Port/endpoint configuration for this cluster

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\EndpointConfig
Returns
TypeDescription
$this

getMetastoreConfig

Optional. Metastore configuration.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\MetastoreConfig|null

hasMetastoreConfig

clearMetastoreConfig

setMetastoreConfig

Optional. Metastore configuration.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\MetastoreConfig
Returns
TypeDescription
$this

getDataprocMetricConfig

Optional. The config for Dataproc metrics.

Returns
TypeDescription
Google\Cloud\Dataproc\V1\DataprocMetricConfig|null

hasDataprocMetricConfig

clearDataprocMetricConfig

setDataprocMetricConfig

Optional. The config for Dataproc metrics.

Parameter
NameDescription
var Google\Cloud\Dataproc\V1\DataprocMetricConfig
Returns
TypeDescription
$this

getAuxiliaryNodeGroups

Optional. The node group settings.

Returns
TypeDescription
Google\Protobuf\Internal\RepeatedField

setAuxiliaryNodeGroups

Optional. The node group settings.

Parameter
NameDescription
var array<Google\Cloud\Dataproc\V1\AuxiliaryNodeGroup>
Returns
TypeDescription
$this