Reference documentation and code samples for the Google Cloud Dataproc V1 Client class ClusterConfig.
The cluster config.
Generated from protobuf message google.cloud.dataproc.v1.ClusterConfig
Namespace
Google \ Cloud \ Dataproc \ V1Methods
__construct
Constructor.
Parameters | |
---|---|
Name | Description |
data |
array
Optional. Data for populating the Message object. |
↳ config_bucket |
string
Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a |
↳ temp_bucket |
string
Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a |
↳ gce_cluster_config |
Google\Cloud\Dataproc\V1\GceClusterConfig
Optional. The shared Compute Engine config settings for all instances in a cluster. |
↳ master_config |
Google\Cloud\Dataproc\V1\InstanceGroupConfig
Optional. The Compute Engine config settings for the cluster's master instance. |
↳ worker_config |
Google\Cloud\Dataproc\V1\InstanceGroupConfig
Optional. The Compute Engine config settings for the cluster's worker instances. |
↳ secondary_worker_config |
Google\Cloud\Dataproc\V1\InstanceGroupConfig
Optional. The Compute Engine config settings for a cluster's secondary worker instances |
↳ software_config |
Google\Cloud\Dataproc\V1\SoftwareConfig
Optional. The config settings for cluster software. |
↳ initialization_actions |
array<Google\Cloud\Dataproc\V1\NodeInitializationAction>
Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's |
↳ encryption_config |
Google\Cloud\Dataproc\V1\EncryptionConfig
Optional. Encryption settings for the cluster. |
↳ autoscaling_config |
Google\Cloud\Dataproc\V1\AutoscalingConfig
Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset. |
↳ security_config |
Google\Cloud\Dataproc\V1\SecurityConfig
Optional. Security settings for the cluster. |
↳ lifecycle_config |
Google\Cloud\Dataproc\V1\LifecycleConfig
Optional. Lifecycle setting for the cluster. |
↳ endpoint_config |
Google\Cloud\Dataproc\V1\EndpointConfig
Optional. Port/endpoint configuration for this cluster |
↳ metastore_config |
Google\Cloud\Dataproc\V1\MetastoreConfig
Optional. Metastore configuration. |
↳ dataproc_metric_config |
Google\Cloud\Dataproc\V1\DataprocMetricConfig
Optional. The config for Dataproc metrics. |
↳ auxiliary_node_groups |
array<Google\Cloud\Dataproc\V1\AuxiliaryNodeGroup>
Optional. The node group settings. |
getConfigBucket
Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output.
If you do not specify a staging bucket, Cloud
Dataproc will determine a Cloud Storage location (US,
ASIA, or EU) for your cluster's staging bucket according to the
Compute Engine zone where your cluster is deployed, and then create
and manage this project-level, per-location bucket (see
Dataproc staging and temp
buckets).
This field requires a Cloud Storage bucket name, not a gs://...
URI to
a Cloud Storage bucket.
Returns | |
---|---|
Type | Description |
string |
setConfigBucket
Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output.
If you do not specify a staging bucket, Cloud
Dataproc will determine a Cloud Storage location (US,
ASIA, or EU) for your cluster's staging bucket according to the
Compute Engine zone where your cluster is deployed, and then create
and manage this project-level, per-location bucket (see
Dataproc staging and temp
buckets).
This field requires a Cloud Storage bucket name, not a gs://...
URI to
a Cloud Storage bucket.
Parameter | |
---|---|
Name | Description |
var |
string
|
Returns | |
---|---|
Type | Description |
$this |
getTempBucket
Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets).
This field requires a Cloud Storage bucket name, not a gs://...
URI to
a Cloud Storage bucket.
Returns | |
---|---|
Type | Description |
string |
setTempBucket
Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets).
This field requires a Cloud Storage bucket name, not a gs://...
URI to
a Cloud Storage bucket.
Parameter | |
---|---|
Name | Description |
var |
string
|
Returns | |
---|---|
Type | Description |
$this |
getGceClusterConfig
Optional. The shared Compute Engine config settings for all instances in a cluster.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\GceClusterConfig|null |
hasGceClusterConfig
clearGceClusterConfig
setGceClusterConfig
Optional. The shared Compute Engine config settings for all instances in a cluster.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\GceClusterConfig
|
Returns | |
---|---|
Type | Description |
$this |
getMasterConfig
Optional. The Compute Engine config settings for the cluster's master instance.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null |
hasMasterConfig
clearMasterConfig
setMasterConfig
Optional. The Compute Engine config settings for the cluster's master instance.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\InstanceGroupConfig
|
Returns | |
---|---|
Type | Description |
$this |
getWorkerConfig
Optional. The Compute Engine config settings for the cluster's worker instances.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null |
hasWorkerConfig
clearWorkerConfig
setWorkerConfig
Optional. The Compute Engine config settings for the cluster's worker instances.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\InstanceGroupConfig
|
Returns | |
---|---|
Type | Description |
$this |
getSecondaryWorkerConfig
Optional. The Compute Engine config settings for a cluster's secondary worker instances
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\InstanceGroupConfig|null |
hasSecondaryWorkerConfig
clearSecondaryWorkerConfig
setSecondaryWorkerConfig
Optional. The Compute Engine config settings for a cluster's secondary worker instances
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\InstanceGroupConfig
|
Returns | |
---|---|
Type | Description |
$this |
getSoftwareConfig
Optional. The config settings for cluster software.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\SoftwareConfig|null |
hasSoftwareConfig
clearSoftwareConfig
setSoftwareConfig
Optional. The config settings for cluster software.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\SoftwareConfig
|
Returns | |
---|---|
Type | Description |
$this |
getInitializationActions
Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes.
You can test a node's role
metadata to run an executable on
a master or worker node, as shown below using curl
(you can also use
wget
):
ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
... master specific actions ...
else
... worker specific actions ...
fi
Returns | |
---|---|
Type | Description |
Google\Protobuf\Internal\RepeatedField |
setInitializationActions
Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes.
You can test a node's role
metadata to run an executable on
a master or worker node, as shown below using curl
(you can also use
wget
):
ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
... master specific actions ...
else
... worker specific actions ...
fi
Parameter | |
---|---|
Name | Description |
var |
array<Google\Cloud\Dataproc\V1\NodeInitializationAction>
|
Returns | |
---|---|
Type | Description |
$this |
getEncryptionConfig
Optional. Encryption settings for the cluster.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\EncryptionConfig|null |
hasEncryptionConfig
clearEncryptionConfig
setEncryptionConfig
Optional. Encryption settings for the cluster.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\EncryptionConfig
|
Returns | |
---|---|
Type | Description |
$this |
getAutoscalingConfig
Optional. Autoscaling config for the policy associated with the cluster.
Cluster does not autoscale if this field is unset.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\AutoscalingConfig|null |
hasAutoscalingConfig
clearAutoscalingConfig
setAutoscalingConfig
Optional. Autoscaling config for the policy associated with the cluster.
Cluster does not autoscale if this field is unset.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\AutoscalingConfig
|
Returns | |
---|---|
Type | Description |
$this |
getSecurityConfig
Optional. Security settings for the cluster.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\SecurityConfig|null |
hasSecurityConfig
clearSecurityConfig
setSecurityConfig
Optional. Security settings for the cluster.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\SecurityConfig
|
Returns | |
---|---|
Type | Description |
$this |
getLifecycleConfig
Optional. Lifecycle setting for the cluster.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\LifecycleConfig|null |
hasLifecycleConfig
clearLifecycleConfig
setLifecycleConfig
Optional. Lifecycle setting for the cluster.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\LifecycleConfig
|
Returns | |
---|---|
Type | Description |
$this |
getEndpointConfig
Optional. Port/endpoint configuration for this cluster
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\EndpointConfig|null |
hasEndpointConfig
clearEndpointConfig
setEndpointConfig
Optional. Port/endpoint configuration for this cluster
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\EndpointConfig
|
Returns | |
---|---|
Type | Description |
$this |
getMetastoreConfig
Optional. Metastore configuration.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\MetastoreConfig|null |
hasMetastoreConfig
clearMetastoreConfig
setMetastoreConfig
Optional. Metastore configuration.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\MetastoreConfig
|
Returns | |
---|---|
Type | Description |
$this |
getDataprocMetricConfig
Optional. The config for Dataproc metrics.
Returns | |
---|---|
Type | Description |
Google\Cloud\Dataproc\V1\DataprocMetricConfig|null |
hasDataprocMetricConfig
clearDataprocMetricConfig
setDataprocMetricConfig
Optional. The config for Dataproc metrics.
Parameter | |
---|---|
Name | Description |
var |
Google\Cloud\Dataproc\V1\DataprocMetricConfig
|
Returns | |
---|---|
Type | Description |
$this |
getAuxiliaryNodeGroups
Optional. The node group settings.
Returns | |
---|---|
Type | Description |
Google\Protobuf\Internal\RepeatedField |
setAuxiliaryNodeGroups
Optional. The node group settings.
Parameter | |
---|---|
Name | Description |
var |
array<Google\Cloud\Dataproc\V1\AuxiliaryNodeGroup>
|
Returns | |
---|---|
Type | Description |
$this |