Update a Dataproc cluster by using a template
This page shows you how to use an Google APIs Explorer template to update a Dataproc cluster to change the number of workers in a cluster. Scaling a cluster up to include more workers is a common task when additional workers are needed to process larger jobs.
For other ways to update a Dataproc cluster, see:
Before you beginThis quickstart assumes you have already created a Dataproc cluster. You can use the APIs Explorer, the Google Cloud console, the gcloud CLI gcloud command-line tool, or the Quickstarts using Cloud Client Libraries to create a cluster.
Update a cluster
To update the number of workers in your cluster, fill in and execute the Google APIs Explorer Try this API template.
- Insert your projectId.
- Specify the region where your cluster is located (confirm or replace "us-central1"). Your cluster's region is listed on the Dataproc Clusters page in the Google Cloud console.
- Specify the clusterName of the existing cluster that you are updating (confirm or replace "example-cluster").
"config.worker_config.num_instances". This is the JSON PATH
relative to the Cluster
resource to the
numInstancesparameter to be updated (see the Request body instructions).
- config.workerConfig.numInstances: ("3": the new number of workers). You can change this value to add fewer or more workers. For example, if your standard cluster has the default number of 2 workers, specifying "3" will add 1 worker; specifying "4 will add 2). A standard Dataproc cluster must have at least 2 workers.
Click EXECUTE. The first time you run the API template, you may be asked to choose and sign into your Google account, then authorize the Google APIs Explorer to access your account. If the request is successful, the JSON response shows that cluster update is pending.
To confirm that the number of workers in the cluster has been updated, open the Dataproc Clusters page in the Google Cloud console and view the cluster's Total worker nodes column. You may need to click REFRESH at the top of the page to view the updated value after the cluster update completes.
To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.
You can use this quickstart template to restore the cluster to its previous
Learn how to write and run a Spark Scala job.