Dataproc Auto Zone placement

When you create a Dataproc cluster, you can use a single "global" endpoint or regional endpoints based on Compute Engine zones. If you choose a region, you can select a zone within that region, or you can omit the zone to have the Dataproc Auto Zone feature select a zone for you in the region you choose. Once a zone is selected, all nodes for that cluster will be deployed to that zone. The Auto Zone feature is not available when you choose the global endpoint or create a cluster using the Cloud Console.

Using Auto Zone placement

gcloud command

To create a Dataproc cluster that uses Auto Zone placement, use the gcloud dataproc clusters create command. Set the --region flag to a non-global region, and omit the --zone flag (or leave the flag empty: --zone= or zone="").

Note: If you ran the gcloud config set compute/zone command to set a default zone, you must supply the --zone= or zone="" flag to gcloud dataproc clusters create in order to enable the Auto Zone feature. Power Shell users must surround the empty quotes with single quotes: --zone='""'.

gcloud dataproc clusters create cluster-name \
    --region=region \
    --zone="" \
    other args ...

REST API

To create a Dataproc cluster that uses Auto Zone placement, construct a JSON clusters.create API request, leaving the gceClusterConfig.zoneUri field empty. In the REST endpoint, https://dataproc.googleapis.com/v1/projects/projectId/regions/region/clusters, insert a non-global region. Dataproc Auto Zone will choose a zone for the cluster within the specified region.

Use short resource names with Auto Zone placement: When specifying a resource URI, such as machineTypeUri or acceleratorTypeUri, in an Auto Zone placement REST API cluster creation request, use a short resource name without a zone specification, for example, "n1-standard-2" or "nvidia-tesla-k80".

Console

Auto Zone placement cannot be selected when creating a cluster using the Cloud Console.