Troubleshoot Dataproc error messages

This page lists Dataproc error messages, and their common causes and solutions.

For general guidance, see

Cluster creation error messages

  • Operation timed out: Only 0 out of 2 minimum required datanodes/node managers running.

    Cause: The master node is unable to create the cluster because it cannot communicate with worker nodes.

    Solution:

  • Required 'compute.subnetworks.use' permission for 'projects/{projectId}/regions/{region}/subnetworks/{subnetwork}

    Cause: This error can occur when you attempt to setup a Dataproc cluster using a VPC network in another project and the Dataproc Service Agent service account does not have the necessary permissions on the shared VPC project that is hosting the network.

    Solution: Follow the steps listed in Create a cluster that uses a VPC network in another project.

  • The zone 'projects/zones/{zone}' does not have enough resources available to fulfill the request '(resource type:compute)'

    Cause: The zone being used to create the cluster does not have sufficient resources.

    Solution:

  • Quota Exceeded errors

    Insufficient CPUS/CPUS_ALL_REGIONS quota
    Insufficient 'DISKS_TOTAL_GB' quota
    Insufficient 'IN_USE_ADDRESSES' quota

    Cause: Your CPU, disk, or IP address request exceeds your available quota.

    Solution: Request additional quota from the Cloud Console.

  • Initialization action failed

    Cause: The initialization action provided during cluster creation failed to install.

    Solution:

  • Failed to initialize node {cluster-name}: {component}

    Cause: A Dataproc component failed to initialize.

    Solution: Refer to: