Problem
The Dataproc Activity log contains the initial creation requests and the associated final logs of the operation. They can also be seen in Cloud Logging with this filter:
logName:"/logs/cloudaudit.googleapis.com%2Factivity" resource.type="cloud_dataproc_cluster" protoPayload.methodName:"CreateCluster"
The creation failed with an error message like:
"The zone 'projects/<PROJECT>/zones/<ZONE>' does not have enough resources available to fulfill the request. '(resource type:compute)'."
The initial log message for the failed creation is either directly visible as the previous log messages or can be found be further restricting the logging filter by the cluster UUID, for example:
resource.labels.cluster_uuid="<CLUSTER-UUID>"
The first operation log message shows that a zone was specified, thus the Dataproc Auto Zone feature was not used.
Environment
- Cloud Logging
- Cloud Dataproc
Solution
- Switch to a different zone or use the Dataproc Auto Zone feature.
Cause
Compute resources in Google's Data centers are limited and there is no guarantee that your request can be satisfied within a specific zone. However for normally sized requests it should be possible to find available resources in one zone of a region.