Dataproc Serverless pricing
Dataproc Serverless for Spark pricing is based on the number of Data Compute Units (DCUs) and the amount of shuffle storage used. Both DCUs and shuffle storage are billed per second, with a 1 minute minimum charge.
A DCU is defined as 1 Dataproc vCPU plus 4GB RAM. Each Dataproc Serverless for Spark batch workload consumes a minimum of 12 DCUs (4 DCUs for driver node and 8 DCUs for 2 executor nodes) for the duration of the batch workload. No additional Compute Engine VM or PD charges apply.
Data Compute Unit (DCU) pricing
The DCU rate shown below is an hourly rate. It is prorated and billed per
second, with a 1 minute minimum charge.
Shuffle storage pricing
The shuffle storage rate shown below is a monthly rate. It is prorated and billed per second, with a 1 minute minimum charge.
If the Dataproc Serverless for Spark batch workload runs with 8 DCUs for 24 hours in the us-central1 region and consumes 25GB of shuffle storage, the price calculation is as follows:
Total compute cost = 8 * 24 * $0.060000 = $11.52 Total storage cost = 25 * ($0.040/301) = $0.03 ------------------------------------------------ Total cost = $11.52 + $0.03 = $11.55 Notes: 1. The example assumes a 30-day month. Since the batch workload duration is one day, the monthly shuffle storage rate is divided by 30.
Use of other Google Cloud resources
Your Dataproc Serverless for Spark workload can optionally utilize the following resources, each billed at its own pricing, including but not limited to:
- Read the Dataproc Serverless documentation.
- Get started with Dataproc Serverless.
- Try the Pricing calculator.