IMPORTANT: The pricing model detailed in this section took effect on January 9, 2017. Dataflow jobs that were executed prior to January 9 were billed according to the previous Cloud Dataflow pricing model.

Usage of the Cloud Dataflow service is billed per minute on a per job basis. Each Dataflow job will use at least one Cloud Dataflow worker. The Cloud Dataflow service provides two worker types: batch and streaming. There are separate service charges for batch and streaming mode.

Dataflow workers 1 will consume the following resources, each billed on a per minute basis: vCPU, memory, and local storage. Users may override the default worker count 2 for a job and override the default resource settings allocated to each worker.

In addition to Dataflow worker resource usage, a Dataflow job might consume the following resources, each billed at their own pricing, including but not limited to:

You can view the total vCPU, memory, and local storage generated by a job either in the Google Cloud Platform Console or via the gcloud command line tool.

Future releases of Cloud Dataflow may have different service charges and or bundling of related services.

Iowa Oregon Northern Virginia South Carolina São Paulo Belgium London Frankfurt Mumbai Singapore Sydney Taiwan Tokyo
Dataflow Worker Type vCPU
(per Hour)
(per GB per Hour)
Local storage - Persistent Disk
(per GB per Hour)
Local storage - SSD based
(per GB per Hour)
Dataflow Shuffle (per GB per Hour) 5
Batch 3
Streaming 4

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

1 Batch and streaming workers are specialized resources which utilize Google Compute Engine. However, a Dataflow job will not emit Google Compute Engine billing for Compute Engine resources managed by the Dataflow service.

2 When using auto-scaling mode users specify the maxWorker count allocated to a job. Workers and respective resources will be added and removed automatically based on auto-scaling actuation.

3 Batch worker defaults: 1 vCPU, 3.75GB memory, 250GB PD.

4 Streaming worker defaults: 4 vCPU, 15GB memory, 420GB PD.

5 Service-based Dataflow Shuffle is currently available in beta for batch pipelines in the us-central1 (Iowa) and europe-west1 (Belgium) regions only. It will become available in other regions in the future.

Send feedback about...

Cloud Dataflow Documentation