Save up to 40 percent with Dataflow streaming committed use discounts

Dataflow is an industry-leading data processing platform that provides unified batch and streaming capabilities for a wide variety of analytics and machine learning use cases: real-time patient monitoring, fraud prevention and real-time inventory management. It’s a fully managed service that comes with flexible development options like pre-built templates, notebooks, and SDKs for Java, Python and Go, and delivers a rich set of built-in management tools that give data engineers choice and flexibility. Dataflow integrates with Google Cloud products like Pub/Sub, BigQuery, Vertex AI, Cloud Storage, Spanner, and BigTable. It also integrates with open-source technologies like Kafka and JDBC, as well as third-party services like AWS S3 and Snowflake, to best meet your analytics and machine learning needs.

As streaming analytics and machine learning needs continue to grow, customers with predictable processing volumes want to better optimize their Dataflow costs. Today, we are announcing the general availability of Dataflow streaming committed use discounts (CUDs), providing a new way for you to save money on a key driver of your streaming costs: streaming compute. By committing to a baseline amount of Dataflow streaming compute usage for a one-year or three-year period, you can get deeper discounts: a 20% discount for a one-year commitment, and a 40% discount for a three-year commitment.

Dataflow streaming CUDs are spend-based commitments, and apply to the following Dataflow resources across all projects or regions that are associated with a single Cloud Billing account: 

• Worker CPU and memory for streaming jobs

• Streaming engine usage

• Data compute units (DCUs) for Dataflow Prime streaming jobs

Dataflow streaming CUDs are available for purchase from the Google Cloud console.

How to save money with Dataflow Streaming CUDs

To illustrate how Dataflow streaming CUDs can help you save money, let's look at an example. Let’s assume a Dataflow streaming job is running in us-central1 (Iowa). The streaming job in our example is using the following resources:

• 10 nodes of instance type n1-standard-1 (vCPUs: 1, RAM: 3.75 GB)

• 20 streaming engine compute units per hour

From the Dataflow pricing page, you can calculate the approximate hourly cost of your job to be $2.6034:

• 10 nodes * 1 streaming vCPU per node * $0.069 per streaming vCPU per hour = $0.69 per hour

• 10 nodes * 3.75GB per node * $0.003557 per GB per hour = $0.1334 per hour

• 20 streaming engine compute units * $0.089 per compute unit per hour = $1.78 per hour

(Please note that the above prices are examples. For current prices, see Dataflow pricing.)

If you purchase a one-year CUD for the same job, you will get a 20% discount. This means that the cost of the job will be reduced from $2.6034 to $2.0827 per hour. Over the course of a year, Dataflow streaming CUDs will help you save $4,561.33.

If you purchase a three-year CUD for the job, you will get a 40% discount. This means that the cost of the job will be reduced from $2.6034 to $1.562 per hour. Dataflow streaming CUDs will help you save $9122.31 annually, or $27,366.99 over the course of three years

When to use Dataflow streaming CUDs

Dataflow streaming CUDs are ideal for workloads with predictable resource needs like personalized product recommendations, predictive maintenance, and smart supply chains. As streaming jobs are expected to be used quite consistently, you can purchase Dataflow streaming CUDs to better manage the cost of your streaming jobs.

Dataflow streaming CUDs are also a good choice for workloads that are growing steadily, since bringing more workloads to Dataflow streaming increases the value that existing workloads can get from Dataflow streaming CUDs. For example, if you expect your streaming usage to grow by 30% each year, you can purchase a three-year Dataflow streaming CUD at your current usage levels to lock in a 40% discount. As your workload grows, you can purchase additional Dataflow streaming CUDs to maintain or grow your CUD utilization rate, and cover even more of your Dataflow streaming spend with attractive discounts.

How to purchase Dataflow streaming CUDs

To purchase a Dataflow streaming CUD, go to the committed use discounts page in the Cloud console. You can make your commitment based on the current usage patterns and projected growth of the streaming jobs that you would like to cover with your commitment. Once you choose your commitment period, you will see the cost of your commitment after the discounts and how much you’re saving. It’s that easy!

Take the next step

Dataflow provides a versatile, scalable and cost-effective platform to run your streaming workloads. With Dataflow streaming CUDs, you can save even more on your Dataflow costs! Check out our documentation and pricing page for more details on Dataflow streaming CUDs.