Committed use discounts

Committed use discounts (CUDs) for Dataflow streaming jobs provide discounted prices in exchange for your commitment to continuously use a certain amount of Dataflow compute resources for a year or longer.

Dataflow CUDs are ideal when your spending on Dataflow compute capacity for streaming jobs involves a predictable minimum that you can commit to for at least a year.

Dataflow CUD pricing

Dataflow CUDs offer two levels of discounts, depending on the commitment period:

  • A one-year CUD gives you a 20% discount from the on-demand rate.
  • A three-year CUD gives you a 40% discount from the on-demand rate.

When you purchase a commitment, you choose a one- or three-year period, as well as a commitment amount: your expected Dataflow expenditure on compute capacity per hour over that period. This commitment amount, after the application of your discount, becomes your commitment fee. You are then billed for this commitment fee on a monthly basis for the duration of the commitment period. For a full example, see the An example Dataflow CUD section of this document.

The discount applies to any eligible usage in Dataflow projects associated with the Cloud Billing account used to purchase the commitment, regardless of instance configuration or region. All CUDs apply to both regional and multi-region configurations.

Any expenditure beyond the commitment is billed at the on-demand rate. As your Dataflow usage grows, you can purchase additional commitments to receive discounts on increased expenditures not covered by previous commitments.

If the on-demand rates for Dataflow change after you purchase a commitment, your commitment fee doesn't change. You receive the same discount percentage on applicable usage.

Resources eligible for Dataflow CUDs

Dataflow committed use discounts automatically apply to your spending on the Dataflow compute capacity used by streaming jobs across projects. This flexibility helps you achieve a high utilization rate of your commitment across regions and projects without manual intervention, saving you time and money. Dataflow CUDs apply to your spending on the following resources:

  • Worker CPU and memory for streaming jobs
  • Streaming Engine data processed
  • Streaming Engine compute units
  • Data Compute Units (DCUs) for streaming jobs

Dataflow CUDs don't apply to your spending on the following resources:

  • Worker CPU and memory for batch and FlexRS jobs
  • Dataflow Shuffle data processed
  • Data Compute Units (DCUs) for batch jobs
  • Persistent Disk storage
  • GPUs
  • Snapshots
  • Confidential VMs

Purchase a Dataflow commitment

To purchase or manage Dataflow committed use discounts for your Cloud Billing account, you must have a Billing Account Administrator role with that account.

You can purchase a Dataflow CUD in the Google Cloud console Commitments page. Select your Cloud Billing account, then click Purchase. For more details, read the Purchasing spend-based commitments section in the Google Cloud documentation.

After purchasing a commitment, the commitment goes into effect within the next hour. Its discounts are automatically applied to subsequent eligible usage.

After you purchase a commitment, you can't cancel it. Make sure the size and duration of your commitment aligns with both your historical and your expected minimum expenditure on Dataflow compute capacity for streaming jobs.

In addition, before you purchase a commitment, read the Service Specific Terms regarding Committed Units.

An example Dataflow CUD scenario

Ideally, your commitment represents at least your expected minimum hourly expenditure on Dataflow streaming jobs across your projects over the next one or three years.

As an example, say that you run Dataflow streaming jobs in two different regions: us-central1 and us-west2.

The streaming job in us-central1 uses the following resources:

  • 10 nodes of instance type n1-standard-1 (vCPUs: 1, RAM: 3.75 GB)
  • 20 Streaming Engine Compute Units per hour

The streaming job in us-west2 uses the following resources:

  • 20 nodes of instance type n1-standard-1 (vCPUs: 1, RAM: 3.75 GB)
  • 40 Streaming Engine Compute Units per hour

From the pricing page, you can calculate the approximate hourly commitment cost:

  • Total expenditure in us-central1 = $2.6034 per hour
    • 10 nodes * 1 streaming vCPU per node * $0.069 per streaming vCPU per hour = $0.69 per hour
    • 10 nodes * 3.75GB per node * $0.003557 per GB per hour = $0.1334 per hour
    • 20 Streaming Engine Compute Units * $0.089 per compute unit per hour = $1.78 per hour
  • Total expenditure in us-west2= $3.128 per hour
    • 10 nodes * 1 streaming vCPU per node * $0.0828 per streaming vCPU per hour = $0.828 per hour
    • 10 nodes * 3.75GB per node * $0.0042684 per GB per hour = $0.16 per hour
    • 20 Streaming Engine Compute Units * $0.107 per compute unit per hour = $2.14 per hour
  • Total expenditure across all regions = $5.7314 per hour

If you expect to spend that minimum of $5.73 per hour continuously for the next year or more, then you can make a commitment for that amount. When purchasing the commitment, you enter $5.73 as the hourly on-demand commitment amount.

If you expect to scale down your clusters sometimes, you can make a commitment for a lower amount. Any expenditure above the commitment amount is charged at the on-demand rate.

Continuing this example, assume that you decide on a commitment of $5.73 per hour. As your next step, you choose the length of the commitment period.

As a basis for comparison, calculate the on-demand cost of Dataflow streaming job resource usage at the chosen commitment rate, without the application of any commitment discounts:

  • Monthly cost based on on-demand pricing: $5.73 per hour * 730 hours = $4,182.9 per month.

From here, you can calculate the monthly costs and savings that you would see under a one-year commitment with a 20% discount compared to a year of paying the full rates:

  • Monthly cost of a one-year, $5.73/hour commitment: ($5.73 per hour - 20% discount) * 730 hours = $3,346.32 per month
  • Total savings per month: $4,182.90 - $3,346.32 = $836.58
  • Total savings with a one-year, $5.73/hour commitment: $836.58 per month * 12 months = $10,038.96

You can apply similar math to calculating the costs and savings of a three-year CUD, with its 40% discount compared to on-demand rates:

  • Monthly cost of a three-year, $5.73/hour commitment: ($5.73 per hour - 40% discount) * 730 hours = $2,509.74 per month
  • Total savings per month: $4,182.90 - $2,509.74 = $1,673.16
  • Total savings with a three-year, $5.73/hour CUD: $1,673.16 per month * 36 months = $60,233.76

A commitment that covers your expected minimum Dataflow streaming usage over the years to come can lead to significant savings.

Recommendations for choosing a commitment

When you consider the purchase of Dataflow CUDs for streaming jobs, consider these factors:

  • Region: Dataflow CUDs apply to all streaming jobs in a project across regions. They apply to both regional and multi-region configurations. If you have Dataflow streaming jobs in multiple regions, calculate the expenditure across all of the regions that your projects use when making a commitment.
  • Projects: Determine the consistent baseline expenditure per project when calculating total commitment. Consider that production streaming jobs usually run 100% of the time, whereas development or staging jobs might run intermittently.
  • Jobs: If your Dataflow streaming jobs use a steady number of resources around the clock, consider purchasing a commitment that covers all of your resource usage. However, if you frequently scale your resources up or down, either manually or by using autoscaling, consider purchasing commitments only for your baseline predictable usage.

If you have jobs that you run only for bursts or brief durations, exclude them from your calculations.

Your commitment fee applies to every hour during the term of the commitment, regardless of actual usage. Choose your commitment amount based on both your historical resource usage for Dataflow streaming jobs and your future expectations. As long as your use of Dataflow compute resources for streaming jobs stays above your committed expenditure level, you receive the maximum possible discount for the length of the commitment.

What's next