Cloud Composer pricing

This document explains Cloud Composer pricing.

Pricing models

Cloud Composer uses the following pricing models:

Cloud Composer environments are billed in short time intervals, for the actual time period when it was running. For example, if you create an environment, run it for 6 hours and 30 minutes, and delete it afterwards, then the total costs are for 6.5 hours.

Although some pricing is stated in hours or by the month, Cloud Composer is still billed for the actual usage time.

Cloud Composer 3 pricing

This section describes pricing in Cloud Composer 3.

Pricing table for Cloud Composer 3

This section summarizes Cloud Composer 3 costs for different regions. Also see additional costs.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Cloud Composer 3 SKUs

Cloud Composer 3 has the following SKUs:

  • Cloud Composer 3 standard milli DCU-hours

    Data Compute Unit is an abstract metering unit that represents computational resources allocated by a Cloud Composer environment at a point in time. Because DCU is a measure of the currently provisioned resources of an environment, costs are charged for the environment's DCU value over time, DCU-hours.

    This SKU is measured in milliDCU-hours. For example, if your environment uses 1 DCU for 1 hour, this is equal to using 1000 milliDCU-hours.

    DCU-hours represent the usage of the following resources:

    • vCPU, Memory and Storage resources, used by the environment's components that run the workloads: Airflow workers, schedulers, DAG processors, triggerers, and the web server. Airflow workers are autoscaled, and as such the corresponding costs follow the changing number of workers in the environment.

      If you deploy your own workloads in your environment's cluster, such as when you create Pods using KubernetesPodOperator or execute tasks with Kubernetes Executor, then used resources also count as DCU-hours and follow Cloud Composer 3 pricing model.

    • Size of the environment. The environment size is the scale of the managed infrastructure of your Cloud Composer environment. Environment infrastructure comes in different sizes: Small, Medium, and Large.

      The cost of environment's size in DCUs covers the cost of resources used by infrastructure components required to run Airflow. For example, by the Cloud SQL instance (with a separate SKU covering the costs of storage) or by the Redis queue of your environment.

  • Cloud Composer 3 database storage

    This SKU covers the cost of the storage used by the Cloud SQL instance that stores the Airflow database of your environment.

    The minimum disk size of the Cloud SQL instance is 10 GiB.

    The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage and does not decrease if the size of the Airflow database is reduced later.

  • Cloud Composer Network Data Transfer

    This group of SKUs covers the billable outbound traffic generated by the environment's components that run the workloads in your environment.

    Cloud Composer network data transfer SKUs cover the following types of traffic generated by workloads runnng in your environment:

    For more information about network egress billing and descriptions of the related SKUs, see Network Pricing.

  • Your environment also has additional costs that are not a part of Cloud Composer pricing.

Additional costs for Cloud Composer 3

Costs for the following services are billed in addition to costs for Cloud Composer 3 environments:

Pricing example for Cloud Composer 3

For example, you create a Cloud Composer 3 environment in Iowa (us-central1), use it for 10 days to run your workloads, and then delete it. The environment operates during the a month that has 30 days.

As an example, your environment uses the following amount of resources:

  • For 5 days it uses 12 DCUs. These DCUs cover the combined costs of vCPU, memory, and storage resources used by the environment's components ( schedulers, DAG processors, triggeres, workers, the web server), and the costs of operating the environment's infrastructure.
  • For other 5 days your environment uses 15 DCUs because the number of workers was scaled up to address the large amount of tasks.
  • Your environment's database does not go over the initial size of 10 GiB.
  • The environment does not generate any billable network egress traffic because it operates within the same zone and does not send data over the internet. In this case all Cloud Composer Network Data Transfer SKUs do not generate costs.

In this case, the breakdown of costs is the following:

SKU First period Second period Cost in Iowa (us-central1) Total cost (10 days)
DCU-hours
(or 1000 milliDCU-hours)
1440 DCU-hours
(12 DCUs for 120 hours)
1800 DCU-hours
(15 DCUs for 120 hours)
$0.06 $194.4
(3240 DCU-hours)
Database storage 1.67 GiBy/Mo
(10 GiBy for 5 days out of 30 days)
1.67 GiBy/Mo
(10 GiBy for 5 days out of 30 days)
$0.17 $0.5678
(10 GiBy for 10 days)

Additional considerations:

  • Your environment is auto-scaling. This means that if the actual load is lower during the described period, then the costs are also lower. For example, if you configure your environment to scale between 1 and 6 workers and your environment uses only a single worker during the whole period, then you pay for DCU-hours used by 1 worker, not by 6 workers.

  • If your environment transfers data between regions or over the internet, then Cloud Composer Network Data Transfer SKUs apply.

  • Your environment also has additional costs that are not a part of Cloud Composer 3 SKUs.

Pricing model transition for Cloud Composer 3

This section describes how the pricing model changes in Cloud Composer 3 compared to Cloud Composer 2.

Cloud Composer 2 SKUs are displayed alongside Cloud Composer 3 SKUs
Figure 1. Pricing model transition from Cloud Composer 2 to Cloud Composer 3 (click to enlarge)

Cloud Composer 3 follows the path of further simplification and hides most of the remaining infrastructure components including the cluster of your environment and dependencies on other services such as Artifact Registry, Cloud Build, or Pub/Sub.

The overall costs of an Cloud Composer environment are now combined into a fewer number of SKUs:

  • SKUs that represented Compute Engine resources used by your environment's components are combined into the DCU-hours SKU.

  • SKUs for the environment's size are included into the DCU-hours SKU.

  • Cloud Composer 3 Network Egress SKUs replace corresponding Cloud Composer 2 additional costs related to network traffic usage by the Airflow components that previously ran in your project.

  • You do not pay additional costs for Google services that were utilized in Cloud Composer 2 by the environment's cluster located in your project.

Cloud Composer 2 pricing

This section describes pricing in Cloud Composer 2.

Pricing table for Cloud Composer 2

The following table summarizes Cloud Composer 2 costs for different regions. Also see Additional costs.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Cloud Composer 2 SKUs

Cloud Composer 2 has the following SKUs:

  • Cloud Composer Compute SKUs

    Cloud Composer Compute SKUs represent Compute Engine capacity used by Airflow schedulers, web server and workers. Airflow workers are autoscaled, and as such the corresponding costs follow the changing number of workers in the environment.

    In addition, if you deploy your own workloads in your environment's cluster, then the pricing for these workloads also follows the Cloud Composer 2 pricing model and uses Compute Engine Compute SKUs. For example, pods started in your environment's cluster as Spot Pods follow Cloud Composer 2 Compute pricing model and not GKE Autopilot Spot pricing model.

    • Cloud Composer Compute CPUs

      Associated costs depend on the combined number of vCPUs used by all your environment's components that run on Compute Engine. This includes costs for pods and services in your environment's cluster. For example, all your environment's Airflow workers run in pods in your environment cluster.

      This SKU is measured in 1000 mCPU (millicores) per hour. For example, if your environment uses 1 vCPU for 1 hour, this is equal to using 1000 mCPU for 1 hour.

    • Cloud Composer Compute Memory

      Associated costs depend on the combined amount of memory used by all your environment's components that run on Compute Engine.

    • Cloud Composer Compute Storage

      Associated costs depend on the combined amount of storage used by all your environment's components that run on Compute Engine.

  • Cloud Composer Database Storage

    Associated costs depend on the amount of disk space used by the Cloud SQL instance. The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage.

    This SKU component covers the cost of Airflow database storage.

    The minimum disk size of Cloud SQL instances is 10 GiB.

  • Small/Medium/Large Cloud Composer Environment Fee

    Associated costs depend on the size of your environment. The environment size is the scale of the managed infrastructure of your Cloud Composer environment.

    This SKU covers the cost of infrastructure components required to run Airflow, including Cloud SQL database, task queue, connection proxies. Environment infrastructure comes in three different sizes: Small, Medium and Large.

  • Small/Medium/Large Highly Resilient Cloud Composer Environment Fee

    These SKU represents the cost of infrastructure components required to run Airflow in a highly resilient Cloud Composer environment.

    If your environment is highly resilient, these SKUs replace the Small/Medium/Large Cloud Composer Environment Fee SKUs.

  • Your environment also has additional costs that are not a part of Cloud Composer pricing.

Additional costs for Cloud Composer 2

Costs for the following services are billed in addition to costs for Cloud Composer 2 environments:

Pricing example for Cloud Composer 2

Assume that you create a Cloud Composer 2 environment in Iowa (us-central1) and use the default Small environment preset.

In this case, your environment has the following default parameters that affect Cloud Composer 2 SKUs:

  • Your environment has one scheduler.
  • Your environment scales automatically between 1 and 3 workers.
  • Your environment's scheduler and web server use 0.5 vCPU each.
  • Your environment's scheduler and web server use 1.875 GiB of memory each.
  • Your environment's scheduler and web server use 1 GiB of disk space each.
  • Your environment's workers scale automatically between 0.5 and 1.5 vCPUs, depending on the number of workers.
  • Your environment's workers scale automatically between 1.875 and 5.625 GiB of memory, depending on the number of workers.
  • Your environment's workers scale automatically between 1 and 3 GiB of storage, depending on the number of workers.
  • Your environment's database uses 10 GiB of storage. The storage size increases automatically, following the demand coming from the database storage usage. This example assumes that the database storage does not increase.
  • Your environment uses the small infrastructure size.

Assume that you run this environment for 7 days and 12 hours (180 hours total). Your environment's load is 1 worker for 50% of the time and 2 workers for the other 50% of the time. After using the environment for this period of time, you delete it. In this case, your Cloud Composer 2 SKUs are:

  • Cloud Composer Compute CPUs is ( 90 hours * 1.5 vCPU + 90 hours * 2 vCPU ) * $0.045 per 1000 mCPU hours, for a total of $14.175.

  • Cloud Composer Compute Memory is ( 90 hours * 5.625 GiB + 90 hours * 7.5 GiB ) * $0.005 per GiB / hour, for a total of $5.906.

  • Cloud Composer Compute Storage is ( 90 hours * 3 GiB + 90 hours * 4 GiB ) * $0.0002 per GiB / hour, for a total of $0.126.

  • Cloud Composer Database Storage is 180 hours out of 740 hours * 10 GiB * $0.17 per GiB / month, for a total of $0.413.

  • Small Cloud Composer Environment Fee is 180 hours * $0.35 per hour, for a total of $63.00.

  • The total Cloud Composer 2 fees in this example are:

    Cloud Composer 2 SKU Fee
    Cloud Composer Compute CPUs $14.175
    Cloud Composer Compute Memory $5.906
    Cloud Composer Compute Storage $0.126
    Cloud Composer Database Storage $0.17 per GiB / month
    Small Cloud Composer Environment Fee $63.00
    Total $83.62
  • Your environment is auto-scaling. This means that if the actual load is lower during the described period, then the costs are also lower. For example, if you configure your environment to scale between 1 and 6 workers, and your environment uses only a single worker during the whole period, you only pay for this single worker.

  • Your environment also has additional costs that are not a part of Cloud Composer 2 SKUs.

Pricing model transition for Cloud Composer 2

This section describes how the pricing model changes in Cloud Composer 2 compared to Cloud Composer 1.

Cloud Composer 1 SKUs are displayed alongside Cloud Composer 2 SKUs
Figure 2. Pricing model transition from Cloud Composer 1 to Cloud Composer 2 (click to enlarge)

Introduction of Cloud Composer 2 features is accompanied by a new pricing model, enabling you to fully benefit from the efficiency of autoscaling. The new model also provides a clear perspective on a Total Cost of Ownership for Cloud Composer environments.

In Cloud Composer 1 environments, the cost of the Compute Engine layer used to run Airflow is visible as a charge for Compute Engine instances used by environment's GKE cluster.

In Cloud Composer 2 this cost is no longer associated with Compute Engine. Instead, it is included in Cloud Composer 2 SKUs for Compute Engine CPU cores, Memory and Storage. This approach provides a clear perspective on the overall cost of Cloud Composer environments.

Because the pricing model of Cloud Composer 2 is more encompassing than its predecessor, you might notice higher charges for Cloud Composer environments while at the same time seeing lower costs for Compute Engine instances.

Autoscaling introduced in Cloud Composer 2 brings additional efficiency in resource utilization because environments no longer need to be continuously scaled for the peak. The extent of cost savings generated by autoscaling is highly dependent on the pattern of DAG runs and environment configuration.

In Cloud Composer 2, compared to Cloud Composer 1:

  • Costs for your environments might vary because of environment autoscaling. You only pay for resources that are utilized by your environment.
  • You do not pay additional costs for Compute Engine components of your environment. Cloud Composer 2 SKUs replace these costs.
  • Even though Cloud Composer 2 environments rely on GKE Autopilot clusters, you are not charged a cluster management fee that is present in regular GKE Autopilot clusters.

Cloud Composer 1 pricing

Expand

This section describes pricing in Cloud Composer 1.

Pricing table for Cloud Composer 1

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Cloud Composer 1 SKUs

Cloud Composer 1 has the following SKUs:

  • Cloud Composer vCPU time

    Associated costs depend on the web server machine type of your environment.

  • Cloud Composer SQL vCPU time

    Associated costs depend on the machine type of the Cloud SQL instance.

  • Cloud Composer data storage

    Associated costs depend on the web server machine type and on the amount of disk space used by the Cloud SQL instance.

    The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage.

    The minimum disk size of Cloud SQL instances is 10 GiB.

  • Cloud Composer data transfer

    Associated costs depend on the amount of network traffic generated by web server and Cloud SQL. For example, making queries to the Airflow database, scheduling tasks and DAGs, and using Airflow web interface generates data transfer costs.

  • Your environment also has additional costs that are not a part of Cloud Composer pricing.

For Private IP environments in Cloud Composer 1, costs related to the web server are doubled. This is because Private IP Cloud Composer environments have two web server instances running behind a load balancer.

Supported machine types

The following machine types are used by components of your Cloud Composer 1 environment.

  • Pricing for these machine types is a part of additional costs.
  • Shared-core machine types are not supported.
  • The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage. The initial storage size is 10 GiB.
Environment component Supported machine types
Nodes n1-standard, n1-highmem, n1-highcpu, n2-standard, n2-highmem, n2-highcpu, e2-standard, e2-highmem, e2-highcpu, c2-standard, m1-megamem, m1-ultramem, m2-megamem, m2-ultramem, n2d-standard, n2d-highmem, n2d-highcpu
Cloud SQL instance db-n1-standard-2, db-n1-standard-4, db-n1-standard-8, db-n1-standard-16
Web server composer-n1-webserver-2, composer-n1-webserver-4, composer-n1-webserver-8

Additional costs for Cloud Composer 1

Costs for the following services are billed in addition to costs for Cloud Composer 1 environments:

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Pricing example for Cloud Composer 1

Assume that you create a Cloud Composer 1 environment in Iowa (us-central1) and use the default parameters.

In this case, your environment has the following default parameters that affect Cloud Composer 1 SKUs:

  • Your environment's web server uses the composer-n1-webserver-2 machine type. This machine type has 2 vCPUs and 20 GiB of storage.

  • Your environment's Cloud SQL instance uses the db-n1-standard-2 machine type. This machine type has 2 vCPUs.

These default parameters affect additional costs for your environment:

  • The environment's GKE cluster has 3 nodes. The nodes run environment workers and the scheduler.
  • Nodes use the n1-standard-1 machine type.
  • Nodes use 20 GiB of storage each.

Assume that you run this environment for 7 days and 12 hours (180 hours total) with 6.5 GiB of outbound data transfer, and then you delete the environment. In this case, your Cloud Composer 1 SKUs are:

  • Cloud Composer vCPU time is 180 hours * 2 vCPU * 0.074 / vCPU hour, for a total of $26.64.

  • Cloud Composer SQL vCPU time is 180 hours * 2 vCPU * 0.125 / vCPU hour, for a total of $45.00.

  • Cloud Composer data storage is 10 GiB for the database (this is the initial storage that grows as the database increases in size), plus 20 GiB for the web server, for a total of 30 GiB. The resulting fee for the storage (assuming that the database storage does not increase) is 180 hours out of 740 hours * 30 GiB * $0.273 per GiB / month for a total of $1.99.

  • Cloud Composer outbound data transfer is 6.5 GiB * $0.156 / GiB for a total of $1.04.

  • The total Cloud Composer 1 fees in this example are:

    Cloud Composer 1 SKU Fee
    Cloud Composer vCPU time $26.64
    Cloud Composer SQL vCPU time $45.00
    Cloud Composer data storage $1.99
    Cloud Composer outbound data transfer $1.04
    Total $74.67
  • Your environment also has additional costs that are not a part of Cloud Composer 1 SKUs. For example, these costs include fees for running 3 nodes of your environment's cluster for the time period when you used your environment.

What's next

Request a custom quote

With Google Cloud's pay-as-you-go pricing, you only pay for the services you use. Connect with our sales team to get a custom quote for your organization.
Contact sales