Cloud Composer pricing

This document explains Cloud Composer pricing.

To see the pricing for other products, read the Pricing documentation.

To search for individual SKUs associated with Cloud Composer, go to Google Cloud SKUs.

Pricing overview

This section describes general concepts of Cloud Composer pricing.

Pricing calculator

For Cloud Composer 1, you can use the Google Cloud Pricing Calculator to estimate the cost of using Cloud Composer.

Billing interval

Cloud Composer environments are billed at 10-minute intervals. For example, if you create an environment, run it for 6 hours and 30 minutes, and delete it afterwards, then the total costs are for the actual time period when you ran your environment, 6.5 hours.

Units of measurement

Disk size and network usage are calculated in Gibibytes (GiB). GiB (Gibibytes) is a standard unit used in the field of data processing and transmission and is defined as base 1024 rather than base 1000. For example, 1 Gigabyte (GB) is defined as 10003 bytes, whereas 1 GiB is defined as 10243 bytes.

Although some pricing is stated in hours or by the month, Cloud Composer is still billed for the actual usage time.

Pricing models

Cloud Composer uses two pricing models:

Pricing model transition from Cloud Composer 1 to Cloud Composer 2

Cloud Composer pricing model comparison
Figure 1. Pricing model transition from Cloud Composer 1 to Cloud Composer 2 (click to enlarge)

Introduction of Cloud Composer 2 features is accompanied by a new pricing model, enabling you to fully benefit from the efficiency of autoscaling. The new model also provides a clear perspective on a Total Cost of Ownership for Cloud Composer environments.

In Cloud Composer 1 environments, the cost of the Compute Engine layer used to run Airflow is visible as a charge for Compute Engine instances used by environment's GKE cluster.

In Cloud Composer 2 this cost is no longer associated with Compute Engine. Instead, it is included in Cloud Composer 2 SKUs for Compute Engine CPU cores, Memory and Storage. This approach provides a clear perspective on the overall cost of Cloud Composer environments.

The following diagram visualizes the pricing model transition from Cloud Composer 1 to Cloud Composer 2.

Because the pricing model of Cloud Composer 2 is more encompassing than its predecessor, you might notice higher charges for Cloud Composer environments while at the same time seeing lower costs for Compute Engine instances.

Autoscaling introduced in Cloud Composer 2 brings additional efficiency in resource utilization as environments no longer need to be continuously scaled for the peak. The extent of cost savings generated by autoscaling is highly dependent on the pattern of DAG runs and environment configuration.

SKUs for Cloud Composer 2

Compared to Cloud Composer 1:

  • Costs for your environments may vary because of environment autoscaling. You only pay for resources that are utilized by your environment.
  • You do not pay additional costs for Compute Engine components of your environment. Cloud Composer 2 SKUs replace these costs.
  • Even though Cloud Composer 2 environments rely on GKE Autopilot clusters, you are not charged a cluster management fee that is present in regular GKE Autopilot clusters.

Cloud Composer 2 has the following SKUs:

  • Cloud Composer Compute SKUs

    Cloud Composer Compute SKUs represent Compute Engine capacity used by Airflow schedulers, web server and workers. Airflow workers are autoscaled, and as such the corresponding costs follow the changing number of workers in the environment.

    In addition, if you deploy your own workloads in your environment's cluster, then the pricing for these workloads also follows the Cloud Composer 2 pricing model and uses Compute Engine Compute SKUs. For example, pods started in your environment's cluster as Spot Pods follow Cloud Composer 2 Compute pricing model and not GKE Autopilot Spot Pricing model.

    • Cloud Composer Compute CPUs

      Associated costs depend on the combined number of vCPUs used by all your environment's components that run on Compute Engine. This includes costs for pods and services in your environment's cluster. For example, all your environment's Airflow workers run in pods in your environment cluster.

      This SKU is measured in 1000 mCPU (millicores) per hour. For example, if your environment uses 1 vCPU for 1 hour, this is equal to using 1000 mCPU for 1 hour.

    • Cloud Composer Compute Memory

      Associated costs depend on the combined amount of memory used by all your environment's components that run on Compute Engine.

    • Cloud Composer Compute Storage

      Associated costs depend on the combined amount of storage used by all your environment's components that run on Compute Engine.

  • Cloud Composer Database Storage

    Associated costs depend on the amount of disk space used by the Cloud SQL instance. The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage.

    This SKU component covers the cost of Airflow database storage.

    The minimum disk size of Cloud SQL instances is 10 GiB.

  • Small/Medium/Large Cloud Composer Environment Fee

    Associated costs depend on the size of your environment. The environment size is the scale of the managed infrastructure of your Cloud Composer environment.

    This SKU covers the cost of infrastructure components required to run Airflow, including Cloud SQL database, task queue, connection proxies. Environment infrastructure comes in three different sizes: Small, Medium and Large.

  • Small/Medium/Large Highly Resilient Cloud Composer Environment Fee

    These SKU represents the cost of infrastructure components required to run Airflow in a highly resilient Cloud Composer environment.

    If your environment is highly resilient, these SKUs replace the Small/Medium/Large Cloud Composer Environment Fee SKUs.

  • Your environment also has additional costs that are not a part of Cloud Composer pricing.

Pricing table for Cloud Composer 2

The following table summarizes Cloud Composer 2 costs for different regions.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Pricing example for Cloud Composer 2

Assume that you create a Cloud Composer 2 environment in Iowa (us-central1) and use the default Small environment preset.

In this case, your environment has the following default parameters that affect Cloud Composer 2 SKUs:

  • Your environment has one scheduler.
  • Your environment scales automatically between 1 and 3 workers.
  • Your environment's scheduler and web server use 0.5 vCPU each.
  • Your environment's scheduler and web server use 1.875 GiB of memory each.
  • Your environment's scheduler and web server use 1 GiB of disk space each.
  • Your environment's workers scale automatically between 0.5 and 1.5 vCPUs, depending on the number of workers.
  • Your environment's workers scale automatically between 1.875 and 5.625 GiB of memory, depending on the number of workers.
  • Your environment's workers scale automatically between 1 and 3 GiB of storage, depending on the number of workers.
  • Your environment's database uses 10 GiB of storage. The storage size increases automatically, following the demand coming from the database storage usage. This example assumes that the database storage does not increase.
  • Your environment uses the small infrastructure size.

Assume that you run this environment for 7 days and 12 hours (180 hours total). Your environment's load is 1 worker for 50% of the time and 2 workers for the other 50% of the time. After using the environment for this period of time, you delete it. In this case, your Cloud Composer 2 SKUs are:

  • Cloud Composer Compute CPUs is ( 90 hours * 1.5 vCPU + 90 hours * 2 vCPU ) * $0.045 per 1000 mCPU hours, for a total of $14.175.

  • Cloud Composer Compute Memory is ( 90 hours * 5.625 GiB + 90 hours * 7.5 GiB ) * $0.005 per GiB / hour, for a total of $5.906.

  • Cloud Composer Compute Storage is ( 90 hours * 3 GiB + 90 hours * 4 GiB ) * $0.0002 per GiB / hour, for a total of $0.126.

  • Cloud Composer Database Storage is 180 hours out of 740 hours * 10 GiB * $0.17 per GiB / month, for a total of $0.413.

  • Small Cloud Composer Environment Fee is 180 hours * $0.35 per hour, for a total of $63.00.

  • The total Cloud Composer 2 fees in this example are:

    Cloud Composer 2 SKU Fee
    Cloud Composer Compute CPUs $14.175
    Cloud Composer Compute Memory $5.906
    Cloud Composer Compute Storage $0.126
    Cloud Composer Database Storage $0.17 per GiB / month
    Small Cloud Composer Environment Fee $63.00
    Total $83.62
  • Your environment is auto-scaling. This means that if the actual load is lower during the described period, then the costs are also lower. For example, if you configure your environment to scale between 1 and 6 workers, and your environment uses only a single worker during the whole period, you only pay for this single worker.

  • Your environment also has additional costs that are not a part of Cloud Composer 2 SKUs. For example, these costs include fees for the Cloud Storage bucket of an environment, which is used for managing DAGs and holding task logs.

SKUs for Cloud Composer 1

Cloud Composer 1 has the following SKUs:

  • Cloud Composer vCPU time

    Associated costs depend on the web server machine type of your environment.

  • Cloud Composer SQL vCPU time

    Associated costs depend on the machine type of the Cloud SQL.

  • Cloud Composer data storage

    Associated costs depend on the web server machine type and on the amount of disk space used by the Cloud SQL instance.

    The disk size of Cloud SQL instances increases automatically, following the demand coming from the database storage usage.

    The minimum disk size of Cloud SQL instances is 10 GiB.

  • Cloud Composer data transfer

    Associated costs depend on the amount of network traffic generated by web server and Cloud SQL. For example, making queries to the Airflow database, scheduling tasks and DAGs, and using Airflow web interface generates data transfer costs.

  • Your environment also has additional costs that are not a part of Cloud Composer pricing.

For Private IP environments in Cloud Composer 1, costs related to the web server are doubled. This is because Private IP Cloud Composer environments have two web server instances running behind a load balancer.

Pricing table for Cloud Composer 1

The following table summarizes Cloud Composer 1 costs for different regions.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Pricing example for Cloud Composer 1

Assume that you create a Cloud Composer 1 environment in Iowa (us-central1) and use the default parameters.

In this case, your environment has the following default parameters that affect Cloud Composer 1 SKUs:

  • Your environment's web server uses the composer-n1-webserver-2 machine type. This machine type has 2 vCPUs and 20 GiB of storage.

  • Your environment's Cloud SQL instance uses the db-n1-standard-2 machine type. This machine type has 2 vCPUs.

These default parameters affect additional costs for your environment:

  • The environment's GKE cluster has 3 nodes. The nodes run environment workers and the scheduler.
  • Nodes use the n1-standard-1 machine type.
  • Nodes use 20 GiB of storage each.

Assume that you run this environment for 7 days and 12 hours (180 hours total) with 6.5 GiB of outbound data transfer, and then you delete the environment. In this case, your Cloud Composer 1 SKUs are:

  • Cloud Composer vCPU time is 180 hours * 2 vCPU * 0.074 / vCPU hour, for a total of $26.64.

  • Cloud Composer SQL vCPU time is 180 hours * 2 vCPU * 0.125 / vCPU hour, for a total of $45.00.

  • Cloud Composer data storage is 10 GiB for the database (this is the initial storage that grows as the database increases in size), plus 20 GiB for the web server, for a total of 30 GiB. The resulting fee for the storage (assuming that the database storage does not increase) is 180 hours out of 740 hours * 30 GiB * $0.273 per GiB / month for a total of $1.99.

  • Cloud Composer outbound data transfer is 6.5 GiB * $0.156 / GiB for a total of $1.04.

  • The total Cloud Composer 1 fees in this example are:

    Cloud Composer 1 SKU Fee
    Cloud Composer vCPU time $26.64
    Cloud Composer SQL vCPU time $45.00
    Cloud Composer data storage $1.99
    Cloud Composer outbound data transfer $1.04
    Total $74.67
  • Your environment also has additional costs that are not a part of Cloud Composer 1 SKUs. For example, these costs include fees for running 3 nodes of your environment's cluster for the time period when you used your environment.

Additional costs

Costs for the following services are billed in addition to costs for Cloud Composer environments:

  1. (Cloud Composer 1 only) Google Kubernetes Engine nodes used for workers and schedulers in an environment. These nodes are subject to separate Compute Engine pricing based on the number and type of instances used. This includes fees for Persistent Disk used by every node and the Redis queue. The Redis queue disk persist unless manually deleted.

  2. The Cloud Storage bucket of an environment, which is used for managing DAGs and holding task logs. This bucket persists unless manually deleted.

    For more information, see Cloud Storage pricing.

  3. The data that Cloud Monitoring collects to help you understand your environment's performance and health. By default, Monitoring is enabled, and the data is subject to separate Monitoring pricing.

  4. The storage and outbound data transfer generated when using Container Registry and Artifact Registry. These services are used to store and serve container images created when you install additional PyPI modules.

    For more information, see Container Registry pricing and Artifact Registry pricing pages.

  5. If you use Customer Managed Encryption Keys, there might be additional costs for the usage of the Cloud Key Management Service. See Cloud Key Management Service pricing for details.

  6. If you use Private Service Connect then the following additional charges are applied:

    See Virtual Private Cloud pricing for details.

  7. Creating and storing environment snapshots produces additional costs related to Cloud Storage. These additional costs are:

    The costs depend on the snapshot creation frequency and the size of a snapshot. Snapshot size depends on the Airflow database size and the size of the data that is included in the snapshot (/dags, /data, and /plugins folders in the environment's bucket).

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Machine types for nodes (Cloud Composer 1)

These machine types are used by nodes of your environment. Pricing for these machine types are part of additional costs.

Cloud Composer environments support the following Compute Engine instance types:

  • N1 standard machine types (n1-standard)
  • N1 high-memory machine types (n1-highmem)
  • N1 high-CPU machine types (n1-highcpu)
  • N2 standard machine types (n2-standard)
  • N2 high-memory machine types (n2-highmem)
  • N2 high-CPU machine types (n2-highcpu)
  • E2 standard machine types (e2-standard)
  • E2 high-memory machine types (e2-highmem)
  • E2 high-CPU machine types (e2-highcpu)
  • C2 compute-optimized machine types (c2-standard)
  • M1 memory-optimized machine types (m1-megamem, m1-ultramem)
  • M2 memory-optimized machine types (m2-megamem, m2-ultramem)
  • N2D standard machine types running on AMD processors (n2d-standard)
  • N2D high-memory machine types running on AMD processors (n2d-highmem)
  • N2D high-CPU machine types running on AMD processors (n2d-highcpu)

Shared-core machine types are not supported.

Machine types for the Cloud SQL instance (Cloud Composer 1)

The following machines types are supported for the Cloud SQL running the Airflow database of your environment.

Machine Type vCPUs Memory Disk
db-n1-standard-2 2 7.5 GiB See note
db-n1-standard-4 4 15 GiB See note
db-n1-standard-8 8 30 GiB See note
db-n1-standard-16 16 60 GiB See note

Machine types for the web server (Cloud Composer 1)

The following machine types are supported for the VM instance that runs the Airflow web server:

Machine Type vCPUs Memory Disk
composer-n1-webserver-2 2 1.6 GiB 20 GiB
composer-n1-webserver-4 4 3.6 GiB 20 GiB
composer-n1-webserver-8 8 7.6 GiB 20 GiB

What's next

Request a custom quote

With Google Cloud's pay-as-you-go pricing, you only pay for the services you use. Connect with our sales team to get a custom quote for your organization.
Contact sales