Google Cloud Managed Service for Apache Kafka pricing
The table below offers a rough idea of the total monthly cost of running a cluster. The table compares the cost of running open source code yourself with the cost of the managed service.
| Producer bandwidth | Estimated monthly price | |
|---|---|---|
| Apache Kafka on Compute Engine | Managed Service for Apache Kafka | |
| 10 MiB/s | $0.9K | $1.1K |
| 100 MiB/s | $9.1K | $11K |
- Three replicas distributed across three zones
- 4 GiB of RAM per vCPU
- RAM and vCPU utilization of 50%
- Storage of 24 hours
- Consumers configured to use local follower replica to minimize cross-zone data transfer
- Consumer bandwidth equal to the producer bandwidth
- N2 CPU family for the Compute Engine clusters.
- Prices for
us-central1in thousands of US dollars per month, so that "$1.0K" means "$1000 per month."
In general, the total cost of running a Managed Service for Apache Kafka cluster depends on the size of the cluster, the volume of data you publish and consume, as well as the duration for which you store the data. As a rough guide, you can assume that a single vCPU can handle 20MiB/s of publish traffic and 80MiB/s of consumer traffic.
In addition, an important parameter to consider is the average utilization of RAM and vCPU capacity. Unpredictable and variable traffic requires low average utilization to accommodate the peaks in producer or consumer bandwidth.
For clusters with utilization above 20%, the cost of data transfer between zones the largest component of the total cost. This cost comes from replication of data between leader and follower brokers as well as data transfer traffic between clients and brokers. You can reduce this cost by configuring your consumer clients to use local replicas, compressing your data. If high availability and durability of storage is not critical for a topic, you can also reduce the number of replicas stored for a given topic to reduce replication charges.
Pricing details
The cost of operating a Managed Service for Apache Kafka consists of charges related to running Kafka clusters and Private Service Connect (PSC) fees for access to the cluster.
Cluster charges
To size or scale a Managed Service for Apache Kafka cluster, you must set the total vCPU count and RAM size for the cluster. To do so, you must first understand the throughput that your cluster can handle. For more information on how to estimate the throughput, see Estimate vCPUs and memory for your cluster.
Cluster charges are billed using the three SKUs in the region where you run the cluster:
- Compute, covering vCPU and RAM
- Storage, including local storage of data on brokers (Persistent Disk) and long term storage.
- Networking, covering charges for inter-zone data transfer for replication of data within a cluster.
Compute charges
The vCPU and RAM usage is reported in terms of abstract units called Data Compute Units (DCUs). A simple way to think about this unit is that a single vCPU with 4 GiB of RAM corresponds to 1 DCU.
For example, a cluster with 6 vCPUs and 4 GiB of RAM per vCPU
consumes 6 DCU. The cost of this cluster is
6 DCU * 9 cents / hour / DCU = 54 cents / hour in us-central1.
You can, however, use different amounts of RAM per vCPU in your clusters. The conversion factors from compute resources to DCUs are described in the following table:
| Compute resources | Equivalent DCU amounts |
|---|---|
| 1 vCPU | 0.6 |
| 1 GiB RAM | 0.1 |
When you use 1 GiB of RAM for an hour, you pay for 0.1 DCUs for that hour.
See the create cluster page for estimating your cluster size.
Storage charges
The local storage is provisioned automatically. As a rough guide, you should expect to pay for 100GB of local storage for every vCPU you provision.
The cost of persistent storage is determined by how much data you retain in your topics. You do not pay for provisioning persistent storage capacity. You are billed for only a single replica of data in persistent storage. This is independent of the cluster configuration. For example, if you publish 1GiB of data and retain it for 1 day, you will pay for 1GiB-day of persistent storage.
In contrast, local storage is used by every replica of the data. So if you publish 1 GiB of data you will typically use 3 GiB of local disk space. Persistent storage charges are based on the total volume of data stored, while local storage charges are based on the volume of provisioned storage. Therefore, the actual volume of data stored in local storage does not affect what you pay for local storage.
Inter-zone network charges
A Managed Service for Apache Kafka cluster replicates data across multiple zones. You are charged for inter-zone data transfer within the cluster. The cost is $0.01 per 1 gibibyte. By default, the replication factor for a cluster is 3. This means you pay for inter-zone data transfer for replicating data to 2 out of the 3 zones.
Let's say you have a cluster with a replication factor of 3, which is the default. You write 10 GiB of data within a single zone. With a replication factor of 3, your 10 GiB of data is copied to 2 other zones. A total of 20 GiB of data (10 GiB x 2 copies) is transferred. The cost for this inter-zone data transfer is $0.20 (20 GiB x $0.01/GiB).
Private Service Connect charges
Managed Service for Apache Kafka service sets up all Private Service Connect (PSC) endpoints for your service. These endpoints are resources in a project you own and are billed separately. You can read more about how PSC is used here. The PSC charges include the following components:
VPC consumer data processing charges. Data transfer between clients and brokers in different zones costs between $0.004 to $0.01 per gigabyte (GiB). Data transfer between clients and brokers in the same zone does not incur this charge.
Hourly charge for each endpoint. A cluster has a minimum of four endpoints. Each endpoint costs $0.01 per hour. Since a cluster requires at least four endpoints, expect a minimum charge of $0.04 per hour for total endpoints for each cluster.
For the most up-to-date information about VPC pricing, see the subsection on "Using a Private Service Connect endpoint (forwarding rule) to access a published service" in the "Private Service Connect" section of the VPC pricing guide.