Google Cloud Managed Service for Apache Kafka pricing
Preview notice
Managed Service for Apache Kafka is not free to use during the Preview.
Pricing
The cost of operating an Managed Service for Apache Kafka consists of charges related to running Kafka clusters and Private Service Connect (PSC) fees for access to the cluster.
Cluster charges
To size or scale a Managed Service for Apache Kafka cluster, you must set the total vCPU count and RAM size for the cluster. To do so, you must first understand the throughput that your cluster can handle. For more information on how to estimate the throughput, see Estimate vCPUs and memory for your cluster.
Cluster charges are billed using the three SKUs in the region where you run the cluster:
- Compute, covering vCPU and RAM
- Storage, including local storage of data on brokers (Persistent Disk) and long term storage.
- Networking, covering charges for inter-zone data transfer for replication of data within a cluster.
Understanding compute costs
The vCPU and RAM usage is reported in terms of abstract units called Data Compute Units (DCUs). A simple way to think about this unit is that a single vCPU with 4 GiB of RAM corresponds to 1 DCU.
For example, a cluster with 6 vCPUs and 4 GiB of RAM per vCPU
consumes 6 DCU. The cost of this cluster is
6 DCU * 9 cents / hour / DCU = 54 cents / hour in us-central1.
You can, however, use different amounts of RAM per vCPU in your clusters. The conversion factors from compute resources to DCUs are described in the following table:
| Compute resources | Equivalent DCU amounts |
|---|---|
| 1 vCPU | 0.6 |
| 1 GiB RAM | 0.1 |
When you use 1 GiB of RAM for an hour, you pay for 0.1 DCUs for that hour.
See the create cluster page for estimating your cluster size.
Understanding storage costs
The local storage is provisioned automatically. As a rough guide, you should expect to pay for 100GB of local storage for every vCPU you provision.
The cost of persistent storage is determined by how much data you retain in your topics. You do not pay for provisioning persistent storage capacity. You are billed for only a single replica of data in persistent storage. This is independent of the cluster configuration. For example, if you publish 1GiB of data and retain it for 1 day, you will pay for 1GiB-day of persistent storage.
In contrast, local storage is used by every replica of the data. So if you publish 1 GiB of data you will typically use 3 GiB of local disk space. Persistent storage charges are based on the total volume of data stored, while local storage charges are based on the volume of provisioned storage. Therefore, the actual volume of data stored in local storage does not affect what you pay for local storage.
Understanding inter-zone network costs
A Managed Service for Apache Kafka cluster replicates data across multiple zones. You are charged for inter-zone data transfer within the cluster. The cost is $0.01 per 1 gibibyte. By default, the replication factor for a cluster is 3. This means you pay for inter-zone data transfer for replicating data to 2 out of the 3 zones.
Let's say you have a cluster with a replication factor of 3, which is the default. You write 10 GiB of data within a single zone. With a replication factor of 3, your 10 GiB of data is copied to 2 other zones. A total of 20 GiB of data (10 GiB x 2 copies) is transferred. The cost for this inter-zone data transfer is $0.20 (20 GiB x $0.01/GiB).
Private Service Connect charges
Managed Service for Apache Kafka service sets up all Private Service Connect (PSC) endpoints for your service. These endpoints are resources in a project you own and are billed separately, as PSC charges. You can read more about how PSC is used here. The PSC charges include the following components:
VPC consumer data processing charges. Data transfer between clients and brokers in different zones costs between $0.004 to $0.01 per gigabyte (GiB). Data transfer between clients and brokers in the same zone does not incur this charge.
Hourly charge for each endpoint. A cluster has a minimum of four endpoints. Each endpoint costs $0.01 per hour. Since a cluster requires at least four endpoints, expect a minimum charge of $0.04 per hour for total endpoints for each cluster.
For the most up-to-date information about VPC pricing, see the subsection on "Using a Private Service Connect endpoint (forwarding rule) to access a published service" in the "Private Service Connect" section of the VPC pricing guide.
Total cost of ownership
The total cost of running an Managed Service for Apache Kafka cluster depends on the size of the cluster, the volume of data you publish and consume, as well as the duration for which you store the data. In addition, an important parameter to consider is the average utilization of your cluster. The utilization determines the balance of inter-zone data transfer charges and compute charges. As a rough guide, you can assume that a single vCPU can handle 20MiBps of publish traffic and 80MiBps of consumer traffic.
For example, consider the estimates of the total cost of a 100MB/s workload on Managed Service for Apache Kafka in us-central1, assuming 40% average CPU utilization and 24 hour message storage. For comparison we also provide an estimate of what it would take to operate the same workload with an open source Apache Kafka cluster on Compute Engine using N2 CPUs and local replica followers for consumers.
| Publish throughput in MiBps | Fanout (How many times each message is read by a consumer) | Estimated total monthly price of operating a Kafka cluster to handle this workload, thousands of US dollars per month in us-central | |
|---|---|---|---|
| Open source Apache Kafka on Compute Engine using N2 CPUs | Managed Service for Apache Kafka | ||
| 100 | 1 | $9.1 | $14.6 |
| 100 | 2 | $9.2 | $17.4 |