Because Cloud Bigtable is typically used for large, high-volume workloads, it has the potential to generate an extremely high number of logs. Before you enable Data Access audit logging for Cloud Bigtable, you should estimate and understand the Cloud Audit Logs ingestion and storage costs that audit logging can incur each month.
Your Data Access audit logging costs are directly related to the number of Bigtable requests that you choose to log each month. The following table shows rough estimates of the Cloud Audit Logs costs that you can expect based on your average requests per second and the length of time that you store your logs, assuming that you log all data requests. See Calculating your costs for a detailed explanation of how these estimates are calculated.
|Average requests per second||Log retention time||Approximate monthly cost|
Calculating your costs
Start with the following assumptions:
- The number of seconds in an average month is about 2,628,000.
- The average audit size is around 1 kb.
- You are not charged for the first 50 GiB of audit logs that are ingested per month, and after you pass that amount, you're charged $0.50/GiB.
- Storage is free for 30 days; after that you're charged $0.01/GiB for storage.
The method described on this page provides a gross estimate based on all traffic. In production, you are encouraged to restrict service account logging.
Calculate your monthly log volume
First, estimate the average amount of logs your traffic will generate in an average month.
- Gather the average number of requests per second that your
application sends to Bigtable over the course of a month.
- If you use client-side metrics, you can use them to determine your average queries per second (QPS) for the last month.
- If you prefer to use your instance's Monitoring page in the Cloud Console, use it to determine the average values for Read requests and Write requests per second over the last month, then add those two values together.
- Multiply the requests per second by 2,628,000 to get the average requests per month.
- Divide that number by 10e6, or 1,000,000. The result is the estimated monthly log volume in GB that you might generate each month.
- Multiply the monthly log volume in GB by .93 to get the approximate monthly log volume in GiB.
Calculate your ingestion costs
- Subtract 50 GiB from the monthly log volume in GiB that you calculated. There is no charge for the first 50 GiB.
- Multiply the remainder by $0.50 to arrive at your estimated monthly ingestion costs.
Calculate your storage costs
- If you plan to let your logs expire after 30 days, your cost for storage is $0.00.
- If you store your logs for longer than 30 days, your storage costs can be estimated by multiplying the monthly log volume by $0.01. These costs start to incur after the first month.
5,000 request per second, logs retained for 90 days
In this example, suppose that your average number of requests per second is 5,000 and you plan to keep your logs for 90 days. Using the steps on this page, you calculate the following estimates:
- Multiply 5,000 by 2,628,000 to arrive at 13,140,000,000 requests per month.
- Divide 13,140,000,000 by 10e6 to arrive at roughly 13,140 GB of monthly log volume.
- Convert that number to GiB by multiplying it by .93 to arrive at 12,220.
- Subtract 50 GiB from your monthly log volume to get 12,170 GiB.
- Multiply by $0.50 to get $6,085 in ingestion costs.
- For the first month that your logs exist, the storage cost is $0.
- The second month, the log storage cost is 12,170 multiplied by $0.01, or about $122.
- Every month after the second month, the monthly storage cost is double that, or $244.
- After the second month, your estimated Data Access audit logging costs would be around $6,329 per month.
Presented in equation form, this looks like (((((5,000 rps * 2,628,000 sec)/1,000,000) * .93) - 50 GiB) * $0.50) + $122 = $6,207.
In this example, your monthly Data Access logging costs are around $6,329 per month.