You can use the Google Cloud Pricing Calculator to help calculate Data Catalog costs:
- Open the Google Cloud Pricing Calculator.
- From the product list, select Data Catalog.
- In the API calls and metadata fields, enter the projected values for your expected loads.
- Click Add To Estimate.
Data Catalog pricing considerations
Metadata storage charges for a tag are billed to the project where the tag template was created.
API charges are billed to different projects depending on specific calls:
- Read methods (get, lookup, list, and search) are billed to the project of the account that authorized the API calls.
- Write methods (create, update, delete, and rename) for all stored entities - except tags - are billed to the project of the mutated entity.
- Write methods for tags are billed to the project of the tag template used to create the tag.
- For Data Lineage API, read API calls are billed to the project that contains the artifact.
Data Catalog pricing examples
- User A creates and applies small tags (1024 bytes each). For $100 per month, the user can store 1GiB of metadata, which corresponds approximately to 1M tags. Assuming one tag per table, it amounts to a total of 1M tagged tables.
User B creates 1M tags of 1K bytes each on the 10th of the month, and deletes the tags on 20th. The cost is $100, calculated as 1 GiB of data divided by one-third month:
1GiB * $100 // 1M tags using 1K bytes memory each amounts to 1GiB of metadata * 1/3 // The tags were stored for 10 days (between the 10th and 20th day of the month), so the cost is divided by one-third. --- = $100
User C creates and applies large tags (10K bytes each). For $100 per month, the user can store 1GiB of metadata, which corresponds to approximately 100K tags. Assuming one tag per table, it amounts to a total of 100K tagged tables.
User D creates 10 templates (for example ETL, Data Governance, Data Quality), and applies large tags (10K bytes each) using each of the 10 templates. For $100/month, the user can store 1GiB of metadata, which corresponds approximately to 100K tags. Assuming 10 tags per table, it amounts to a total of 10K tagged tables.
Estimate data lineage pricing
Data lineage pricing example
User E enables data lineage to track lineage for BigQuery in their project. The project is in the
us-central1location. During 1 month, data lineage consumes 100 DCU-hours of Dataplex Premium processing, and generates 1GiB of data lineage metadata. The cost is:
100 * $0.089 // 100 DCU-hours of Dataplex Premium processing billed at $0.089 per DCU-hr + (1GiB - 1MiB) * 100 // cost of storing 1GiB data lineage metadata excluding 1MiB of free storage per month --- = $108.8
User F turns on Data Lineage API but they don't use BigQuery in their project. Over 1 month, the user makes 50 000 calls to Data Lineage API ingesting 100MiB of data lineage metadata. The cost is:
+ 0.0967GiB * $100.00 // cost of storing 100MiB data lineage metadata excluding 1MiB of free storage per month + 0 // 50 000 calls to Data Lineage API is below the 1M free calls per month limit --- = $9.67