Quotas and limits

This document contains current content limits and request quotas for the Cloud Data Fusion API.

Request quotas

The following Cloud Data Fusion API usage quotas apply to Cloud Data Fusion control plane API requests, such as requests to create, update, or delete Cloud Data Fusion instances. They do not apply to requests for other Google Cloud resources, such as Compute Engine or Cloud Storage resources, to run data processing pipelines on Cloud Data Fusion instances (other quota limits may apply to requests for pipeline resources).

Quotas are subject to change.

Request Quota Value
Requests per minute from a single user in a single region 600

These limits apply to each Google Cloud Console project and are shared across all applications and IP addresses using that project.

Other Google Cloud quotas

Your Cloud Data Fusion instances can use other Google Cloud products to run pipelines. These products have project-level quotas, which include quotas that apply to Cloud Data Fusion use.

Dataproc

Cloud Data Fusion uses Dataproc as the execution environment for pipelines. Dataproc quota applies to all pipelines that you execute.

Cloud Logging

Cloud Data Fusion saves logs in Logging. The Logging quota applies to your Cloud Data Fusion instances.

Additional quotas

Depending on the Google Cloud services your pipelines use, additional quotas might apply, for example:

  • BigQuery—When reading or writing data into BigQuery, the BigQuery quota applies.
  • Cloud Storage—When reading or writing data into Cloud Storage, the Cloud Storage quota applies.
  • Cloud Bigtable—When reading or writing data into Cloud Bigtable, the Cloud Bigtable quota applies.
  • Spanner—When reading or writing data into Spanner, the Spanner quota applies.