Quotas

Stay organized with collections Save and categorize content based on your preferences.

This document lists the quotas and limits that apply to Dataplex.

A quota restricts how much of a particular shared Google Cloud resource your Cloud project can use, including hardware, software, and network components.

Quotas are part of a system that does the following:

  • Monitors your use or consumption of Google Cloud products and services.
  • Restricts your consumption of those resources for reasons including ensuring fairness and reducing spikes in usage.
  • Maintains configurations that automatically enforce prescribed restrictions.
  • Provides a means to make or request changes to the quota.

When a quota is exceeded, in most cases, the system immediately blocks access to the relevant Google resource, and the task that you're trying to perform fails. In most cases, quotas apply to each Cloud project and are shared across all applications and IP addresses that use that Cloud project.

Dataplex enforces quotas on the number of API requests during a specific period of time and the number of resources that can be created in a project. Quotas protect you from unforeseen spikes in usage and overloaded services. Learn more about why quotas are enforced.

Your quota needs might increase the more that you use Dataplex. If a quota is too low to meet your needs, or you expect an upcoming increase in usage, you can proactively request a quota increase.

Request quotas

The following quotas apply to Dataplex requests:

Quotas Examples
Read requests for resource management Get a lake, list zones
Write requests for resource management Create a lake, delete an asset
Get requests for metadata Get an entity, get a partition
List requests for metadata List entities, list partitions
Write requests for metadata Update an entity, delete a partition
Read requests for task and job resources Get a task, list jobs
Write requests for task and job resources Create a task, cancel a job
Set IAM policy requests Set an IAM policy.

Every request quota above is enforced at both per-project level and per-project-per-user level.

Resource quotas

Resource quota is enforced at per-project level for the following resource types: lake, zone, asset, on-demand task, recurring task. You can check your current usage of Dataplex quota on the Quotas page in the Google Cloud console.

Request a quota increase

  1. In the Google Cloud console, go to the Quotas page.

    Go to Quotas

    The page displays a list of quotas.

  2. Select the quotas you want to modify and click Edit quotas.

  3. Enter the new quota for the quota you selected.

  4. Click Submit request.

To increase or decrease most quotas, use the Google Cloud console. For more information, see Requesting a higher quota.

Quota dependencies

Your Dataplex usage might involve other Google Cloud products. For example, data discovery requires invoking Cloud Storage and/or BigQuery APIs to interact with the underlying resources. These products have project-level quotas, which include quotas that apply to Dataplex use.

Quota pages of other products that you might use with Dataplex:

This list is not exhaustive.