Troubleshoot issues with the data profiler

This page shows you how to resolve issues with the data profiler of Cloud Data Loss Prevention (DLP).

Permissions

This section lists access issues that you might encounter, and provides suggestions for how to fix each of them.

The service agent doesn't have permission to read an access-controlled column

This issue occurs when profiling a table that enforces column-level security through policy tags. If the service agent doesn't have permission to access the restricted column, Cloud DLP shows the following error:

Permission denied for DLP API service account 'SERVICE_AGENT_ID'
while accessing a BigQuery table. Access Denied: BigQuery BigQuery: User does
not have permission to access policy tag "POLICY_TAG_ID" on column FIELD_NAME.

To resolve this issue, on the Identity and Access Management (IAM) page, grant your service agent the Fine-Grained Reader role.

Go to IAM

Cloud DLP periodically retries profiling tables that it failed to profile.

For more information on granting a role, see Grant a single role.

The service agent doesn't have data profiling access

This issue occurs after someone in your organization creates an organization- or folder-level scan configuration. When you view the scan configuration details, you see that the value for Scan status is Active with errors. When you view the error, Cloud DLP shows the following error message:

None of the driver projects (PROJECT_ID) have MISSING_PERMISSION
permission for organizations/ORGANIZATION_ID.

This error occurred because Cloud DLP couldn't automatically grant the DLP Organization Data Profiles Driver role to your service agent while it was creating your scan configuration. The creator of the scan configuration doesn't have permissions to grant data profiling access, and so Cloud DLP was unable to do it on their behalf.

To resolve this issue, see Grant data profiling access to a service agent.

The service account doesn't have permission to query a table

This issue occurs when Cloud DLP attempts to profile a table that the service agent doesn't have permission to query. Cloud DLP shows the following error:

Permission denied error: Permission denied for DLP API service account 'SERVICE_AGENT_ID'
while accessing BigQuery table. Access Denied: Table TABLE: User does not have
permission to query table TABLE. Permission denied for DLP API service account
'SERVICE_AGENT_ID' while accessing BigQuery table. Access Denied: Table TABLE:
User does not have permission to query TABLE. [TIMESTAMP]

To resolve this issue, follow these steps:

  1. Confirm that the table still exists. If the table exists, perform the next steps.

  2. Activate Cloud Shell.

    Activate Cloud Shell

    If you're prompted to authorize Cloud Shell, click Authorize.

    Alternatively, if you want to use the bq command-line tool from the Cloud SDK, install and initialize the Cloud SDK.

  3. Get the current IAM policy for the table, and print it to stdout:

    bq get-iam-policy TABLE
    

    Replace TABLE with the full resource name of the BigQuery table, in the format PROJECT_ID:DATASET_ID.TABLE_ID—for example, project-id:dataset-id.table-id.

  4. Grant the DLP API Service Agent (roles/dlp.serviceAgent) role to the service agent:

    bq add-iam-policy-binding --member=serviceAccount:SERVICE_AGENT_ID \
        --role=roles/dlp.serviceAgent TABLE
    

    Replace the following:

    • SERVICE_AGENT_ID: the ID of the service agent that needs to query the table—for example, service-0123456789@dlp-api.iam.gserviceaccount.com.
    • TABLE: the full resource name of the BigQuery table, in the format PROJECT_ID:DATASET_ID.TABLE_ID—for example, project-id:dataset-id.table-id.

      The output is similar to the following:

    Successfully added member 'SERVICE_AGENT_ID' to role 'roles/dlp.serviceAgent' in IAM policy for table 'TABLE':
    
    {
     "bindings": [
       {
         "members": [
           "serviceAccount:SERVICE_AGENT_ID"
         ],
         "role": "roles/dlp.serviceAgent"
       }
     ],
     "etag": "BwXNAPbVq+A=",
     "version": 1
    }
    

    Cloud DLP periodically retries profiling tables that it failed to profile.

Inspection templates

This section lists issues that you might encounter with inspection templates, and provides suggestions for how to fix each of them.

The inspection template can't be used to profile data in a different region

This issue occurs when Cloud DLP attempts to profile data that doesn't reside in the same region where the inspection template resides. Cloud DLP shows the following error:

Data in region DATA_REGION cannot be profiled using template in region
TEMPLATE_REGION. Regional template can only be used to profile data
in the same region. If profiling data in multiple regions, use a global template.

In this error message, DATA_REGION is the region where the data resides, and TEMPLATE_REGION is the region where the inspection template resides.

To resolve this issue, you can copy the region-specific template to the global region:

  1. Go to your configurations list.

    Go to Configurations

  2. If needed, switch to the project that contains the inspection template that you want to use.

  3. Click the Templates tab, and then click the Inspect subtab.

  4. Find the template that you want to use.

  5. Click Actions, and then click Copy.

  6. On the Create template page, in the Resource location list, select Global (any region).

  7. Click Create.

  8. On the Inspection template details page, copy the full resource name of the template. The full resource name follows this format:

    projects/PROJECT_ID/locations/REGION/inspectTemplates/TEMPLATE_ID
  9. Edit the scan configuration and enter the full resource name of the new inspection template.

  10. Click Save.

Cloud DLP periodically retries profiling tables that it failed to profile.