Troubleshoot issues with the discovery service

This page shows you how to resolve issues with the discovery service of Sensitive Data Protection. For more information about the discovery service, see Data profiles.

The service agent doesn't have permission to read an access-controlled column

This issue occurs when profiling a table that enforces column-level security through policy tags. If the service agent doesn't have permission to access the restricted column, Sensitive Data Protection shows the following error:

Permission denied for DLP API service account 'SERVICE_AGENT_ID'
while accessing a BigQuery table. Access Denied: BigQuery BigQuery: User does
not have permission to access policy tag "POLICY_TAG_ID" on column FIELD_NAME.

To resolve this issue, on the Identity and Access Management (IAM) page, grant your service agent the Fine-Grained Reader role.

Go to IAM

Sensitive Data Protection periodically retries profiling tables that it failed to profile.

For more information on granting a role, see Grant a single role.

The service agent doesn't have data profiling access

This issue occurs after someone in your organization creates an organization- or folder-level scan configuration. When you view the scan configuration details, you see that the value for Scan status is Active with errors. When you view the error, Sensitive Data Protection shows the following error message:

None of the driver projects (PROJECT_ID) have MISSING_PERMISSION
permission for organizations/ORGANIZATION_ID.

This error occurred because Sensitive Data Protection couldn't automatically grant the DLP Organization Data Profiles Driver role to your service agent while it was creating your scan configuration. The creator of the scan configuration doesn't have permissions to grant data profiling access, and so Sensitive Data Protection was unable to do it on their behalf.

To resolve this issue, see Grant data profiling access to a service agent.

The service account doesn't have permission to query a table

This issue occurs when Sensitive Data Protection attempts to profile a table that the service agent doesn't have permission to query. Sensitive Data Protection shows the following error:

Permission denied error: Permission denied for DLP API service account 'SERVICE_AGENT_ID'
while accessing BigQuery table. Access Denied: Table TABLE: User does not have
permission to query table TABLE. Permission denied for DLP API service account
'SERVICE_AGENT_ID' while accessing BigQuery table. Access Denied: Table TABLE:
User does not have permission to query TABLE. [TIMESTAMP]

To resolve this issue, follow these steps:

  1. Confirm that the table still exists. If the table exists, perform the next steps.

  2. Activate Cloud Shell.

    Activate Cloud Shell

    If you're prompted to authorize Cloud Shell, click Authorize.

    Alternatively, if you want to use the bq command-line tool from the Google Cloud CLI, install and initialize the Google Cloud CLI.

  3. Get the current IAM policy for the table, and print it to stdout:

    bq get-iam-policy TABLE
    

    Replace TABLE with the full resource name of the BigQuery table, in the format PROJECT_ID:DATASET_ID.TABLE_ID—for example, project-id:dataset-id.table-id.

  4. Grant the DLP API Service Agent (roles/dlp.serviceAgent) role to the service agent:

    bq add-iam-policy-binding --member=serviceAccount:SERVICE_AGENT_ID \
        --role=roles/dlp.serviceAgent TABLE
    

    Replace the following:

    • SERVICE_AGENT_ID: the ID of the service agent that needs to query the table—for example, service-0123456789@dlp-api.iam.gserviceaccount.com.
    • TABLE: the full resource name of the BigQuery table, in the format PROJECT_ID:DATASET_ID.TABLE_ID—for example, project-id:dataset-id.table-id.

      The output is similar to the following:

    Successfully added member 'SERVICE_AGENT_ID' to role 'roles/dlp.serviceAgent' in IAM policy for table 'TABLE':
    
    {
     "bindings": [
       {
         "members": [
           "serviceAccount:SERVICE_AGENT_ID"
         ],
         "role": "roles/dlp.serviceAgent"
       }
     ],
     "etag": "BwXNAPbVq+A=",
     "version": 1
    }
    

    Sensitive Data Protection periodically retries profiling tables that it failed to profile.

The service account doesn't have permission to publish to a Pub/Sub topic

This issue occurs when Sensitive Data Protection attempts to publish notifications to a Pub/Sub topic where the service agent has no publishing access. Sensitive Data Protection shows the following error:

Permission missing to publish notifications on Cloud Pub/Sub topic 'TOPIC_NAME'.
The DLP API service account 'SERVICE_AGENT_ID' must must have at least the Pub/Sub Publisher role.

To resolve this issue, grant publishing access, at a project or topic level, to your service agent. An example of a role that has publishing access is the Pub/Sub Publisher role.

If there are configuration or permission issues with the Pub/Sub topic, Sensitive Data Protection retries sending the Pub/Sub notification for up to two weeks. After two weeks, the notification is discarded.

The inspection template can't be used to profile data in a different region

This issue occurs when Sensitive Data Protection attempts to profile data that doesn't reside in the same region where the inspection template resides. Sensitive Data Protection shows the following error:

Data in region DATA_REGION cannot be profiled using template in region
TEMPLATE_REGION. Regional template can only be used to profile data
in the same region. If profiling data in multiple regions, use a global template.

In this error message, DATA_REGION is the region where the data resides, and TEMPLATE_REGION is the region where the inspection template resides.

To resolve this issue, you can copy the region-specific template to the global region:

  1. Copy the inspection template to the global region.

  2. On the Inspection template details page, copy the full resource name of the template. The full resource name follows this format:

    projects/PROJECT_ID/locations/REGION/inspectTemplates/TEMPLATE_ID
  3. Edit the scan configuration and enter the full resource name of the new inspection template.

  4. Click Save.

Sensitive Data Protection periodically retries profiling tables that it failed to profile.

Sensitive Data Protection attempted to profile an unsupported table

This issue occurs when Sensitive Data Protection attempts to profile a table that is not supported. For that table, you still get a partial profile containing the table's metadata. However, the partial profile shows the following error:

Unimplemented error: Table of type `TABLE_TYPE` is not currently supported for inspection. [DATE_TIME].

If you don't want to get partial profiles and errors for unsupported tables, follow these steps:

  1. Edit the scan configuration.
  2. In the Manage schedules step, click Edit schedule.
  3. In the pane that appears, click the Conditions tab.
  4. In the Tables to profile section, click Profile supported tables.

For more information, see Manage schedules.

The premade Looker report isn't loading properly

See Troubleshoot errors with the premade report.