Exporting data for Cloud IAM recommendations

This page explains how to export the data that the Cloud IAM recommender uses to generate recommendations.

For more information about the Cloud IAM recommender, see the Cloud IAM recommender overview.

To disable the Cloud IAM recommender, or any other recommender, see Opting out of recommendations.

Exporting data used to generate recommendations

The Cloud IAM role recommender uses personal metadata collected during the usage of services in Google Cloud to provide recommendations. This personal metadata can be exported to BigQuery for each project in an organization.

You can use the BigQuery Data Transfer Service for Recommender to export personal metadata used for recommendations by following the steps below.

BigQuery Data Transfer Service configuration for Cloud IAM role recommender

Configuration Description
Schedule Every 24 hours, non-configurable
Refresh window Last 2 days, non-configurable
Maximum backfill duration Last 60 days.

Before you begin

Before you create a Recommendations data transfer:

  • Verify that you have completed all actions required to enable the BigQuery Data Transfer Service.
  • You must allow the BigQuery Data Transfer Service permission to manage your transfer.

  • Create a BigQuery dataset to store data.

    • Currently, only datasets in US and EU are supported.
    • The transfer you set up will use the same region as the dataset is created in, and it is immutable once the dataset and transfer are created.

Required permissions

The following BigQuery permissions are required to export data:

  • bigquery.transfers.update: Allows creating the transfer
  • bigquery.datasets.update: Allows update actions on the target dataset

Ensure that you grant either the BigQuery Admin role (roles/bigquery.admin), which contains both of these permissions, or create a custom role that has these permissions.

The following Recommender permissions are required to export data:

  • dataProcessing.iamAccessHistory.exportData: Allows exporting data

Ensure that you grant either the Data Processing IAM Access History Exporter role (roles/dataprocessing.iamAccessHistoryExporter), which contains this permission, or create a custom role that has this permission.

Creating a data transfer for personal metadata used for recommendations

  1. Open the Cloud Console.

    Go to the Cloud Console

  2. First, enroll the IAM Recommender Aggregated Access datasource.

  3. In the navigation pane, click BigQuery.

    You can also open the BigQuery web UI directly by entering the following URL in your browser.

    https://console.cloud.google.com/bigquery

    Clicking the button below will open the BigQuery web UI directly using your most recently accessed project.

    Go to the BigQuery web UI

  4. Click Transfers.

  5. Click Create Transfer.

  6. Click on the Create Transfer page.

  7. In the Source type section, for Source, choose IAM Recommender Aggregated Access:

  8. In the Transfer config name section, for Display name, enter a name for the transfer such as My Transfer. The transfer name can be any value that allows you to easily identify the transfer if you need to modify it later:

  9. In the Schedule options section, for Schedule, leave the default value (Start now) or click Start at a set time.

    • For Repeats, choose an option for how often to run the transfer.
    • If you choose an option other than Daily, additional options are available. For example, if you choose Weekly, an option appears for you to select the day of the week.
    • For Start date and run time, enter the date and time to start the transfer. If you choose Start now, this option is disabled. There will be a 1 day delay before your newly created transfer starts if you choose Start now in the schedule.

  10. In the Destination settings section, for Destination dataset, choose the dataset you created to store your data. The transfer will be run in the same region as the dataset. If after creating the transfer, you need to edit the dataset, the new dataset will also have to be setup in the same region.

  11. In the Data source details section, for a project number, enter the appropriate project numbers comma separated. A maximum of 10 projects can be supported in one transfer.

Next steps