Grant Dataform required access

This document shows you how to grant the Identity and Access Management (IAM) roles required by Dataform service accounts to execute workflows in BigQuery.

About service accounts in Dataform

When you create your first Dataform repository, Dataform automatically generates a default service account. Dataform uses the default service account to interact with BigQuery on your behalf. The default Dataform service account is not granted any BigQuery roles or permissions by default. You must grant the required access to the default Dataform service account.

Your default Dataform service account ID is in the following format:

[e-mail]

Replace YOUR_PROJECT_NUMBER with the numeral ID of your Google Cloud project. You can find your Google Cloud project ID in the Google Cloud console dashboard. For more information, see Identifying projects.

In addition to the default Dataform service account, you can use other service accounts to execute workflows on your behalf. You can configure custom service accounts:

• At the repository level, to run all workflows in a given repository.

• Individually for each workflow configuration.

When you create a Dataform repository or workflow configuration, you can select any service account associated with your Google Cloud project that you have access to. You must configure required permissions for all service accounts associated with your Dataform resources.

A custom service account associated with a Dataform repository is only used to execute workflows from that repository. All other repository operations are still performed by the default Dataform service account.

Required roles for Dataform service accounts

Default and custom service accounts used in Dataform require the following BigQuery IAM roles to be able to execute workflows in BigQuery:

• BigQuery Data Editor on projects to which Dataform needs both read and write access. They usually include the project hosting your Dataform repository.
• BigQuery Data Viewer on projects to which Dataform needs read-only access.
• BigQuery Job User on the project hosting your Dataform repository.
• BigQuery Data Owner if you want to query BigQuery datasets.
• BigQuery roles for column-level access control if you want to use BigQuery policy tags.

Additionally, you need to grant the default Dataform service account Service Account Token Creator(roles/iam.serviceAccountTokenCreator) access to any custom service accounts that you want to use in Dataform.

Security considerations for Dataform service accounts

Granting the roles required by Dataform to a service account comes with the following security considerations:

• Any service account granted the required roles might gain access to BigQuery or Secret Manager in the project that service account belongs to, regardless of VPC Service Controls.

  For more information, see Configure VPC Service Controls for Dataform.

• Any user who has the dataform.repositories.create IAM permission can execute code using the default Dataform service account and all permissions granted to that service account.

  For more information, see Security considerations for Dataform permissions.

To restrict the data that a user or service account can read or write in BigQuery, you can grant granular BigQuery IAM permissions to selected BigQuery datasets or tables. For more information, see Controlling access to datasets and Controlling access to tables and views.

Before you begin

1. In the Google Cloud console, go to the Dataform page.

   Go to the Dataform page

2. Select or create a repository.

Grant required BigQuery roles to a service account used in Dataform

To grant the required BigQuery IAM roles to your default Dataform service account, or a custom service account that you want to use in Dataform, follow these steps:

1. In the Google Cloud console, go to the IAM page.

   Go to the IAM page

2. Click Grant Access.

3. In the New principals field, enter the service account ID.

4. In the Select a role drop-down list, select the BigQuery Job User role.

5. Click Add another role, and then in the Select a role drop-down list, select the BigQuery Data Editor role.

6. Click Add another role, and then in the Select a role drop-down list, select the BigQuery Data Viewer role.

7. Click Save.

Grant token creation access to a custom service account

To use a custom service account in Dataform, the default Dataform service account must be able to access the custom service account. To grant this access, you need to add the default Dataform service account as a principal to the custom service account with the Service Account Token Creator role.

To grant the default Dataform service account access to a custom service account, follow these steps:

1. In the Google Cloud console, go to IAM > Service accounts.

   Go to the Service accounts page

2. Select a project.

3. On the Service accounts for project "YOUR_PROJECT_NAME" page, select your custom Dataform service account.

4. Go to Permissions, and then click Grant Access.

5. In the New principals field, enter your default Dataform service account ID.

   Your default Dataform service account ID is in the following format:

   [e-mail]
   

6. In the Select a role drop-down list, select the Service Account Token Creator role.

7. Click Save.

What's next

• To learn more about BigQuery IAM roles and permissions, see Access control with IAM.
• To learn more about granting granular permissions to BigQuery datasets, see Controlling access to datasets.
• To learn more about granting granular permissions to BigQuery tables, see Controlling access to tables and views.
• To learn how to control Dataform access with VPC Service Controls, see Configure VPC Service Controls for Dataform.