This page describes how to grant the Dataproc Service Account User role to Cloud Data Fusion to allow it to provision and run pipelines on Dataproc clusters.
For service accounts that are used by Dataproc, you also need to
datafusion.instances.runtime permission to access
Cloud Data Fusion runtime resources.
Whether you use a user-managed service account, or the default Compute Engine service account on the virtual machines in a cluster, you must grant the Service Account User role to Cloud Data Fusion. Otherwise, Cloud Data Fusion cannot provision a Dataproc cluster and the following error appears when you execute a data pipeline:
PROVISION task failed in REQUESTING_CREATE state for program run [pipeline-name] due to Dataproc operation failure: INVALID_ARGUMENT: User not authorized to act as service account '[service-account-name]'
Get the service account name
- In the Google Cloud Console, go to the Identity and Access Management page.
Go to the IAM page
- From the project selector at the top of the page, choose the project, folder, or organization to which the Cloud Data Fusion instance belongs.
- Find and copy the Cloud Data Fusion service account
name. Use the following format:
Grant service account user permission
- In the Cloud Console, go to the Service Accounts page.
Go to the Service Accounts page
- Click Select a project, choose a project where the service account you want to use for the Dataproc cluster is located, and then click Open.
Select the checkbox by the Dataproc service account.
If the info panel is not already visible, click Show info panel. The panel displays a list of principals that have been granted roles on the service account.
Click Add Principal.
In the New principals field, paste the Cloud Data Fusion service account name that you previously copied.
Select the Service Account User role.
Grant roles to Dataproc service accounts
Grant runner role permission
Grant the Cloud Data Fusion runner role
roles/datafusion.runner) to service accounts that are used by
Dataproc. This authorizes the Dataproc service
account to run Cloud Data Fusion pipelines in your project.
For more information, see Requiring permission to attach service accounts to resources.
Grant Cloud Storage admin permission
- Learn more about Access control in Cloud Data Fusion.
- Learn more about Cloud Data Fusion service accounts.