Dataproc Granular IAM

Dataproc Granular IAM is feature that allows you grant permissions at the cluster, jobs, operations, workflow template, or autoscaling policy level.

Example: You can grant one user a cluster Viewer role, which allows the user to view a cluster within a project, and grant another user a jobs Editor role, which allows that user to update and cancel, as well as view the job. See SDK Commands Enabled by Granular IAM to understand the specific Google Cloud CLI commands enabled by each Dataproc Granular IAM role.

Dataproc Granular IAM Roles and Permissions

Dataproc Granular IAM can set the following roles with the following permissions on Dataproc resources.

Cluster Roles

Role	Permissions
Viewer	dataproc.clusters.get
Editor	dataproc.clusters.get dataproc.clusters.list dataproc.clusters.delete dataproc.clusters.update dataproc.clusters.use dataproc.clusters.start dataproc.clusters.stop
Owner	dataproc.clusters.get dataproc.clusters.list dataproc.clusters.delete dataproc.clusters.update dataproc.clusters.use dataproc.clusters.start dataproc.clusters.stop dataproc.clusters.setIamPolicy dataproc.clusters.getIamPolicy

Job Roles

Role	Permissions
Viewer	dataproc.jobs.get
Editor	dataproc.jobs.get dataproc.jobs.cancel dataproc.jobs.delete dataproc.jobs.update
Owner	dataproc.jobs.get dataproc.jobs.cancel dataproc.jobs.delete dataproc.jobs.update dataproc.jobs.setIamPolicy dataproc.jobs.getIamPolicy

Operation Roles

Role	Permissions
Viewer	dataproc.operations.get
Editor	dataproc.jobs.get dataproc.operations.cancel dataproc.operations.delete
Owner	dataproc.jobs.get dataproc.operations.cancel dataproc.operations.delete dataproc.operations.setIamPolicy dataproc.operations.getIamPolicy

Workflow Template Roles

Role	Permissions
Viewer	dataproc.workflowTemplates.get
Editor	dataproc.workflowTemplates.get dataproc.workflowTemplates.delete dataproc.workflowTemplates.update
Owner	dataproc.workflowTemplates.get dataproc.workflowTemplates.delete dataproc.workflowTemplates.update dataproc.workflowTemplates.setIamPolicy dataproc.workflowTemplates.getIamPolicy

Autoscaling Policy Roles

Role	Permissions
Viewer	dataproc.autoscalingPolicies.get
Editor	dataproc.autoscalingPolicies.get dataproc.autoscalingPolicies.use dataproc.autoscalingPolicies.delete dataproc.autoscalingPolicies.update
Owner	dataproc.autoscalingPolicies.get dataproc.autoscalingPolicies.use dataproc.autoscalingPolicies.delete dataproc.autoscalingPolicies.update dataproc.autoscalingPolicies.setIamPolicy dataproc.autoscalingPolicies.getIamPolicy

Using Dataproc Granular IAM

This section explains how to use Dataproc Granular IAM to assign roles to users on an existing Dataproc resource. See Granting, Changing, and Revoking Access for more general information on updating and removing Identity and Access Management (IAM) roles.

gcloud Command

Get the resource's IAM policy, and write it to a JSON file (the resource-type can be "clusters" or "jobs" or "operations" or "workflow-templates" or "autoscaling-policies"):
```
gcloud dataproc resource-type get-iam-policy  resource-id \
    --region=region \
    --format=json > iam.json
```

The contents of the JSON file will look similar to the following:

{
  "bindings": [
    {
      "role": "roles/editor",
      "members": [
        "user:mike@example.com",
        "group:admins@example.com",
        "domain:google.com",
        "serviceAccount:my-other-app@appspot.gserviceaccount.com"
      ]
    }
  ],
  "etag": "string"
}

Using a text editor, add a new binding object to the bindings array that defines users and the resource access role for those users. For example, to grant the Viewer role (roles/viewer) to the user sean@example.com, you would change the example above to add a new binding object (shown in bold, below. Note: make sure to return the etag value you received from gcloud dataproc resource-type get-iam-policy (see the etag documentation).
```
{
  "bindings": [
    {
      "role": "roles/editor",
      "members": [
        "user:mike@example.com",
        "group:admins@example.com",
        "domain:google.com",
        "serviceAccount:my-other-app@appspot.gserviceaccount.com"
      ]
    },
    {
      "role": "roles/viewer",
      "members": [
        "user:sean@example.com"
      ]
    }
  ],
  "etag": "value-from-get-iam-policy"
}
```
Update the cluster's policy with the new bindings array by running the following command (resource-type can be "clusters" or "jobs" or "operations" or "workflow-templates" or "autoscaling-policies"):
```
gcloud dataproc resource-type set-iam-policy resource-name \
    --region=region \
    --format=json iam.json
```

The command outputs the updated policy:

{
  "bindings": [
    {
      "role": "roles/editor",
      "members": [
        "user:mike@example.com",
        "group:admins@example.com",
        "domain:google.com",
        "serviceAccount:my-other-app@appspot.gserviceaccount.com"
      ]
    },
    {
      "role": "roles/viewer",
      "members": [
        "user:sean@example.com"
      ]
    }
  ],
  "etag": "string"
}

REST API

Issue a resource-type ("clusters" or "jobs" or "operations" or "workflowTemplates" or "autoscalingPolicies") getIamPolicy request to get the IAM policy for the resource.
Cluster getIamPolicy Example:
```
GET https://dataproc.googleapis.com/v1/projects/projectName/regions/region/clusters/clusterName:getIamPolicy
```

The contents of the JSON file will look similar to the following:

{
  "bindings": [
    {
      "role": "roles/editor",
      "members": [
        "user:mike@example.com",
        "group:admins@example.com",
        "domain:google.com",
        "serviceAccount:my-other-app@appspot.gserviceaccount.com"
      ]
    }
  ],
  "etag": "string"
}

Using a text editor, construct the following JSON policy object to enclose the bindings array you just received from the Dataproc service. Make sure to return the "etag" value you received in the getIamPolicy response (see the etag documentation). Now, add a new binding object to the bindings array that defines users and the cluster access role for those users. For example, to grant the Viewer role (roles/viewer) to the user sean@example.com, you would change the example above to add a new binding object (shown in bold, below).
```
{
  "policy": {
    "version": "",
    "bindings": [
      {
        "role": "roles/editor",
        "members": [
          "user:mike@example.com",
          "group:admins@example.com",
          "domain:google.com",
          "serviceAccount:my-other-app@appspot.gserviceaccount.com"
        ]
      },
      {
        "role": "roles/viewer",
        "members": [
          "user:sean@example.com"
        ]
      }
    ],
    "etag": "value-from-getIamPolicy"
  }
}
```

Set the updated policy on the resource by issuing a setIamPolicy request.

Cluster setIamPolicy Example:

POST https://dataproc.googleapis.com/v1/projects/projectName/regions/region/clusters/clusterName:setIamPolicy

Request body

{
  "policy": {
    "version": "",
    "bindings": [
      {
        "role": "roles/editor",
        "members": [
          "user:mike@example.com",
          "group:admins@example.com",
          "domain:google.com",
          "serviceAccount:my-other-app@appspot.gserviceaccount.com"
        ]
      },
      {
        "role": "roles/viewer",
        "members": [
          "user:sean@example.com"
        ]
      }
    ],
    "etag": "value-from-getIamPolicy"
  }
}

The contents of the JSON response will look similar to the following:

Response

{
  "bindings": [
    {
      "role": "roles/editor",
      "members": [
        "user:mike@example.com",
        "group:admins@example.com",
        "domain:google.com",
        "serviceAccount:my-other-app@appspot.gserviceaccount.com"
      ]
    },
    {
      "role": "roles/viewer",
      "members": [
        "user:sean@example.com"
      ]
    }
  ],
  "etag": "string"
}

Console

Go to the Dataproc Clusters page in the Google Cloud console, then click the box to the left of the custer name to open the Permissions/Labels panel (if the panel is not showing, click SHOW INFO PANEL at the top-right of the page). Under the Permissions tab, select the Dataproc role, add one or more account addresses in the Add principals box, then click Add.

SDK Commands Enabled by Granular IAM

The tables below show the gcloud dataproc commands enabled on Dataproc resources by each Granular IAM role.

Clusters

IAM Role	Command
Viewer	`gcloud dataproc clusters describe cluster-name`
Editor	`gcloud dataproc clusters describe cluster-name` `gcloud dataproc clusters list` `gcloud dataproc clusters delete cluster-name` `gcloud dataproc clusters diagnose cluster-name` `gcloud dataproc clusters update cluster-name` `gcloud beta dataproc clusters start cluster-name` `gcloud beta dataproc clusters stop cluster-name`
Owner	`gcloud dataproc clusters describe cluster-name` `gcloud dataproc clusters list` `gcloud dataproc clusters delete cluster-name` `gcloud dataproc clusters diagnose cluster-name` `gcloud dataproc clusters update cluster-name` `gcloud beta dataproc clusters start cluster-name` `gcloud beta dataproc clusters stop cluster-name` `gcloud dataproc clusters get-iam-policy cluster-name` `gcloud dataproc clusters set-iam-policy cluster-name`

Jobs

IAM Role	Command
Viewer	`gcloud dataproc jobs describe job-id`
Editor	`gcloud dataproc jobs delete job-id` `gcloud dataproc jobs describe job-id` `gcloud dataproc jobs kill job-id` `gcloud dataproc jobs update job-id` `gcloud dataproc jobs wait job-id`
Owner	`gcloud dataproc jobs delete job-id` `gcloud dataproc jobs describe job-id` `gcloud dataproc jobs kill job-id` `gcloud dataproc jobs update job-id` `gcloud dataproc jobs wait job-id` `gcloud dataproc jobs get-iam-policy job-id` `gcloud dataproc jobs set-iam-policy job-id`

Operations

IAM Role	Command
Viewer	`gcloud dataproc operations describe operation-id`
Editor	`gcloud dataproc operations delete operation-id` `gcloud dataproc operations describe operation-id` `gcloud dataproc operations cancel operation-id`
Owner	`gcloud dataproc operations delete operation-id` `gcloud dataproc operations describe operation-id` `gcloud dataproc operations cancel operation-id` `gcloud dataproc operations get-iam-policy operation-id` `gcloud dataproc operations set-iam-policy operation-id`

Workflow Templates

IAM Role	Command
Viewer	`gcloud dataproc workflow-templates describe template-id`
Editor	`gcloud dataproc workflow-templates delete template-id` `gcloud dataproc workflow-templates describe template-id` `gcloud dataproc workflow-templates remove-job template-id` `gcloud dataproc workflow-templates run template-id`
Owner	`gcloud dataproc workflow-templates delete template-id` `gcloud dataproc workflow-templates describe template-id` `gcloud dataproc workflow-templates remove-job template-id` `gcloud dataproc workflow-templates run template-id` `gcloud dataproc workflow-templates get-iam-policy template-id` `gcloud dataproc workflow-templates set-iam-policy template-id`

Autoscaling Policies

IAM Role	Command
Viewer	`gcloud dataproc autoscaling-policies describe policy-id`
Editor	`gcloud dataproc autoscaling-policies delete policy-id` `gcloud dataproc autoscaling-policies describe policy-id` `gcloud dataproc autoscaling-policies update policy-id` `gcloud dataproc clusters create cluster-name --autoscaling-policy policy-id`
Owner	`gcloud dataproc autoscaling-policies delete policy-id` `gcloud dataproc autoscaling-policies describe policy-id` `gcloud dataproc autoscaling-policies update policy-id` `gcloud dataproc clusters create cluster-name --autoscaling-policy policy-id` `gcloud dataproc autoscaling-policies get-iam-policy policy-id` `gcloud dataproc autoscaling-policies set-iam-policy policy-id`

Submitting Jobs with Granular IAM

To allow a principal (user, group or service account) to submit jobs to a specified cluster using Dataproc Granular IAM, in addition to granting a user an Editor role on a cluster, additional permissions must be set at the project level. Here are the steps to take to allow a principal to submit jobs on a specified Dataproc cluster:

Create a Cloud Storage bucket that your cluster can use to connect to Cloud Storage.
Add the principal to the bucket-level policy, selecting the Storage Object Viewer role for the principal (see roles/storage.objectViewer), which includes the following permissions:
1. storage.objects.get
2. storage.objects.list
When you create the cluster, pass the name of the bucket you just created to your cluster using the --bucket parameter (see gcloud dataproc clusters create --bucket).
After the cluster is created, set a policy on the cluster that grants the principal an Editor or Owner role (see Using Dataproc Granular IAM).
Create an IAM custom role with the following permissions:
1. dataproc.jobs.create
2. dataproc.jobs.get
Select or Add the principal on the Google Cloud console IAM page, then select the custom role to apply it to the principal.