Cloud Dataproc Granular IAM

Cloud Dataproc Granular IAM is feature that allows you grant permissions at the cluster, jobs, operations, or workflow template level.

Example: You can grant one user a cluster Viewer role, which allows the user to view a cluster within a project, and grant another user a jobs Editor role, which allows that user to update and cancel, as well as view the job. See SDK Commands Enabled by Granular IAM to understand the specific Google Cloud SDK commands enabled by each Cloud Dataproc Granular IAM role.

Cloud Dataproc Granular IAM Roles and Permissions

Cloud Dataproc Granular IAM can set the following roles with the following permissions on Cloud Dataproc resources.

Cluster Roles

Role Permissions
Viewer dataproc.clusters.get
Editor dataproc.clusters.get
dataproc.clusters.list
dataproc.clusters.delete
dataproc.clusters.update
dataproc.clusters.use
Owner dataproc.clusters.get
dataproc.clusters.list
dataproc.clusters.delete
dataproc.clusters.update
dataproc.clusters.use
dataproc.clusters.setIamPolicy
dataproc.clusters.getIamPolicy

Job Roles

Role Permissions
Viewer dataproc.jobs.get
Editor dataproc.jobs.get
dataproc.jobs.cancel
dataproc.jobs.delete
dataproc.jobs.update
Owner dataproc.jobs.get
dataproc.jobs.cancel
dataproc.jobs.delete
dataproc.jobs.update
dataproc.jobs.setIamPolicy
dataproc.jobs.getIamPolicy

Operation Roles

Role Permissions
Viewer dataproc.operations.get
Editor dataproc.jobs.get
dataproc.operations.cancel
dataproc.operations.delete
Owner dataproc.jobs.get
dataproc.operations.cancel
dataproc.operations.delete
dataproc.operations.setIamPolicy
dataproc.operations.getIamPolicy

Workflow Template Roles

Role Permissions
Viewer dataproc.workflowTemplates.get
Editor dataproc.workflowTemplates.get
dataproc.workflowTemplates.delete
dataproc.workflowTemplates.update
Owner dataproc.workflowTemplates.get
dataproc.workflowTemplates.delete
dataproc.workflowTemplates.update
dataproc.operations.setIamPolicy
dataproc.operations.getIamPolicy

Using Dataproc Granular IAM

This section explains how to use Cloud Dataproc Granular IAM to assign roles to users on an existing Cloud Dataproc resource. See Granting, Changing, and Revoking Access to Project Members for more general information on updating and removing Cloud IAM roles.

gcloud Command

  1. Get the resource's IAM policy, and write it to a JSON file:
    gcloud beta dataproc resource-type ("clusters" or "jobs" or "operations" or "workflow-templates" \
        get-iam-policy  resource-name --format=json > iam.json
    
  2. The contents of the JSON file will look similar to the following:
    {
      "bindings": [
        {
          "role": "roles/editor",
          "members": [
            "user:mike@example.com",
            "group:admins@example.com",
            "domain:google.com",
            "serviceAccount:my-other-app@appspot.gserviceaccount.com"
          ]
        }
      ],
      "etag": "string"
    }
    
    
  3. Using a text editor, add a new binding object to the bindings array that defines users and the resource access role for those users. For example, to grant the Viewer role (roles/viewer) to the user sean@example.com, you would change the example above to add a new binding object (shown in bold, below. Note: make sure to return the etag value you received from gcloud beta dataproc resource-type get-iam-policy (see the etag documentation).
    {
      "bindings": [
        {
          "role": "roles/editor",
          "members": [
            "user:mike@example.com",
            "group:admins@example.com",
            "domain:google.com",
            "serviceAccount:my-other-app@appspot.gserviceaccount.com"
          ]
        },
        {
          "role": "roles/viewer",
          "members": [
            "user:sean@example.com"
          ]
        }
      ],
      "etag": "value-from-get-iam-policy"
    }
    
    
  4. Update the cluster's policy with the new bindings array by running the following command:
    gcloud beta dataproc resource-type ("clusters" or "jobs" or "operations" or "workflow-templates") \
        set-iam-policy resource-name --format=json iam.json
    
  5. The command outputs the updated policy:
    {
      "bindings": [
        {
          "role": "roles/editor",
          "members": [
            "user:mike@example.com",
            "group:admins@example.com",
            "domain:google.com",
            "serviceAccount:my-other-app@appspot.gserviceaccount.com"
          ]
        },
        {
          "role": "roles/viewer",
          "members": [
            "user:sean@example.com"
          ]
        }
      ],
      "etag": "string"
    }
    

REST API

  1. Issue a resource-type ("clusters" or "jobs" or "operations" or "workflowTemplates") getIamPolicy request to get the IAM policy for the resource.

    Get Cluster Policy Example

    GET https://dataproc.googleapis.com/v1beta2/projects/projectName/regions/region/clusters/clusterName:getIamPolicy
    
  2. The contents of the JSON file will look similar to the following:
    {
      "bindings": [
        {
          "role": "roles/editor",
          "members": [
            "user:mike@example.com",
            "group:admins@example.com",
            "domain:google.com",
            "serviceAccount:my-other-app@appspot.gserviceaccount.com"
          ]
        }
      ],
      "etag": "string"
    }
    
  3. Using a text editor, construct the following JSON policy object to enclose the bindings array you just received from the Cloud Dataproc service. Make sure to return the "etag" value you received in the getIamPolicy response (see the etag documentation). Now, add a new binding object to the bindings array that defines users and the cluster access role for those users. For example, to grant the Viewer role (roles/viewer) to the user sean@example.com, you would change the example above to add a new binding object (shown in bold, below).
    {
      "policy": {
        "version": "",
        "bindings": [
          {
            "role": "roles/editor",
            "members": [
              "user:mike@example.com",
              "group:admins@example.com",
              "domain:google.com",
              "serviceAccount:my-other-app@appspot.gserviceaccount.com"
            ]
          },
          {
            "role": "roles/viewer",
            "members": [
              "user:sean@example.com"
            ]
          }
        ],
        "etag": "value-from-getIamPolicy"
      }
    }
    
  4. Set the updated policy on the resource by issuing a setIamPolicy request.

    Set Cluster Policy Example

    POST https://dataproc.googleapis.com/v1beta2/projects/projectName/regions/region/clusters/clusterName:setIamPolicy
    

    Request body

    {
      "policy": {
        "version": "",
        "bindings": [
          {
            "role": "roles/editor",
            "members": [
              "user:mike@example.com",
              "group:admins@example.com",
              "domain:google.com",
              "serviceAccount:my-other-app@appspot.gserviceaccount.com"
            ]
          },
          {
            "role": "roles/viewer",
            "members": [
              "user:sean@example.com"
            ]
          }
        ],
        "etag": "value-from-getIamPolicy"
      }
    }
    
  5. The contents of the JSON response will look similar to the following:

    Response

    {
      "bindings": [
        {
          "role": "roles/editor",
          "members": [
            "user:mike@example.com",
            "group:admins@example.com",
            "domain:google.com",
            "serviceAccount:my-other-app@appspot.gserviceaccount.com"
          ]
        },
        {
          "role": "roles/viewer",
          "members": [
            "user:sean@example.com"
          ]
        }
      ],
      "etag": "string"
    }
    

Console

Using Granular Cloud Dataproc IAM from the Google Cloud Platform Console will be supported in a future Cloud Dataproc release.

SDK Commands Enabled by Granular IAM

The tables below show the gcloud dataproc commands enabled on Cloud Dataproc resources by each Granular IAM role.

Clusters

IAM Role Command
Viewer gcloud dataproc clusters describe cluster-name
Editor gcloud dataproc clusters describe cluster-name
gcloud dataproc clusters list
gcloud dataproc clusters delete cluster-name
gcloud dataproc clusters diagnose cluster-name
gcloud dataproc clusters update cluster-name
Owner gcloud dataproc clusters describe cluster-name
gcloud dataproc clusters list
gcloud dataproc clusters delete cluster-name
gcloud dataproc clusters diagnose cluster-name
gcloud dataproc clusters update cluster-name
gcloud beta dataproc clusters get-iam-policy cluster-name
gcloud beta dataproc clusters set-iam-policy cluster-name

Jobs

IAM Role Command
Viewer gcloud dataproc jobs describe job-id
Editor gcloud dataproc jobs delete job-id
gcloud dataproc jobs describe job-id
gcloud dataproc jobs kill job-id
gcloud dataproc jobs update job-id
gcloud dataproc jobs wait job-id
Owner gcloud dataproc jobs delete job-id
gcloud dataproc jobs describe job-id
gcloud dataproc jobs kill job-id
gcloud dataproc jobs update job-id
gcloud dataproc jobs wait job-id
gcloud beta dataproc jobs get-iam-policy job-id
gcloud beta dataproc jobs set-iam-policy job-id

Operations

IAM Role Command
Viewer gcloud dataproc operations describe operation-id
Editor gcloud dataproc operations delete operation-id
gcloud dataproc operations describe operation-id
gcloud dataproc operations cancel operation-id
Owner gcloud dataproc operations delete operation-id
gcloud dataproc operations describe operation-id
gcloud dataproc operations cancel operation-id
gcloud beta dataproc operations get-iam-policy operation-id
gcloud beta dataproc operations set-iam-policy operation-id

Workflow Templates

IAM Role Command
Viewer gcloud beta dataproc workflow-templates describe template-id
Editor gcloud beta dataproc workflow-templates delete template-id
gcloud beta dataproc workflow-templates describe template-id
gcloud beta dataproc workflow-templates remove-job template-id
gcloud beta dataproc workflow-templates run template-id
Owner gcloud beta dataproc workflow-templates delete template-id
gcloud beta dataproc workflow-templates describe template-id
gcloud beta dataproc workflow-templates remove-job template-id
gcloud beta dataproc workflow-templates run template-id
gcloud beta dataproc workflow-templates get-iam-policy template-id
gcloud beta dataproc workflow-templates set-iam-policy template-id

Submitting Jobs with Granular IAM

To allow a member (user, group or service account) to submit jobs to a specified cluster using Cloud Dataproc Granular IAM, in addition to granting a user an Editor role on a cluster, additional permissions must be set at the project level. Here are the steps to take to allow a member to submit jobs on a specified Cloud Dataproc cluster:

  1. Create a Cloud Storage bucket that your cluster can use to connect to Cloud Storage.
  2. Add the member to the bucket-level policy, selecting the Storage Object Viewer role for the member (see roles/storage.objectViewer), which includes the following permissions:
    1. storage.objects.get
    2. storage.objects.list
  3. When you create the cluster, pass the name of the bucket you just created to your cluster using the --bucket parameter (see gcloud dataproc clusters create --bucket).
  4. After the cluster is created, set a policy on the cluster that grants the member an Editor or Owner role (see Using Dataproc Granular IAM).
  5. Create a Cloud IAM custom role with the following permissions:
    1. dataproc.jobs.create
    2. dataproc.jobs.get
  6. Select or Add the member on the GCP Console IAM page, then select the custom role to apply it to the member.
Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataproc Documentation