Identity and Access Management (IAM) lets you control access to your project's resources. This document focuses on the IAM permissions relevant to Serverless for Apache Spark and the IAM roles that grant those permissions.
Dataproc permissions for Serverless for Apache Spark
Dataproc permissions allow users and
service accounts,
to perform actions on Serverless for Apache Spark
resources. For example, the dataproc.batches.create
permission lets you create batch workloads in a project.
You don't directly give users permissions; instead, you grant them IAM roles, which have one or more permissions bundled within them. You can grant predefined roles that contain a list of permissions, or you can create and grant custom roles that contain one or more permissions that you include in the custom role.
The following tables list the basic permissions necessary to call
Dataproc APIs (methods) that create or access Serverless for Apache Spark
resources. The tables are organized according to the APIs associated with each
Serverless for Apache Spark resource, which include batches
, sessions
,
sessionTemplates
, and operations
.
Examples:
dataproc.batches.create
allows the creation of batches in the containing project.dataproc.sessions.create
allows the creation of interactive sessions in the containing project.
Batch permissions
Method | Required Permission(s) |
---|---|
projects.locations.batches.create | dataproc.batches.create 1 |
projects.locations.batches.delete | dataproc.batches.delete |
projects.locations.batches.get | dataproc.batches.get |
projects.locations.batches.list | dataproc.batches.list |
1 dataproc.batches.create
also requires dataproc.batches.get
and
dataproc.operations.get
permissions to allow it to get status updates
from the gcloud
command-line tool.
Session permissions
Method | Required Permission(s) |
---|---|
projects.locations.sessions.create | dataproc.sessions.create 1 |
projects.locations.sessions.delete | dataproc.sessions.delete |
projects.locations.sessions.get | dataproc.sessions.get |
projects.locations.sessions.list | dataproc.sessions.list |
projects.locations.sessions.terminate | dataproc.sessions.terminate |
1 dataproc.sessions.create
also requires dataproc.sessions.get
and
dataproc.operations.get
permissions to allow it to get status updates
from the gcloud
command-line tool.
Session template permissions
Method | Required Permission(s) |
---|---|
projects.locations.sessionTemplates.create | dataproc.sessionTemplates.create 1 |
projects.locations.sessionTemplates.delete | dataproc.sessionTemplates.delete |
projects.locations.sessionTemplates.get | dataproc.sessionTemplates.get |
projects.locations.sessionTemplates.list | dataproc.sessionTemplates.list |
projects.locations.sessionTemplates.update | dataproc.sessionTemplates.update |
1 dataproc.sessionTemplates.create
also requires dataproc.sessionTemplates.get
and
dataproc.operations.get
permissions to allow it to get status updates
from the gcloud
command-line tool.
Operations permissions
Method | Required Permission(s) |
---|---|
projects.regions.operations.get | dataproc.operations.get |
projects.regions.operations.list | dataproc.operations.list |
projects.regions.operations.cancel 1 | dataproc.operations.cancel |
projects.regions.operations.delete | dataproc.operations.delete |
projects.regions.operations.getIamPolicy | dataproc.operations.getIamPolicy |
projects.regions.operations.setIamPolicy | dataproc.operations.setIamPolicy |
1 To cancel batch operations, dataproc.operations.cancel
also requires
dataproc.batches.cancel
permission.
Serverless for Apache Spark 3.0+ runtime permissions
The following permissions apply to Serverless for Apache Spark 3.0
and
later runtimes.
Workloads permissions
Method | Required Permission(s) |
---|---|
dataprocrm.v1.dataprocrm.projects.locations.workloads.create | dataprocrm.workloads.create |
dataprocrm.v1.dataprocrm.projects.locations.workloads.cancel | dataprocrm.workloads.cancel |
dataprocrm.v1.dataprocrm.projects.locations.workloads.delete | dataprocrm.workloads.delete |
dataprocrm.v1.dataprocrm.projects.locations.workloads.get | dataprocrm.workloads.get |
dataprocrm.v1.dataprocrm.projects.locations.workloads.list | dataprocrm.workloads.list |
dataprocrm.v1.dataprocrm.projects.locations.workloads.use | dataprocrm.workloads.use |
NodePools permissions
Method | Required Permission(s) |
---|---|
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.create | dataprocrm.nodePools.create |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.delete | dataprocrm.nodePools.delete |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.resize | dataprocrm.nodePools.resize |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.deleteNodes | dataprocrm.nodePools.deleteNodes |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.update | dataprocrm.nodePools.update |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.get | dataprocrm.nodePools.get |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.list | dataprocrm.nodePools.list |
Nodes permissions
Method | Required Permission(s) |
---|---|
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.create | dataprocrm.nodes.create |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.delete | dataprocrm.nodes.delete |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.update | dataprocrm.nodes.update |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.heartbeat | dataprocrm.nodes.heartbeat |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.get | dataprocrm.nodes.get |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.list | dataprocrm.nodes.list |
dataprocrm.v1.dataprocrm.projects.locations.workloads.nodePools.nodes.mintOAuthToken | dataprocrm.nodes.mintOAuthToken |
Operations permissions
Method | Required Permission(s) |
---|---|
dataprocrm.v1.dataprocrm.projects.locations.operations.get | dataprocrm.operations.get |
dataprocrm.v1.dataprocrm.projects.locations.operations.list | dataprocrm.operations.list |
Dataproc and Dataproc Serverless roles
Dataproc IAM roles can contain permissions associated with both Dataproc and Serverless for Apache Spark resources. Dataproc Serverless IAM roles contain permissions associated with Serverless for Apache Spark resources.
For example, the Dataproc Viewer role contains the
dataproc.batches
and dataproc.sessions
get and list permissions, which
allow a user or service account to get and list batch workloads and sessions
in a project.
User permissions and role requirements
Users must have service account
ActAs
permission to deploy Serverless for Apache Spark resources, for example, to submit batch workloads. See Roles for service account authentication for detailed information.Serverless for Apache Spark
3.0
or later runtime role requirements:- A user account needs the Dataproc Editor or Dataproc Serverless Editor role to run batches and sessions.
- A user account needs the Dataproc Viewer or Dataproc Serverless Editor role to get and list batches and sessions.
- A custom service account needs the Dataproc Worker role to run batches and sessions.
For information on service accounts associated with the Serverless for Apache Spark
3.0
runtime, see Serverless for Apache Spark service accounts.
Lookup Dataproc roles and permissions
You can use the following sections to lookup Dataproc roles and permissions.
Role | Permissions |
---|---|
Dataproc Administrator( Full control of Dataproc resources. |
|
Dataproc Editor( Provides the permissions necessary for viewing the resources required to manage Dataproc, including machine types, networks, projects, and zones. Lowest-level resources where you can grant this role:
|
|
Dataproc Hub Agent( Allows management of Dataproc resources. Intended for service accounts running Dataproc Hub instances. |
|
Dataproc Serverless Editor( Permissions needed to run serverless sessions and batches as a user |
|
Dataproc Serverless Node.( Node access to Dataproc Serverless sessions and batches. Intended for service accounts. |
|
Dataproc Serverless Viewer( Permissions needed to view serverless sessions and batches |
|
Dataproc Service Agent( Gives Dataproc Service Account access to service accounts, compute resources, storage resources, and kubernetes resources. Includes access to service accounts. |
|
Dataproc Viewer( Provides read-only access to Dataproc resources. Lowest-level resources where you can grant this role:
|
|
Dataproc Worker( Provides worker access to Dataproc resources. Intended for service accounts. |
|
Project roles
You can also set permissions at the project level by using IAM Project roles. The following table summarizes the permissions associated with IAM project roles:
Project Role | Permissions |
---|---|
Project Viewer | All project permissions for read-only actions that preserve state (get, list) |
Project Editor | All Project Viewer permissions plus all project permissions for actions that modify state (create, delete, update, use, cancel, stop, start) |
Project Owner | All Project Editor permissions plus permissions to manage access control for the project (get/set IamPolicy) and to set up project billing |
What's next
- Learn how to Manage access to projects, folders, and organizations.