Cloud Dataflow Access Control Guide

Overview

You can use Cloud Dataflow IAM roles to limit access for users within a project or organization, to just Cloud Dataflow-related resources, as opposed to granting users viewer, editor, or owner access to the entire Cloud Platform project.

This page focuses on how to use Cloud Dataflow's IAM roles. For a detailed description of IAM and its features, see the Google Cloud Identity and Access Management developer's guide.

Every Cloud Dataflow method requires the caller to have the necessary permissions. For a list of the permissions and roles Cloud Dataflow supports, see the following section.

Permissions and roles

This section summarizes the permissions and roles Cloud Dataflow IAM supports.

Required permissions

The following table lists the permissions that the caller must have to call each method:

Method	Required Permission(s)
dataflow.jobs.create	dataflow.jobs.create
dataflow.jobs.cancel	dataflow.jobs.cancel
dataflow.jobs.updateContents	dataflow.jobs.updateContents
dataflow.jobs.list	dataflow.jobs.list
dataflow.jobs.get	dataflow.jobs.get
dataflow.jobs.drain	dataflow.jobs.drain
dataflow.messages.list	dataflow.messages.list
dataflow.metrics.get	dataflow.metrics.get

Roles

The following table lists the Cloud Dataflow IAM roles with a corresponding list of all the permissions each role includes. Note that every permission is applicable to a particular resource type.

Role	Title	Description	Permissions	Lowest Resource
roles/ dataflow.admin	Dataflow Admin	Minimal role for creating and managing dataflow jobs.	compute.machineTypes.get dataflow.* resourcemanager.projects.get resourcemanager.projects.list storage.buckets.get storage.objects.create storage.objects.get storage.objects.list
roles/ dataflow.developer	Dataflow Developer	Provides the permissions necessary to execute and manipulate Cloud Dataflow jobs.	dataflow.* resourcemanager.projects.get resourcemanager.projects.list	Project
roles/ dataflow.viewer	Dataflow Viewer	Provides read-only access to all Cloud Dataflow-related resources.	dataflow.jobs.get dataflow.jobs.list dataflow.messages.* dataflow.metrics.* resourcemanager.projects.get resourcemanager.projects.list	Project
roles/ dataflow.worker	Dataflow Worker	Provides the permissions necessary for a Compute Engine service account to execute work units for a Cloud Dataflow pipeline.	compute.instanceGroupManagers.update compute.instances.delete compute.instances.setDiskAutoDelete dataflow.jobs.get logging.logEntries.create storage.objects.create storage.objects.get	Project

Note: The Cloud Dataflow Worker role (roles/dataflow.worker) provides the permissions (dataflow.workItems.lease, dataflow.workItems.update, and dataflow.workItems.sendMessage) necessary for a Compute Engine service account to execute work units for a Apache Beam pipeline. It should typically only be assigned to such an account, and only includes the ability to request and update work from the Cloud Dataflow service.

Creating jobs

In order to a create a job, roles/dataflow.admin includes the minimal set of permissions required to run and examine jobs.

Alternatively, the following permissions are required:

• The roles/dataflow.developer role, to instantiate the job itself.
• The roles/compute.viewer role, to access machine type information and view other settings.
• The roles/storage.objectAdmin role, to provide permission to stage files on Cloud Storage.

Example role assignment

To illustrate the utility of the different Cloud Dataflow roles, consider the following breakdown:

• The developer who creates and examines jobs will need the roles/dataflow.admin role.
• For more sophisticated permissions management, the developer interacting with the Cloud Dataflow job will need the roles/dataflow.developer role.
  • They will need the roles/storage.objectAdmin or a related role in order to stage the required files.
  • For debugging and quota checking, they will need the project roles/compute.viewer role.
  • Absent other role assignments, this will allow the developer to create and cancel Cloud Dataflow jobs, but not interact with the individual VMs or access other Cloud services.
• The controller service account needs the roles/dataflow.worker role to process data for the Cloud Dataflow service. It will need other roles (such as roles/storage.objectAdmin in order to access job data.

Assigning Cloud Dataflow roles

Cloud Dataflow roles can currently be set on organizations and projects only.

To manage roles at the organizational level, see Access Control for Organizations Using IAM.

To set project-level roles, see Access control via the Google Cloud Platform Console.