Using Airflow UI Access Control

Cloud Composer 1 | Cloud Composer 2

This page describes Airflow UI Access Control (also called Airflow Role-Based Access Control, or Airflow RBAC) in Cloud Composer. This feature provides an additional mechanism to separate users in the Airflow UI and DAG UI of your environment.

Overview of Airflow UI access control in Cloud Composer

Cloud Composer supports the community version of Airflow UI Access Control. This functionality allows to reduce DAG visibility in Airflow UI and DAG UI based on user role.

Access to Airflow UI and DAG UI and visibility of data and operations in those UIs is controlled at two levels in Cloud Composer:

  1. Access to the Airflow UI and DAG UI in Cloud Composer is controlled by IAM.

    Once users have access to the Airflow UI, IAM does not provide any additional fine-grained permission control in the Airflow UI or DAG UI.

  2. Apache Airflow Access Control model provides an additional layer that controls visibility and access within individual pages and DAGs in Airflow UI and DAG UI.

Before you begin

Enable the Airflow UI with Access Control

In Cloud Composer 2, the Airflow UI with Access Control is always enabled.

Manage Airflow roles and access control settings

Users with the Admin role (or equivalent) can view and modify access control settings in the Airflow UI.

In the Airflow UI, you can configure access control settings from the Security menu. For more information about the Airflow Access Control model, available permissions, and default roles, see the Airflow UI Access Control documentation.

Airflow maintains its own list of users. Users with the Admin role (or equivalent) can view the list of users who have opened the Airflow UI of an environment, and were registered in Airflow. This list also includes users manually preregistered by an Admin, as described in the following section.

Register users in the Airflow UI

New users are automatically registered when they open the Airflow UI of a Cloud Composer environment for the first time.

At registration, users are granted the role specified in the [webserver]rbac_user_registration_role Airflow configuration option. You can control the role of newly registered users by overriding this Airflow configuration option with a different value.

If not specified, the default registration role is Op in environments with Airflow 2.

The following steps are recommended for creating a basic role configuration for the Airflow UI:

  1. Environment administrators open the Airflow UI for the newly created environment.

  2. Grant the administrator accounts the Admin role. The default role for new accounts in environments with Airflow 2 is Op. To assign the Admin role, run the following Airflow CLI command with gcloud:

      gcloud composer environments run ENVIRONMENT_NAME \
        --location LOCATION \
        users add-role -- -e USER_EMAIL -r Admin
    

    Replace:

    • ENVIRONMENT_NAME with the name of the environment.
    • LOCATION with the region where the environment is located.
    • USER_EMAIL with the email of a user account.
  3. Admins can now configure access control for new users, including granting the Admin role to other users.

Remove users

Deleting a user from Airflow does not revoke access for that user, because they are automatically registered again next time they access the Airflow UI. To revoke access to the entire Airflow UI, remove the composer.environments.get permission from their allow policy for your project.

You can also change the user's role to Public, which keeps the user's registration, but removes all permissions for the Airflow UI.

Configure DAG-level permissions automatically

The Per-folder Roles Registration feature automatically creates a custom Airflow role for each subfolder directly inside the /dags folder and grants this role DAG-level access to all DAGs that have their source file stored in that respective subfolder. This streamlines management of custom Airflow roles and their access to DAGs.

How Per-folder Roles Registration works

Per-folder Roles Registration is an automated way of configuring roles and their DAG-level permissions. As such, it can cause conflicts with other Airflow mechanisms that grant DAG-level permissions:

To prevent such conflicts, enabling Per-folder Roles Registration also changes the behavior of these mechanisms.

In Airflow 2:

  • You can grant DAG access to roles through the access_control property defined in DAG source code.
  • Manually granting DAG permissions (through Airflow UI or gcloud CLI) can cause conflicts. For example, if you manually grant DAG-level permissions to a per-folder role, these permissions can be removed or overwritten when the DAG processor synchronizes a DAG. We recommend not to grant DAG permissions manually.
  • Roles have a union of DAG access permissions registered through Per-folder Roles Registration and defined in the access_control property of the DAG.

DAGs located directly in the top-level /dags folder are not auto-assigned to any per-folder role. They are not accessible with any per-folder role. Other roles like Admin, Op, User or any custom role that is granted permissions can access them through the Airflow UI and DAG UI.

If you have DAGs in subfolders with names that match the built-in Airflow roles (Admin, Op, User, Viewer, Public), or the NoDags and UserNoDags roles, then permissions to DAGs in these subfolders are still assigned to these roles. For example, uploading a DAG to the /dags/Admin folder grants permissions to this DAG to the Admin role.

Airflow performs per-folder roles registration when it processes DAGs in the Airflow scheduler. If there are more than a hundred DAGs in your environment, you might see an increase in DAG parsing time. If this is the case, we recommend to use more memory and CPU for schedulers. You can also increase the value of the [scheduler]parsing_processes Airflow configuration option.

Auto-assign DAGs to per-folder roles

To auto-assign DAGs to per-folder roles:

  1. Override the following Airflow configuration option:

    Section Key Value
    webserver rbac_autoregister_per_folder_roles True
  2. Change the new user registration role to a role without access to any DAGs. In this way, new users do not have access to any DAGs until an Admin assigns a role that has permissions for specific DAGs to their accounts.

    UserNoDags is a role created by Cloud Composer. It is an equivalent to the User role but without access to any DAGs.

    Override the following Airflow configuration option:

    Section Key Value
    webserver rbac_user_registration_role UserNoDags

  3. Make sure that users are registered in Airflow.

  4. Assign roles to users using one of the approaches:

    • Let Airflow automatically create roles based on the DAGs subfolders, then assign users to these roles.
    • Pre-create empty roles for the DAGs subfolders, with role names matching the name of a subfolder, then assign users to these roles. For example, for the /dags/CustomFolder folder, create a role named CustomFolder.
  5. Upload DAGs to subfolders with names that match the roles assigned to users. These subfolders must be located inside the /dags folder in the environment's bucket. Airflow adds permissions to DAGs in such a subfolder, so that only users with the corresponding role can access them through the Airflow UI and DAG UI.

Configure DAG-level permissions manually

You can configure DAG-level permissions for custom roles to specify which DAGs are visible for specific user groups.

To configure DAG-level permissions in Airflow UI:

  1. The Admin creates empty roles for grouping DAGs.
  2. The Admin assigns users to appropriate roles.
  3. The Admin or users assign DAGs to roles.
  4. In Airflow UI, users can only see DAGs assigned to their group.

DAGs can be assigned to roles either through DAG properties, or from the Airflow UI.

Assigning DAGs to roles in the Airflow UI

An Admin can assign the required DAG-level permissions to appropriate role(s) in the Airflow UI.

This operation is not supported in the DAG UI.

Assigning DAGs to roles in DAG properties

You can set the access_control DAG parameter on a DAG, specifying the DAG-grouping role(s) to which the DAG is assigned:

dag = DAG(
  access_control={
    'DagGroup': {'can_edit', 'can_read'},
  },
  ...
  )

In Airflow versions earlier than 2.1.0, the Admin, DAG developer, or an automated process must run the sync_perm Airflow command to apply the new access control settings.

In Airflow 2.1.0 and later versions, running this command is no longer required, because the scheduler applies DAG-level permissions when it parses a DAG.

Map audit logs in Airflow UI to users

Audit logs in Airflow UI are mapped to numeric IDs of Google user accounts. For example, if a user pauses a DAG, an entry is added to the logs.

You can view audit logs on the Browse > Audit Logs page in the Airflow UI.

An entry on the Audit Logs page in Airflow 2
Figure 1. An entry on the Audit Logs page in Airflow 2

A typical entry lists a numeric ID in the Owner field: accounts.google.com:NUMERIC_ID. You can map numeric IDs to user emails on the Security > List Users page. This page is available for users with the Admin role.

Note that the relationship between Google identities (email addresses) and user accounts (user IDs) is not fixed.

What's next