Manage teams for your fleet

A common task for platform admins is making sure that application and service teams have the infrastructure resources they need to run their workloads. Depending on your organization, teams may need to use specific clusters, or they may need to run workloads on every cluster in your fleet, with appropriate access control set up for each team. Fleet team management features make it easier for admins to provision and manage infrastructure resources like this for teams, with each team acting as a separate "tenant" on your fleet.

This page is for platform admins who want to set up and manage fleet usage for a team. Fleet team management features are only available for users who have enabled the entire Anthos platform.

Fleet team management overview

Fleet team management is based around two key concepts that give admins a "team-level" abstraction to use when managing fleets:

Team scopes let you define subsets of fleet resources on a per-team basis, with each scope associated with one or more fleet member clusters. Scopes can include clusters on Google Cloud or outside Google Cloud, though all the clusters must be members of the same fleet. A cluster can be associated with more than one team scope, letting different teams run workloads on the same cluster.
Fleet namespaces provide a way to control who has access to specific namespaces within your fleet. By default, any namespaces with the same name defined on clusters in the fleet are treated as if they were the same namespace. However, fleet team management provides a way to add more granular control over namespaces. You can create fleet namespaces within specific scopes, and then use Kubernetes role-based access control (RBAC) to grant access to team members to those namespaces only on clusters within their team scope. Fleet namespaces can be used in the same way as any other Kubernetes namespace on the member clusters in the scope.

The general procedure for setting up a team is as follows:

Select or create the fleet where you want to set up team access.
Create a scope for the team.
Add one or more (or all) fleet member clusters to the scope.
Define fleet-level namespaces and associate them with the team scope.
Grant the team members access to the namespaces using an RBACRoleBinding resource.

Once setup is complete, we recommend that team members access their namespaces with kubectl using the special cluster credentials for the Connect Gateway. The Connect Gateway is a consistent, secured service that lets users log in with their Google IDs to any cluster in the fleet, including using Google Groups for authorization. While this isn't strictly required to authenticate to GKE clusters on Google Cloud, using the gateway credentials provides a simple, consistent way to authenticate to fleet member clusters, even across projects. Fleet team management does not currently support third-party identity providers.

Before you begin

Ensure that you have the latest version of the Google Cloud CLI, including the Google Cloud CLI alpha component. You need at least version 419.0.0 to use fleet team management commands.
Run the following command to log in to Google Cloud:
```
gcloud auth login
```
Install the alpha component if you don't have it already:
```
gcloud components install alpha
```
Select or create the fleet where you want to set up a new team. For guidelines and examples to help you structure your fleets, see Fleet examples and the other guides in Plan your fleet. If you want to create a new named fleet in a project that doesn't already have one, run the following command:
```
gcloud alpha container fleet create \
  --display-name=NAME \
  --project=FLEET_HOST_PROJECT_ID
```
If you don't specify a display-name, the new fleet is created with a default display name based on the fleet host project name.
Either initialize the gcloud CLI for use with your chosen fleet's host project, or run the following command to set the fleet host project as the default:
```
gcloud config set project PROJECT_ID
```
You can use the --project flag with any of the following commands to specify a different fleet host project, if required.

Required IAM roles

If you don't have roles/owner in the fleet host project, you need roles/gkehub.admin to create and configure scopes and namespaces. A project owner can grant this role with the following command:

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member user:USER_EMAIL_ADDRESS \
    --role='roles/gkehub.admin'

Enable APIs

Ensure that your fleet host project has all the required APIs enabled, including the Anthos API:

gcloud services enable --project=PROJECT_ID \
   gkehub.googleapis.com \
   container.googleapis.com \
   connectgateway.googleapis.com \
   cloudresourcemanager.googleapis.com \
   iam.googleapis.com \
   anthos.googleapis.com

If you disable the Anthos API after configuring fleet team management, some aspects of the feature will continue to work, but you will be unable to update or create scopes or fleet namespaces.

Configure clusters for access control with Google Groups

While you can configure a team's access using RBAC to fleet member clusters on a user-by-user basis without any additional cluster configuration, we recommend giving team members access to clusters on the basis of their membership of a team Google Group. Authorizing based on group membership means you don't have to set up separate authorization for each account, making policies simpler to manage and easier to audit, and removing the need to manually add/remove individual users from clusters when they join or leave the team. Use the following guides to ensure that the clusters you want to assign to team scopes can use Google Groups with the Connect Gateway for access control:

For GKE clusters on Google Cloud, follow the instructions in Configure Google Groups for RBAC.
For fleet member clusters outside Google Cloud, follow the instructions in Set up the Connect Gateway with Google Groups.

Work with team scopes

Use the following commands to create and manage scopes for teams.

Create a team scope

To create a new team scope in a fleet, run the following command, where SCOPE_NAME is the unique identifying name that you have chosen for your new scope:

gcloud alpha container fleet scopes create SCOPE_NAME

Add clusters to scopes

Only existing fleet members can be added to scopes. These instructions assume that the cluster you want to add to a scope or scopes is already a fleet member. If you need to add the cluster to your fleet, follow the instructions for your cluster type in Create your fleet to register the cluster. Ensure that the newly registered cluster is configured to use Google Groups for access control, as described above.

A fleet member cluster can be added to any number of scopes in its fleet host project.

To add a cluster to a scope, run the following command:

gcloud alpha container fleet memberships bindings create BINDING_NAME \
  --membership MEMBERSHIP_NAME \
  --scope  SCOPE_NAME \
  --location MEMBERSHIP_LOCATION

where:

BINDING_NAME is a name that represents the relationship between the cluster and the scope. We suggest using MEMBERSHIP_NAME-SCOPE_NAME.
MEMBERSHIP_NAME is the cluster's unique identifier within the fleet (typically the cluster name).
(optional) MEMBERSHIP_LOCATION is the cluster's membership location. If you omit this the value is global, which is the default for cluster registrations.

List scopes

To list all scopes in a fleet, run the following command:

gcloud alpha container fleet scopes list

To list all scopes associated with a cluster, run the following command:

gcloud alpha container fleet memberships bindings list --membership MEMBERSHIP_NAME

Remove clusters from scopes

To remove a cluster from a scope, run the following command:

gcloud alpha container fleet memberships bindings delete BINDING_NAME --membership MEMBERSHIP_NAME

Delete a scope

To delete a scope from your fleet, run the following command:

gcloud alpha container fleet scopes delete SCOPE_NAME

Work with fleet namespaces

Use the following commands to create and manage namespaces within team scopes.

Create a fleet namespace

To create a namespace in a scope, run the following command, where NAMESPACE_NAME is the unique name you have chosen for the namespace within the fleet, and SCOPE_NAME is the scope where you want to use the namespace:

gcloud alpha container fleet namespaces create NAMESPACE_NAME --scope=SCOPE_NAME

This command creates a Kubernetes namespace called NAMESPACE_NAME in each cluster in the scope. Team members can use NAMESPACE_NAME like any other Kubernetes namespace after you have granted them access. If you already have an existing namespace called NAMESPACE_NAME in the scope, it is considered part of the new fleet namespace.

List fleet namespaces

To list all the namespaces created using fleet namespaces create in a fleet, run the following command:

gcloud alpha container fleet namespaces list

Delete a fleet namespace

To delete a fleet namespace, run the following command:

gcloud alpha container fleet namespaces delete NAMESPACE_NAME

Grant team access

To give teams access to their namespaces and scopes, you must ensure they have both the appropriate Identity and Access Management (IAM) roles and Kubernetes role-based access control (RBAC) configured to work with fleet clusters.

Grant fleet access with IAM

First, give team members access to the fleet with IAM:

gcloud projects add-iam-policy-binding PROJECT_ID \
   --member=group:TEAM_EMAIL \
   --role=roles/gkehub.viewer

gcloud projects add-iam-policy-binding PROJECT_ID \
   --member=group:TEAM_EMAIL \
   --role=roles/gkehub.gatewayEditor

PROJECT_ID is the ID of your fleet host project
TEAM_EMAIL is the email address for the team's Google Group.

These commands let team members view fleet clusters in the Google Cloud console and use the Connect Gateway to log in to fleet member clusters with Google Groups-based authorization.

Grant namespace access with RBAC

Team members can then be granted more privileged access to their fleet namespaces using RBAC. Use the following command to grant a Google Group access to a namespace:

gcloud alpha container fleet namespaces rbacrolebindings create BINDING_NAME \
   --namespace NAMESPACE_NAME \
   --role=ROLE \
   --group=TEAM_EMAIL

where:

BINDING_NAME is a string name to represent this binding.
NAMESPACE_NAME is the namespace's unique identifier.
ROLE is the RBAC role you want to grant to the team's members, which can be admin, edit, or view.
TEAM_EMAIL is the email address for the team's Google Group.

If you need to grant an individual user access to a namespace, run the following command instead, where USER_EMAIL is the user's Google ID email address:

gcloud alpha container fleet namespaces rbacrolebindings create BINDING_NAME \
   --namespace NAMESPACE_NAME \
   --role=ROLE \
   --user=USER_EMAIL

Update namespace access

To update namespace access (for example, to grant team members a different role, or to update a group email address), use the corresponding update command:

gcloud alpha container fleet namespaces rbacrolebindings update BINDING_NAME \
   --namespace NAMESPACE_NAME \
   --role=ROLE \
   --group=TEAM_EMAIL

gcloud alpha container fleet namespaces rbacrolebindings update BINDING_NAME \
   --namespace NAMESPACE_NAME \
   --role=ROLE \
   --user=USER_EMAIL

where:

BINDING_NAME is a string name to represent this binding.
NAMESPACE_NAME is the namespace's unique identifier.
(optional) ROLE is the role you want to change
(optional) TEAM_EMAIL or USER_EMAIL is the email address you want to change.

Get credentials for team members

Once the setup is complete, team members can access the namespaces in their scope by getting the relevant cluster credentials. To get credentials for a fleet member cluster using the Connect Gateway, run the following command, where MEMBERSHIP_NAME is the cluster's fleet membership name:

gcloud container fleet memberships get-credentials  MEMBERSHIP_NAME

For more details, see Using the Connect Gateway.

Troubleshoot

If you cannot update or create fleet team management resources, ensure that the Anthos API is enabled. If you disable the Anthos API in your fleet host project after configuring fleet team management, the following occurs:

Any scopes and fleet namespaces that you have created continue to work as expected, but cannot be updated.
Existing scopes and fleet namespaces can be deleted.
No new scopes and fleet namespaces can be created.