Set up teams for your fleet

This page is for platform admins who want to set up and manage fleet usage for a team. Fleet team management features are only available for users who have enabled GKE Enterprise.

This page assumes that you have already read our Fleet team management overview.

Team setup overview

You can set up teams using the Google Cloud CLI, the Google Cloud console or Terraform.

The general procedure for setting up a team is as follows:

Select or create the fleet where you want to set up team access, and ensure that you have the correct permissions and APIs to complete setup.
(Optional but recommended) Set up access control for Google Groups on your fleet clusters.
Decide which users make up the team and ensure they have access to the fleet. A team can include Google Groups (recommended) and/or individual accounts.
Create a team scope for the team.
Add one or more (or all) fleet member clusters to the team scope.
Define fleet-level namespaces and associate them with the team scope.
Grant the team members access to their new scope using an RBACRoleBinding resource.
(Optional) Use Config Sync to sync Kubernetes resources to team scopes and namespaces.

The team can then get credentials to access their clusters using the Connect Gateway.

Set up the Google Cloud CLI

Even if you create team scopes using the Google Cloud console, you may still need to set up the gcloud CLI to complete some prerequisites while setting up your fleet, such as enabling required APIs.

Ensure that you have the latest version of the Google Cloud CLI, including the Google Cloud CLI alpha component. You need at least version 419.0.0 to use fleet team management commands.
Run the following command to log in to Google Cloud:
```
gcloud auth login
```
Either initialize the gcloud CLI for use with your chosen fleet's host project, or run the following command to set the fleet host project as the default:
```
gcloud config set project PROJECT_ID
```
You can use the --project flag with any of the following commands to specify a different fleet host project, if required.

Set up your fleet

Select or create the fleet where you want to set up a new team. For guidelines and examples to help you structure your fleets, see Fleet examples and the other guides in Plan your fleet.

If you want to create a new named fleet in a project that doesn't already have one, run the following command (you'll need to set up the Google Cloud CLI first):

  gcloud container fleet create \
    --display-name=NAME \
    --project=FLEET_HOST_PROJECT_ID

If you don't specify a display-name, the new fleet is created with a default display name based on the fleet host project name.

Required IAM roles

If you don't have roles/owner in the fleet host project, you need roles/gkehub.admin to create and configure team scopes and namespaces. A project owner can grant this role with the following command:

gcloud projects add-iam-policy-binding PROJECT_ID \
    --member user:USER_EMAIL_ADDRESS \
    --role='roles/gkehub.admin'

Enable APIs

Ensure that your fleet host project has all the required APIs enabled, including the GKE Enterprise API:

gcloud services enable --project=PROJECT_ID \
   gkehub.googleapis.com \
   container.googleapis.com \
   connectgateway.googleapis.com \
   cloudresourcemanager.googleapis.com \
   iam.googleapis.com \
   anthos.googleapis.com

If you disable the GKE Enterprise API after configuring fleet team management, some aspects of the feature will continue to work, but you will be unable to update or create team scopes or fleet namespaces.

Configure clusters for access control with Google Groups

While you can configure a team's access using RBAC to fleet member clusters on a user-by-user basis without any additional cluster configuration, we recommend giving team members access to clusters on the basis of their membership of a team Google Group. Authorizing based on group membership means you don't have to set up separate authorization for each account, making policies simpler to manage and easier to audit, and removing the need to manually add/remove individual users from clusters when they join or leave the team. Use the following guides to ensure that the clusters you want to assign to team scopes can use Google Groups with the Connect Gateway for access control:

For GKE clusters on Google Cloud, follow the instructions in Configure Google Groups for RBAC.
For fleet member clusters outside Google Cloud, follow the instructions in Set up the Connect Gateway with Google Groups.

Grant team members access to the fleet

Next, decide or discover which users make up the team and ensure they have access to the fleet. As mentioned in the previous section, we recommend that you grant team members access to their resources on the basis of Google Group membership, although team management also lets you grant access to individual users. Ensure that the relevant Google Groups or individuals have the appropriate Identity and Access Management (IAM) roles configured to work with fleet clusters:

gcloud projects add-iam-policy-binding PROJECT_ID \
   --member=group:TEAM_EMAIL \
   --role=roles/gkehub.viewer

gcloud projects add-iam-policy-binding PROJECT_ID \
   --member=group:TEAM_EMAIL \
   --role=roles/gkehub.gatewayEditor

PROJECT_ID is the ID of your fleet host project
TEAM_EMAIL is the email address for a team's Google Group.

These commands let team members view all fleet clusters in the Google Cloud console and (for Google Groups only) use the Connect Gateway to log in to fleet member clusters with Google Groups-based authorization.

Set up a new team

The following instructions show you how to create a new team scope for a team.

gcloud

Create a team scope

To create a new team scope in a fleet, run the following command, where SCOPE_NAME is the unique identifying name that you have chosen for your new scope:

gcloud container fleet scopes create SCOPE_NAME

Add clusters to a team scope

Only existing fleet members can be added to team scopes. These instructions assume that the cluster you want to add to the scope is already a fleet member. If you need to add the cluster to your fleet, follow the instructions for your cluster type in Create your fleet to register the cluster. Ensure that the newly registered cluster is configured to use Google Groups for access control, as described earlier.

A fleet member cluster can be added to any number of team scopes in its fleet host project.

To add a cluster to a team scope, run the following command:

gcloud container fleet memberships bindings create BINDING_NAME \
  --membership MEMBERSHIP_NAME \
  --scope  SCOPE_NAME \
  --location MEMBERSHIP_LOCATION

Replace the following:

BINDING_NAME: a name that represents the relationship between the cluster and the team scope. We suggest using MEMBERSHIP_NAME-SCOPE_NAME.
MEMBERSHIP_NAME: the cluster's unique identifier within the fleet (typically the cluster name).
(optional) MEMBERSHIP_LOCATION: the cluster's membership location. If you omit this the value is global, which is the default for cluster registrations.

Create fleet namespaces

To create a namespace in a team scope, run the following command:

gcloud container fleet scopes namespaces create NAMESPACE_NAME --scope=SCOPE_NAME

Replace the following:

NAMESPACE_NAME: the unique name you have chosen for the namespace within the fleet. Ensure that NAMESPACE_NAME does not conflict with the fleet namespace naming restrictions.
SCOPE_NAME: the team scope where you want to use the namespace.

This command creates a Kubernetes namespace called NAMESPACE_NAME in each cluster in the team scope. Team members can use NAMESPACE_NAME like any other Kubernetes namespace after you have granted them access to their scope. If you already have an existing Kubernetes namespace called NAMESPACE_NAME in the team scope, it is considered part of the new fleet namespace. This is sometimes referred to as onboarding the namespace.

Grant team scope access with RBAC

Team members can then be granted access to their scope using RBAC. Use the following command to grant a Google Group access to a team scope:

gcloud container fleet scopes rbacrolebindings create BINDING_NAME \
   --scope=SCOPE_NAME \
   --role=ROLE \
   --group=TEAM_EMAIL

Replace the following:

BINDING_NAME: a string name to represent this binding.
SCOPE_NAME: the team scope's unique identifier.
ROLE: the RBAC role you want to grant to the team's members, which can be admin, edit, or view.
TEAM_EMAIL: the email address for the team's Google Group.

If you need to grant an individual user access to a namespace, run the following command instead, where USER_EMAIL is the user's Google ID email address:

gcloud container fleet scopes rbacrolebindings create BINDING_NAME \
   --scope=SCOPE_NAME \
   --role=ROLE \
   --user=USER_EMAIL

Console

Create a team scope

With your fleet host project selected, go to the Teams section in the Google Cloud console.

Go to Teams
At the top of the page, click Create Team Scope.
In the Team Basics page, for Name, enter a unique name for your team scope. You won't be able to change this name once the team scope is created.
To add team members to the scope, click Add Team Member.
- For Type, select User to add an individual team member, or Group to add a Google Group (recommended).
- For User or Group, type in the email address of the team member or group.
- For Role, select Admin, Editor or Viewer to specify the access level of the team member or group to the scope.
- To add more team members, repeat the preceding three steps.
To create the team scope without adding clusters and namespaces at this stage, click Create Team Scope. Otherwise, continue to the following section to add clusters to the scope.

Add clusters to the team scope

To associate a cluster with a team scope, the cluster must be an existing fleet member. If you need to add the cluster to your fleet, follow the instructions for your cluster type in Create your fleet to register the cluster. Ensure that the newly registered cluster is configured to use Google Groups for access control, as described earlier.

A fleet member cluster can be added to any number of team scopes in its fleet host project, which lets different teams run workloads on the same cluster.

In the Team Basics page, after adding team members to your scope, click Continue.
In the Clusters page, you can select the fleet clusters to associate with this team scope. In the Clusters drop-down, check the clusters you want to add, and click OK.

Create fleet namespaces

Team members can use fleet namespaces like any other Kubernetes namespace. When you create a fleet namespace, a corresponding Kubernetes namespace is created in all clusters in the team scope, if it doesn't exist already.

In the Clusters page, after adding clusters to your team scope, click Continue.
In the Namespaces page, click Add Namespace.
- For Name, enter a unique name for the namespace within the fleet, or the name of an existing namespace if you want to onboard that namespace. Ensure that the name does not conflict with the fleet namespace naming restrictions.
To add more fleet namespaces to the scope, repeat the preceding step.
To create the team scope, click Create Team Scope. Once the team scope is created, you can view and edit your team scope if necessary by clicking on its name in the Teams section.

Terraform

This section shows you how to set up a new team using Terraform. For more information and other examples, see the reference documentation for the following resources:

Create a team scope

To create a team scope, you can use the following block in your Terraform configuration.

  resource "google_gke_hub_scope" "TF_SCOPE_RESOURCE_NAME" {
    scope_id = "SCOPE_NAME"
  }

Replace the following:

TF_SCOPE_RESOURCE_NAME: the name that you choose to uniquely identify the Terraform google_gke_hub_scope resource created by this block.
SCOPE_NAME: a unique identifying name for your team scope.

Add clusters to the scope

Only existing fleet members can be added to team scopes. If you need to add the cluster to your fleet, follow the instructions for your cluster type in Create your fleet to register the cluster. Ensure that the newly registered cluster is configured to use Google Groups for access control, as described earlier.

To add a cluster to a team scope, use the following block in your configuration:

  resource "google_gke_hub_membership_binding" "TF_MEMBERSHIP_BINDING_RESOURCE_NAME" {
    membership_binding_id = "BINDING_NAME"
    scope = SCOPE_NAME
    membership_id = MEMBERSHIP_NAME
    location = "MEMBERSHIP_LOCATION"
  }

Replace the following:

TF_MEMBERSHIP_BINDING_RESOURCE_NAME: a name to identify the google_gke_hub_membership_binding resource created by this block.
BINDING_NAME: a name that represents the relationship between the cluster and the scope. We suggest using MEMBERSHIP_NAME-SCOPE_NAME.
SCOPE_NAME: the name of your team scope.
MEMBERSHIP_NAME: the cluster's unique identifier within the fleet (typically the cluster name).
MEMBERSHIP_LOCATION: the cluster's membership location.

Create fleet namespaces

Team members can use fleet namespaces like any other Kubernetes namespace. You can create a new namespace, or onboard an existing one. When you create a fleet namespace, a corresponding Kubernetes namespace is created in all clusters in the team scope, if it doesn't exist already.

To create a fleet namespace, use the following block in your configuration:

  resource "google_gke_hub_namespace" "TF_NAMESPACE_RESOURCE_NAME" {
    scope_namespace_id = "NAMESPACE_NAME"
    scope_id = SCOPE_NAME
    scope = SCOPE_NAME
  }

Replace the following:

TF_NAMESPACE_RESOURCE_NAME: a name to identify the google_gke_hub_namespace resource created by this block.
NAMESPACE_NAME: a unique name that you have chosen for the fleet namespace. Ensure that this name does not conflict with the fleet namespace naming restrictions.
SCOPE_NAME: the name of the team scope in which the fleet namespace is created.

Grant scope access with RBAC

Team members can be granted access to their scope using RBAC. For example, here is a configuration to grant an individual user access to a team scope:

  resource "google_gke_hub_scope_rbac_role_binding" "TF_RBAC_ROLE_BINDING_NAME" {
    scope_rbac_role_binding_id = "BINDING_NAME"
    scope_id = SCOPE_NAME
    user = USER_EMAIL
    role {
      predefined_role = "ROLE"
    }
  }

Replace the following:

TF_RBAC_ROLE_BINDING_NAME: a name to identify the google_gke_hub_scope_rbac_role_binding resource created by this block.
BINDING_NAME: a name to represent this binding.
SCOPE_NAME: the name of the team scope.
USER_EMAIL: the user's email address.
ROLE: the RBAC role you want to grant to the user, which can be admin, edit, or view.

To grant a Google Group access to a team scope, use group instead of user in the preceding configuration, and use the email address for the team's Google Group.

Access fleet namespaces

Once the setup is complete, team members can access the namespaces in their scope by getting the relevant cluster credentials. To get credentials for a fleet member cluster using the Connect Gateway, run the following command, where MEMBERSHIP_NAME is the cluster's fleet membership name:

   gcloud container fleet memberships get-credentials  MEMBERSHIP_NAME

For more details, see Using the Connect Gateway.

Manage team scopes

Use the following commands to manage team scopes.

gcloud

List team scopes

To list all scopes in a fleet, run the following command:

gcloud container fleet scopes list

To list all scopes associated with a cluster, run the following command:

gcloud container fleet memberships bindings list --membership MEMBERSHIP_NAME

Remove clusters from team scopes

To remove a cluster from a scope, run the following command:

gcloud container fleet memberships bindings delete BINDING_NAME --membership MEMBERSHIP_NAME

Delete a team scope

To delete a scope from your fleet, run the following command:

gcloud container fleet scopes delete SCOPE_NAME

Console

List team scopes

To view all scopes in a fleet, with your fleet host project selected, go to the Teams section in the Google Cloud console.

Go to Teams

The Teams page shows you a list of all team scopes created for your fleet, and a summary of the resource utilization for each scope including number of errors and container restarts.

View team scope details

For each team scope, you can view the labels associated with that scope, team members in that scope, and logs associated with the scope.

On the Teams page, click the team scope whose details you want to view.
In the Team tab, you can see the scope labels, if any, and view the team members that belong to the scope.
Click the Logs tab to view fleet scope logs.

Add or delete clusters in a team scope

To add or delete clusters in an existing team scope:

Go to the Teams page in the Google Cloud console:

Go to Teams
Select the team scope in which you want to add or delete clusters. The Clusters tab shows you a list of the clusters currently bound to the scope.

To add clusters to a team scope:

At the top of the page, click Add Clusters.
In the Clusters drop-down, select the clusters you want to add to the scope, and click OK.
Click Update Team Scope.

To delete clusters from a team scope:

Select the Clusters tab which shows you a list of the clusters currently bound to the scope.
Click the Trash icon next to the cluster you want to delete, and click Remove to confirm the deletion.

Delete a scope

Go to the Teams page in the Google Cloud console:

Go to Teams
Select the team scope you want to delete.
To delete the scope, at the top of the page, click Delete.
Confirm the deletion by entering the name of your scope, and click Delete again.

Manage fleet namespaces

gcloud

Use the following commands to manage namespaces within team scopes.

List fleet namespaces

To list all the namespaces created using fleet scopes namespaces create in a scope, run the following command:

gcloud container fleet scopes namespaces list --scope=SCOPE_NAME

Delete a fleet namespace

To delete a fleet namespace, run the following command:

gcloud container fleet scopes namespaces delete NAMESPACE_NAME --scope=SCOPE_NAME

Note that what happens when you delete a fleet namespace depends on how you added the namespace:

If you created a new fleet namespace: This command deletes the fleet namespace. It also deletes any Kubernetes namespaces created as a result of creating the fleet namespace, together with their workloads.
If you onboarded an existing Kubernetes namespace: This command deletes the fleet namespace. The original namespace that you onboarded is not deleted.

Console

To manage fleet namespaces in your team scope:

Go to the Teams page in the Google Cloud console:

Go to Teams
Select the team scope whose fleet namespaces you want to manage.

List fleet namespaces

In your team scope, select the Namespaces tab which shows you a list of the namespaces created in this scope.

View namespace details

For each fleet namespace, you can view the labels associated with that namespace, and workloads and logs filtered by namespace.

Select the Namespaces tab which shows you a list of the fleet namespaces created in the team scope.
Click the fleet namespace whose details you want to view.
In the Details tab, you can see the fleet namespace and scope labels.
- To view workloads for this namespace, click View Workloads.
- In the Workloads page, you can see the workloads already filtered by the namespace and clusters associated with the team scope for that namespace.
In the Logs tab, you can view fleet scope logs by namespace.

Add fleet namespaces to a team scope

To add a new fleet namespace, at the top of the page, click Add Namespaces.
Enter the name of the new fleet namespace, ensuring that the name does not conflict with the fleet namespace naming restrictions. To add more namespaces, click Add Namespace.
Click Update Team Scope.

Delete a fleet namespace

Select the Namespaces tab which shows you a list of the fleet namespaces created in the team scope.
Click the Trash icon next to the namespace you want to delete.
Confirm the deletion by entering the name of your namespace, and click Delete again.

Note that what happens when you do this depends on how you added the namespace:

If you created a new fleet namespace: The fleet namespace is deleted. Any Kubernetes namespaces created as a result of creating the fleet namespace are also deleted, together with their workloads.
If you onboarded an existing Kubernetes namespace: The fleet namespace is deleted. However, the original namespace that you onboarded is not deleted.

Update a fleet namespace name

You cannot edit a fleet namespace once it's been created. If you need to update a fleet namespace name, delete the namespace, and create a new one in the team scope.

Manage team access

gcloud

To update team scope access (for example, to grant team members a different role, or to update a group email address), use the corresponding update command:

gcloud container fleet scopes rbacrolebindings update BINDING_NAME \
   --scope=SCOPE_NAME \
   --role=ROLE \
   --group=TEAM_EMAIL

gcloud alpha container fleet namespaces rbacrolebindings update BINDING_NAME \
   --scope=SCOPE_NAME \
   --role=ROLE \
   --user=USER_EMAIL

Replace the following:

BINDING_NAME: a string name to represent this binding.
SCOPE_NAME: the team scope's unique identifier.
(optional) ROLE: the role you want to change
(optional) TEAM_EMAIL or USER_EMAIL: the email address you want to change.

Console

Add or remove team members

To manage team members in a team scope:

Go to the Teams page in the Google Cloud console:

Go to Teams
Select the team scope whose members you want to manage.

To add new team members to the scope:

At the top of the page, click Add Team Members. Follow the instructions as detailed in the Create a team scope section.
Click Update Team Scope.

To remove team members from the scope:

In the Team tab, click on the Trash icon next to the team member you want to remove from the team scope.
Click Delete to confirm the deletion.

You cannot edit a team member's details in the Google Cloud console. To update scope access in the Google Cloud console (for example, to grant team members a different role, or to update a group email address), remove the team member from the scope, and add them again with the new details.

Delegate namespace management

Typically only platform admins can create both team scopes and fleet namespaces. However, as a platform admin you might want to delegate namespace creation and/or management to team admins. This allows team admins to create and manage fleet namespaces in their scopes.

To delegate namespace management, create a custom IAM role in your project with the following permissions:

gkehub.namespaces.get
gkehub.namespaces.create
gkehub.namespaces.update
gkehub.namespaces.delete
gkehub.namespaces.list

You can customize the role as you see fit. For example, if you don't want to allow deletion of fleet namespaces, then omit the gkehub.namespaces.delete permission from the custom role.

After you define the custom role, use the following command to explicitly grant permission to manage namespaces for a particular team scope:

gcloud container fleet scopes add-iam-policy-binding SCOPE_NAME \
    --member='user:USER_EMAIL' \
    --role='projects/PROJECT_ID/roles/CUSTOM_ROLE_NAME

Only the user identified by USER_EMAIL will be able to manage fleet namespaces for this team scope. Other users will receive an error.

Fleet namespace naming restrictions

The following names are reserved and forbidden for use when you create a fleet namespace in a team scope:

default
kube-system
gke-connect
kube-node-lease
kube-public
istio-system
gatekeeper-system
asm-system
config-management-system

Manage labels

To help you identify and manage your scopes, you can use the Google Cloud CLI to create and manage labels for your fleet namespaces and team scopes.

Labels added to a team scope are inherited by all fleet namespaces in the scope, which means they are attached to all Kubernetes namespaces in the scope's clusters. Labels added directly to a fleet namespace are attached only to its corresponding Kubernetes namespaces. If a team scope label and fleet namespace label have the same key, the team scope label takes precedence.

You can work on multiple key-value pairs at once by adding a comma-separated list of key-value pairs.

Manage fleet namespace labels

Create a fleet namespace with labels

To create a fleet namespace with labels, run the following command:

gcloud container fleet scopes namespaces create NAMESPACE_NAME \
    --scope SCOPE_NAME \
    --namespace-labels KEY=VALUE

Replace the following:

NAMESPACE_NAME: the unique name you have chosen for the namespace within the fleet.
SCOPE_NAME: the team scope where you want to use the namespace.
KEY: the key for the label's key-value pair.
VALUE: the value for the label's key-value pair.

Add or update labels for existing fleet namespaces

To add or update labels for an existing namespace run the following command:

gcloud container fleet scopes namespaces update NAMESPACE_NAME \
    --scope SCOPE_NAME \
    --update-namespace-labels KEY=VALUE

Delete fleet namespace labels

To delete a specific fleet namespace label, run the following command:

gcloud container fleet scopes namespaces update NAMESPACE_NAME \
    --scope SCOPE_NAME \
    --remove-namespace-labels KEY

Replace KEY with a comma separated list of the keys for the labels that you want to remove.

To delete all fleet namespace labels, run the following command:

gcloud container fleet scopes namespaces update NAMESPACE_NAME \
    --scope SCOPE_NAME \
    --clear-namespace-labels

Manage team scope labels

Create a team scope with labels

To create a scope with a label, run the following command:

gcloud container fleet scopes create SCOPE_NAME \
    --namespace-labels KEY=VALUE

Replace the following:

SCOPE_NAME: the unique identifying name that you have chosen for your new team scope.
KEY: the key for the label's key-value pair.
VALUE: the value for the label's key-value pair.

Add or update labels for existing team scopes

To add or update labels for an existing scope run the following command:

gcloud container fleet scopes update SCOPE_NAME \
    --update-namespace-labels KEY=VALUE

Delete team scope labels

To delete specific labels, run the following command:

gcloud container fleet scopes update SCOPE_NAME \
     --remove-namespace-labels KEY

Replace KEY with a comma separated list of the keys for the labels that you want to remove.

To delete all labels, run the following command:

gcloud container fleet scopes update SCOPE_NAME \
    --clear-namespace-labels

Troubleshoot

If you cannot update or create fleet team management resources, ensure that the GKE Enterprise API is enabled. If you disable the GKE Enterprise API in your fleet host project after configuring fleet team management, the following occurs:

Any team scopes and fleet namespaces that you have created continue to work as expected, but cannot be updated.
Existing team scopes and fleet namespaces can be deleted.
No new team scopes and fleet namespaces can be created.

What's next?

Learn how to view team-level metrics and other team-specific information in Use the team overview.
Learn how to use Config Sync to sync Kubernetes resources to team scopes and namespaces.