Registering a cluster

This page describes how to manually register your Kubernetes clusters with Google Cloud.

Before you begin

  1. Enable the required APIs in your Google Cloud project.
  2. Ensure network connectivity for access to Google Cloud API and service endpoints. If you are registering a GKE cluster running on Google Cloud, you have this by default.

  3. Google Cloud configuration

  4. Kubernetes configuration

Register a cluster

All manual cluster registration options require you to specify an identity for the Connect Agent to use when authenticating to Google, in most cases Google Cloud service account credentials. For GKE clusters, you can also configure the Connect Agent to use Workload Identity. Registering using Workload Identity lets the Connect Agent and other workloads in your cluster authenticate to Google without requiring you to download, manually rotate, and generally manage Google Cloud service account keys. By using Workload Identity, the Connect Agent and other workloads can authenticate using short-lived tokens generated by the cluster. Learn more about Workload Identity in Using Workload Identity.

Registration using Workload Identity for GKE clusters is currently available using the gcloud command-line tool only.

gcloud

Register a GKE cluster

While registering the cluster, you can configure the Connect Agent to either use Workload Identity or use a Google Cloud service account.

Registering a GKE cluster using Workload Identity (recommended)

  1. Determine if Workload Identity is enabled on your GKE cluster

    Run the following command to list which Workload Identity pool your cluster is a member of

    gcloud container clusters describe GKE_CLUSTER --format="value(workloadIdentityConfig.workloadPool)"
    

    Replace the following:

    • GKE_CLUSTER: the name of the GKE cluster for the current project.

    If you see a result similar to the following then Workload Identity is already enabled on your GKE cluster and you can proceed directly to "Register the Cluster" below.

    GKE_PROJECT_ID.svc.id.goog

    If there are no results, then Workload Identity is not enabled on the GKE cluster. Please continue to enable Workload Identity on your GKE cluster before proceeding to register the cluster.

  2. Enable Workload Identity

    Carefully follow the linked instructions to enable GKE Workload Identity. Enabling GKE Workload Identity changes authentication behavior for any new node pools created after it is enabled, which can affect applications that rely on service accounts attached to Compute Engine instances in the node pool.

  3. Register the Cluster

    Run either of the following commands to register the cluster:

    gcloud beta container hub memberships register MEMBERSHIP_NAME \
     --gke-uri=GKE_URI \
     --enable-workload-identity
    

    or

    gcloud beta container hub memberships register MEMBERSHIP_NAME \
     --gke-cluster=GKE_CLUSTER \
     --enable-workload-identity
    

    Replace the following:

    • MEMBERSHIP_NAME: the membership name that you choose to uniquely represent the cluster being registered to the fleet.
    • GKE_URI: the URI of the GKE cluster, for example: https://container.googleapis.com/v1/projects/my-gke-project/locations/us-central1-a/clusters/my-gke-cluster. You can obtain the URI by running gcloud container clusters list --uri.
    • GKE_CLUSTER: the location/name of the GKE cluster from the current project. The location can be a zone or a region, for example: us-central1-a/my-gke-cluster.

Registering a GKE cluster using Service Account

Run either of the following commands:

gcloud container hub memberships register MEMBERSHIP_NAME \
   --gke-uri=GKE_URI \
   --service-account-key-file=SERVICE_ACCOUNT_KEY_PATH

or

gcloud container hub memberships register MEMBERSHIP_NAME \
   --gke-cluster=GKE_CLUSTER \
   --service-account-key-file=SERVICE_ACCOUNT_KEY_PATH

Replace the following:

  • MEMBERSHIP_NAME: the membership name that you choose to uniquely represent the cluster being registered to the fleet.
  • SERVICE_ACCOUNT_KEY_PATH: the local filepath to the service account's private key JSON file downloaded as part of Prerequisites. This service account key is stored as a secret named creds-gcp in the gke-connect namespace.
  • GKE_URI: the URI of the GKE cluster, for example: https://container.googleapis.com/v1/projects/my-gke-project/locations/us-central1-a/clusters/my-gke-cluster. You can obtain the URI by running gcloud container clusters list --uri.
  • GKE_CLUSTER: the location/name of the GKE cluster from the current project. The location can be a zone or a region, for example: us-central1-a/my-gke-cluster.

Register any other cluster

These instructions apply to non-GKE, Anthos clusters on AWS, Anthos clusters on VMware version 1.7 and earlier, and any other Anthos clusters.

These instructions do not apply to Anthos clusters on VMware version 1.8. Starting with version 1.8, Anthos clusters on VMware no longer supports using gcloud to manually change the registration of a cluster.

Run the following command:

 gcloud container hub memberships register MEMBERSHIP_NAME \
   --context=KUBECONFIG_CONTEXT \
   --kubeconfig=KUBECONFIG_PATH \
   --service-account-key-file=SERVICE_ACCOUNT_KEY_PATH

Replace the following:

  • MEMBERSHIP_NAME: the membership name that you choose and that is used to uniquely represent the cluster being registered to the fleet.
  • SERVICE_ACCOUNT_KEY_PATH: the local filepath to the service account's private key JSON file downloaded as part of Prerequisites. This service account key is stored as a secret named creds-gcp in the gke-connect namespace.
  • KUBECONFIG_CONTEXT: the cluster context of the cluster being registered as it appears in the kubeconfig file. You can get this value from the command line by running kubectl config current-context.
  • KUBECONFIG_PATH: the local filepath where your kubeconfig containing an entry for the cluster being registered is stored. This defaults to $KUBECONFIG if that environment variable is set; otherwise, this defaults to $HOME/.kube/config.

Console

Register an Anthos on Google Cloud cluster

These instructions apply to GKE clusters on Google Cloud in projects that have enabled Anthos.

To register a cluster:

  1. In the Google Cloud Console, go to the Anthos Clusters page. This page shows all your registered clusters.

    Go to the Anthos Clusters page

  2. Click Register existing cluster.

  3. Click Register next to the unregistered cluster that you want to add to your fleet.

  4. Specify the service account that you want to use when registering the cluster. We recommend keeping the default behavior and creating a new service account.

    • If you select Create a new service account:
      • Specify a Service account ID to identify your new service account.
      • Choose if you want to download its service account key as a JSON file. We recommend keeping the default behavior and downloading the key because you won't be able to download it later.
    • If you select Use existing service account, paste in the account's service account key in JSON format.
  5. Optional: Specify a new membership name for your cluster. By default, a cluster's membership name is its current name. However, if your cluster's name is not unique within your project, you must specify a new unique membership name because all clusters require a unique identifier within their fleet.

  6. To register your cluster, click Submit.

Register any other cluster

These instructions apply to non-GKE, Anthos clusters on AWS, Anthos clusters on VMware version 1.7 and earlier, and any other Anthos clusters.

These instructions do not apply to Anthos clusters on VMware version 1.8. Starting with version 1.8, Anthos clusters on VMware no longer supports using gcloud to manually change the registration of a cluster.

To register a cluster:

  1. In the Google Cloud Console, go to the Anthos Clusters page. This page shows all your registered clusters.

    Go to the Anthos Clusters page

  2. Click Register existing cluster.

  3. Click Add external cluster.

  4. Enter the name of the cluster that you want to register in the Cluster name field.

  5. Optional: Add Google Cloud labels to your cluster.

  6. Click Generate registration command.

  7. In Cloud Shell or wherever you have saved your service account credentials, edit and run the gcloud command that is displayed on the page. You need to specify the following values:

    • The CLUSTER_CONTEXT is the cluster's context as it appears in the kubeconfig file. You can get this value from the command line by running kubectl config current-context.
    • The KUBECONFIG_PATH is the local filepath where your kubeconfig file is stored. This defaults to $KUBECONFIG if that environment variable is set; otherwise, it defaults to $HOME/.kube/config.
    • The LOCAL_KEY_PATH is the path to your service account key file.

    Running this command deploys the Connect Agent in your user cluster. When the Connect Agent connects to Google Cloud and your cluster is registered, a success message is displayed on the page.

  8. Click Set labels, or click Skip if you didn't set any labels.

Terraform

Only GKE clusters can be registered using Terraform.

You can find a complete reference for the gke_hub_membership resource used to configure registration in the terraform registry.

Register a GKE cluster

To register the cluster, use the following blocks in your configuration.

  1. Specify the google-beta provider as a required provider, as in the following snippet. The version should be above 3.62.0.

    terraform {
      required_providers {
        google-beta = {
          source = "hashicorp/google-beta"
          version = "3.67.0"
        }
      }
    }
    

    This provider is required if you want to use gke_hub_membership.

  2. Set the following default values for the provider:

    provider "google-beta" {
      credentials = file("SERVICE_ACCOUNT_KEY_PATH")
      project = "PROJECT_ID"
    }
    

    Replace the following:

    • SERVICE_ACCOUNT_KEY_PATH: the local filepath to the service account's private key JSON file downloaded as part of Prerequisites. This service account key is stored as a secret named creds-gcp in the gke-connect namespace. This service account key enables Terraform to access your Google Cloud account.
    • PROJECT_ID: the default project that you choose to provision your Terraform resources.
  3. Create a GKE cluster (optional)

    If you want to register an existing GKE Cluster, skip this step.

    resource "google_container_cluster" "TF_CLUSTER_RESOURCE_NAME" {
      provider = google-beta
      name               = "CLUSTER_NAME"
      location           = "ZONE"
      initial_node_count = 1
    }
    

    Replace the following:

    • TF_CLUSTER_RESOURCE_NAME: the name that you choose to uniquely identify the Terraform google_container_cluster resource created by this block.
    • CLUSTER_NAME: the name that you choose to uniquely represent the cluster you created.
    • ZONE: the default zone that you choose to provision your GKE cluster resource. For example: us-central1-a.
  4. Register a membership for the GKE cluster.

    resource "google_gke_hub_membership" "TF_MEMBERSHIP_RESOURCE_NAME" {
      provider = google-beta
      project_id = "HUB_PROJECT_ID"
      membership_id = "MEMBERSHIP_NAME"
      endpoint {
        gke_cluster {
         resource_link = "//container.googleapis.com/CLUSTER_RESOURCE_NAME"
        }
      }
    }
    

    Replace the following:

    • TF_MEMBERSHIP_RESOURCE_NAME: the name that you choose to uniquely identify the Terraform google_gke_hub_membership resource created by this block.
    • MEMBERSHIP_NAME: the membership name that you choose to uniquely represent the cluster being registered to the fleet.
    • CLUSTER_RESOURCE_NAME: the Google Cloud resource name for the GKE cluster. For example: projects/my-project/zones/us-west1-a/clusters/my-cluster. If the cluster is provisioned with Terraform, this is ${google_container_cluster.TF_CLUSTER_RESOURCE_NAME.id}.
    • HUB_PROJECT_ID: If this is not set, the cluster's membership will be created in the default project you specified earlier (PROJECT_ID). Set this field if you want to register the GKE cluster to a different project.

Enabling Workload Identity on a registered GKE cluster

Enabling Workload Identity uses the same configuration as registering a cluster, with the following additional steps.

  1. Enable GKE Workload Identity by adding the following in the relevant google_container_cluster resource block. You need to do this for both new and existing clusters.

    workload_identity_config {
      identity_namespace = "PROJECT_ID.svc.id.goog"
    }
    

    Replace the following:

    • PROJECT_ID: the default project that you choose to provision your Terraform resources.
  2. Enable Fleet Workload Identity by adding the following in the relevant google_gke_hub_membership resource block.

    authority {
      issuer = "https://container.googleapis.com/v1/CLUSTER_RESOURCE_NAME"
    }
    

    Replace the following:

    • CLUSTER_RESOURCE_NAME: the Google Cloud resource for the GKE cluster. For example: projects/my-project/zones/us-west1-a/clusters/my-cluster. If the cluster is provisioned with Terraform, this is ${google_container_cluster.TF_CLUSTER_RESOURCE_NAME.id}.

Config Connector

Only GKE clusters can be registered using Config Connector.

The version of Config Connector should be above 1.47.0. You can find a complete reference for the GKEHubMembership resource in the Config Connector Reference.

Register a GKE cluster

To register the cluster, first specify where you want to create your resources, following the instructions in the Config Connector guide. Then create a YAML file to register and (optionally) create a cluster, as follows:

Create and register a GKE cluster

  1. Create a GKE cluster:

    apiVersion: container.cnrm.cloud.google.com/v1beta1
    kind: ContainerCluster
    metadata:
      name: CLUSTER_NAME
    spec:
      location: ZONE
      initialNodeCount: 1
    

    Replace the following:

    • CLUSTER_NAME: the name that you chose to uniquely represent the cluster you created with Config Connector.
    • ZONE: the zone that you chose to provision your GKE cluster resource. For example: us-central1-a.
  2. Register a membership for the GKE cluster.

    apiVersion: gkehub.cnrm.cloud.google.com/v1beta1
    kind: GKEHubMembership
    metadata:
      name: MEMBERSHIP_NAME
    spec:
      location: global
      endpoint:
        gkeCluster:
          resourceRef:
            name: CLUSTER_NAME
    

    Replace the following:

    • MEMBERSHIP_NAME: the membership name that you chose to uniquely represent the cluster being registered to the fleet.
    • CLUSTER_NAME: the name that you chose to uniquely represent the cluster you created with Config Connector.

Register the cluster in a different project

Add the following to the metadata field of GKEHubMembership resource.

   metadata:
     annotations:
       cnrm.cloud.google.com/project-id: HUB_PROJECT_ID

Replace the following:

  • HUB_PROJECT_ID: the different project you chose to register the GKE cluster.

Register an existing cluster

Use the following configuration if you want to register any existing cluster, regardless of how it was created. In this case you need to specify the full resource name of the cluster so that Config Connector can find it. Replace the resourceRef field of GKEHubMembership resource with:

   resourceRef:
     external: //container.googleapis.com/CLUSTER_RESOURCE_NAME

Replace the following:

  • CLUSTER_RESOURCE_NAME: the Google Cloud resource name for the GKE cluster. For example: projects/my-project/zones/us-west1-a/clusters/my-cluster.

Enabling Workload Identity on a registered GKE cluster

Enabling Workload Identity uses the same configuration as registering a cluster, with the following additional steps.

  1. Enable GKE Workload Identity by adding the following in the spec field of ContainerCluster resource. You need to do this for both new and existing clusters.

    spec:
      workloadIdentityConfig:
        identityNamespace: PROJECT_ID.svc.id.goog
    

    Replace the following:

  2. Enable Fleet Workload Identity by adding the following in the spec field of GKEHubMembership resource block.

    spec:
      authority:
        issuer: https://container.googleapis.com/v1/projects/PROJECT_ID/locations/ZONE/clusters/CLUSTER_NAME
    }
    

    Replace the following:

    • PROJECT_ID: the default project you chose through namespace annotation to create your GKE cluster resource.
    • ZONE: the zone that you chose to provision your GKE cluster resource. For example: us-central1-a.
    • CLUSTER_NAME: the name that you chose to uniquely represent the cluster you created.

Advanced registration options (command line only)

Download the Connect Agent manifest

To download the Connect Agent installation manifest without deploying the agent, for example if you want to examine or edit the manifest before installation, pass in the --manifest-output-file flag to the gcloud container hub memberships register command. For example:

--manifest-output-file=[MANIFEST_FILE_PATH]

where [MANIFEST_FILE_PATH] is the local filepath where you want the Connect Agent installation manifest to be stored.

Using this option will not deploy the Connect Agent into the cluster. To deploy the Connect Agent, manually apply the downloaded manifest to your cluster.

Using a proxy server

To configure a proxy server, pass in the --proxy flag to gcloud container hub memberships register command. For example:

--proxy=[URL]

where [URL] is the proxy address.

The Connect Agent supports CONNECT-based HTTP and HTTPS proxies only, and accepts IP addresses and hostnames. Be sure to specify the protocol corresponding to the proxy type in the URL. For example, to pass in an HTTPS hostname:

--proxy=https://mycorpproxy.com:443

Unless you specify otherwise, Connect Agent uses port 3128 for the proxy.

If your proxy requires authorization, ensure that you pass in your credentials, such as:

--proxy=http://user:password@10.10.10.10:8888

Installing Connect Agent in a cluster with Windows and Linux nodes

The Connect Agent has to run on a Linux node. If you are installing in a mixed cluster with both Linux and Windows nodes, you can ensure that the Connect Agent is deployed on to a Linux node by adding an appropriate node selector to the deployment definition.

Run the following command to update the deployment with the appropriate node selector:

kubectl patch deployment \
$(kubectl get deployment -o=jsonpath='{.items[*].metadata.name}' -n gke-connect) \
-p '{"spec":{"template":{"spec":{"nodeSelector":{"kubernetes.io/os":"linux"}}}}}' -n gke-connect

To validate that the update succeeded run the following command:

kubectl get deployment -o=jsonpath='{.items[].spec.template.spec.nodeSelector}' -n gke-connect

The command should return:

{"kubernetes.io/os":"linux"}

What's next