This page shows you how to manually add a cluster outside Google Cloud to your fleet. Because most Anthos cluster types outside Google Cloud are automatically registered at cluster creation time, this is typically used for registering third-party Kubernetes clusters (also known as attached clusters). However in rare circumstances you may need to update an on-premises Anthos cluster's registration. Clusters created on AWS or Azure with the Anthos Multi-Cloud API cannot be manually registered.
You can find out more about registration options in the fleet creation overview.
Before you begin
Ensure that you have followed the general prerequisites for registering a cluster.
All manual registrations outside Google Cloud also require the following:
- Ensure network connectivity for access to Google Cloud API and service endpoints.
- Set up the identity you want the Connect Agent to use to authenticate to Google Cloud. Attached clusters can be registered with fleet Workload Identity enabled if they meet the relevant requirements. Otherwise, attached clusters need a Google Cloud service account for authentication.
You may also have other special requirements for some attached cluster types.
Register your cluster
Registering clusters outside Google Cloud installs the Connect Agent in the cluster to manage the cluster's connection to its fleet. The agent needs an identity to authenticate to Google, which can be a Google Cloud service account or (recommended) fleet Workload Identity, as described in the prerequisites. You can find out much more about the Connect Agent in the Connect overview.
We recommend getting your cluster's membership status after you register it to ensure that it is properly connected to Google Cloud. See our troubleshooting guide if you have any issues with your registration.
To register or update a registration with the Connect Agent:
gcloud
Registering a cluster with the Google Cloud CLI requires the Google Cloud CLI version 281.0.0 or higher
These instructions apply to attached clusters, although they can also be used to update registration for Anthos clusters on VMware version 1.7 and earlier, Anthos clusters on AWS previous generation, and Anthos clusters on bare metal. Use gkectl
if you need to manually register an Anthos clusters on VMware admin cluster in version 1.8 onwards.
Run the following command:
gcloud container fleet memberships register MEMBERSHIP_NAME \ --context=KUBECONFIG_CONTEXT \ --kubeconfig=KUBECONFIG_PATH \ --service-account-key-file=SERVICE_ACCOUNT_KEY_PATH
Replace the following:
- MEMBERSHIP_NAME: the membership name that you choose and that is used to uniquely represent the cluster being registered to the fleet.
- SERVICE_ACCOUNT_KEY_PATH: the local filepath to the
service account's private key JSON file downloaded as part of Prerequisites.
This service account key is stored as a secret named
creds-gcp
in thegke-connect
namespace. - KUBECONFIG_CONTEXT: the cluster context of the cluster being registered
as it appears in the kubeconfig file. You can get this value from the command
line by running
kubectl config current-context
. - KUBECONFIG_PATH: the local filepath where your
kubeconfig containing an entry for the cluster being registered is stored.
This defaults to
$KUBECONFIG
if that environment variable is set; otherwise, this defaults to$HOME/.kube/config
.
Register an attached cluster with fleet Workload Identity
To register an attached cluster with fleet Workload Identity enabled, run one of the following commands. For more information about which attached cluster types can use this feature and any additional requirements, see Attached cluster prerequisites.
Get the OIDC provider URL for your cluster and ensure that it is publicly visible. If no provider exists, follow the instructions in Create an IAM OIDC provider for your cluster, and then run the command again.
aws eks describe-cluster --name MEMBERSHIP_NAME \ --region AWS_REGION \ --query "cluster.identity.oidc.issuer" \ --output text
Run the following command, replacing OIDC_URL with the URL returned by the previous command:
gcloud container fleet memberships register MEMBERSHIP_NAME \ --context=KUBECONFIG_CONTEXT \ --kubeconfig=KUBECONFIG_PATH \ --enable-workload-identity \ --public-issuer-url=OIDC_URL
Attach AKS, OpenShift, or other clusters
gcloud container fleet memberships register MEMBERSHIP_NAME \ --context=KUBECONFIG_CONTEXT \ --kubeconfig=KUBECONFIG_PATH \ --enable-workload-identity \ --has-private-issuer
gkectl
Update an Anthos clusters on VMware registration
From Anthos 1.8, you can only update an Anthos clusters on VMware registration by updating your cluster configuration with gkectl
. Do not use gkectl
to move an already registered cluster to a different fleet, as this can result in unwanted or unexpected behavior.
- To register an unregistered admin cluster to your fleet, ensure the
gkeConnect
section of your cluster configuration file is filled in correctly, and rungkectl update cluster
.
Console
Generate a registration command
You can use the Google Cloud console to help generate a gcloud
registration command for manually registering clusters. These instructions apply to attached clusters, as well as updating registration for Anthos clusters on VMware version 1.7 and earlier, Anthos clusters on AWS previous generation, and Anthos clusters on bare metal.
To register a cluster:
In the Google Cloud console, go to the Anthos Clusters page. This page shows all your registered clusters.
Click Register existing cluster.
Click Add external cluster.
Enter the name of the cluster that you want to register in the Cluster name field.
Optional: Add Google Cloud labels to your cluster.
Click Generate registration command.
In Cloud Shell or wherever you have saved your service account credentials, edit and run the
gcloud
command that is displayed on the page. You need to specify the following values:- The CLUSTER_CONTEXT is the cluster's context as it appears in the kubeconfig file. You can get this value from the command line by running
kubectl config current-context
. - The KUBECONFIG_PATH is the local filepath where your kubeconfig file is stored. This defaults to
$KUBECONFIG
if that environment variable is set; otherwise, it defaults to$HOME/.kube/config
. - The LOCAL_KEY_PATH is the path to your service account key file.
Running this command deploys the Connect Agent in your user cluster. When the Connect Agent connects to Google Cloud and your cluster is registered, a success message is displayed on the page.
- The CLUSTER_CONTEXT is the cluster's context as it appears in the kubeconfig file. You can get this value from the command line by running
Click Set labels, or click Skip if you didn't set any labels.
Advanced registration options (command line only)
Download the Connect Agent manifest
To download the Connect Agent installation manifest without deploying the agent, for example if you want to examine or edit the manifest before installation, pass in the --manifest-output-file
flag to the
gcloud container fleet memberships register
command. For example:
--manifest-output-file=[MANIFEST_FILE_PATH]
where [MANIFEST_FILE_PATH] is the local filepath where you want the Connect Agent installation manifest to be stored.
Using this option will not deploy the Connect Agent into the cluster. To deploy the Connect Agent, manually apply the downloaded manifest to your cluster.
Using a proxy server
To configure a proxy server, pass in the --proxy
flag to
gcloud container fleet memberships register
command. For example:
--proxy=[URL]
where [URL] is the proxy address.
The Connect Agent supports CONNECT-based HTTP and HTTPS proxies only, and accepts IP addresses and hostnames. Be sure to specify the protocol corresponding to the proxy type in the URL. For example, to pass in an HTTPS hostname:
--proxy=https://mycorpproxy.com:443
Unless you specify otherwise, Connect Agent uses port 3128 for the proxy.
If your proxy requires authorization, ensure that you pass in your credentials, such as:
--proxy=http://user:password@10.10.10.10:8888
Installing Connect Agent in a cluster with Windows and Linux nodes
The Connect Agent has to run on a Linux node. If you are installing in a mixed cluster with both Linux and Windows nodes, you can ensure that the Connect Agent is deployed on to a Linux node by adding an appropriate node selector to the deployment definition.
Run the following command to update the deployment with the appropriate node selector:
kubectl patch deployment \
$(kubectl get deployment -o=jsonpath='{.items[*].metadata.name}' -n gke-connect) \
-p '{"spec":{"template":{"spec":{"nodeSelector":{"kubernetes.io/os":"linux"}}}}}' -n gke-connect
To validate that the update succeeded run the following command:
kubectl get deployment -o=jsonpath='{.items[].spec.template.spec.nodeSelector}' -n gke-connect
The command should return:
{"kubernetes.io/os":"linux"}
Troubleshooting
If you run into any problems during this setup, see our troubleshooting guide.
What's next?
- Validate successful cluster registration and get other fleet membership details in Get fleet membership status
- Set up authentication to registered clusters:
- Log in to a cluster from the Google Cloud console