This page shows you how to set up a Google Distributed Cloud hybrid cluster in High Availability (HA) mode using Virtual Machines (VMs) running on Compute Engine.
You can try out Google Distributed Cloud quickly and without having to prepare any hardware. Completing the steps on this page provides you with a working Google Distributed Cloud test environment that runs on Compute Engine.
To try Google Distributed Cloud on Compute Engine VMs, complete the following steps:
- Create six VMs in Compute Engine
- Create a
vxlan
network between all VMs with L2 connectivity - Install prerequisites for Google Distributed Cloud
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
- Make a note of the project ID because you need it to set an environment variable that is used in the scripts and commands on this page. If you selected an existing project, make sure that you are either a project owner or editor.
-
On your Linux workstation, make sure you have installed the latest
Google Cloud CLI, the command line tool for
interacting with Google Cloud. If you already have gcloud CLI
installed, update its components by running the following command:
gcloud components update
Depending on how the gcloud CLI was installed, you might see the following message: "You cannot perform this action because the Google Cloud CLI component manager is disabled for this installation. You can run the following command to achieve the same result for this installation:" Follow the instructions to copy and paste the command to update the components.
The steps in this guide are taken from the installation script in the
anthos-samples
repository. The
FAQ section
has more information on how to customize this script to work with some popular
variations.
Create six VMs in Compute Engine
Complete these steps to create the following VMs:
- One VM for the admin workstation. An admin workstation hosts command-line interface (CLI) tools and configuration files to provision clusters during installation, and CLI tools for interacting with provisioned clusters post-installation. The admin workstation will have access to all the other nodes in the cluster via SSH.
- Three VMs for the three control plane nodes needed to run the Google Distributed Cloud control plane.
- Two VMs for the two worker nodes needed to run workloads on the Google Distributed Cloud cluster.
Setup environment variables:
export PROJECT_ID=PROJECT_ID export ZONE=ZONE export CLUSTER_NAME=CLUSTER_NAME export BMCTL_VERSION=1.30.200-gke.101
For the
ZONE
, you can useus-central1-a
or any of the other Compute Engine zones .Run the following commands to log in with your Google account and set your project as the default:
gcloud auth login gcloud config set project $PROJECT_ID gcloud config set compute/zone $ZONE
Create the
baremetal-gcr
service account and key:Enable Google Cloud APIs and services:
Give the
baremetal-gcr
service account additional permissions to avoid needing multiple service accounts for different APIs and services:Create the variables and arrays needed for all the commands on this page:
Use the following loop to create six VMs:
This command creates VM instances with the following names:
- abm-ws: The VM for the admin workstation.
- abm-cp1, abm-cp2, abm-cp3: The VMs for the control plane nodes.
- abm-w1, abm-w2: The VMs for the nodes that run workloads.
Use the following loop to verify that SSH is ready on all VMs:
Create a vxlan
network with L2 connectivity between VMs
Use the standard vxlan
functionality of Linux to create a network that
connects all the VMs with L2 connectivity.
The following command contains two loops that perform the following actions:
- SSH into each VM.
- Update and install needed packages.
Execute the required commands to configure the network with
vxlan
.
You now have L2 connectivity within the 10.200.0.0/24 network. The VMs have the following IP addresses:
- Admin workstation VM: 10.200.0.2
- VMs running the control plane nodes:
- 10.200.0.3
- 10.200.0.4
- 10.200.0.5
- VMs running the worker nodes:
- 10.200.0.6
- 10.200.0.7
Install prerequisites for Google Distributed Cloud
You need to install the following tools on the admin workstation before installing Google Distributed Cloud:
bmctl
kubectl
- Docker
To install the tools and prepare for Google Distributed Cloud installation:
Run the following commands to download the service account key to the admin workstation and install the required tools:
Run the following commands to ensure that
root@10.200.0.x
works. The commands perform these tasks:- Generate a new SSH key on the admin workstation.
- Add the public key to all the other VMs in the deployment.
Deploy a Google Distributed Cloud hybrid cluster
The following code block contains all commands and configurations needed to complete the following tasks:
- Create the configuration file for the needed hybrid cluster.
- Run the preflight checks.
- Deploy the cluster.
Verify your cluster
You can find your cluster's kubeconfig
file on the admin workstation in the
bmctl-workspace
directory of the root account. To verify your deployment,
complete the following steps.
SSH into the admin workstation as root:
gcloud compute ssh root@abm-ws --zone ${ZONE}
You can ignore any messages about updating the VM and complete this tutorial. If you plan to keep the VMs as a test environment, you might want to update the OS or upgrade to the next release as described in the Ubuntu documentation.
Set the
KUBECONFIG
environment variable with the path to the cluster's configuration file to runkubectl
commands on the cluster.export clusterid=CLUSTER_NAME export KUBECONFIG=$HOME/bmctl-workspace/$clusterid/$clusterid-kubeconfig kubectl get nodes
Set the current context in an environment variable:
export CONTEXT="$(kubectl config current-context)"
Run the following
gcloud
command. This command:- Grants your user account the Kubernetes
clusterrole/cluster-admin
role on the cluster. - Configures the cluster so that you can run
kubectl
commands on your local computer without having to SSH to the admin workstation.
Replace
GOOGLE_ACCOUNT_EMAIL
with the email address that is associated with your Google Cloud account. For example:--users=alex@example.com
.gcloud container fleet memberships generate-gateway-rbac \ --membership=CLUSTER_NAME \ --role=clusterrole/cluster-admin \ --users=GOOGLE_ACCOUNT_EMAIL \ --project=PROJECT_ID \ --kubeconfig=$KUBECONFIG \ --context=$CONTEXT\ --apply
The output of this command is similar to the following, which is truncated for readability:
Validating input arguments. Specified Cluster Role is: clusterrole/cluster-admin Generated RBAC policy is: -------------------------------------------- ... Applying the generate RBAC policy to cluster with kubeconfig: /root/bmctl-workspace/CLUSTER_NAME/CLUSTER_NAME-kubeconfig, context: CLUSTER_NAME-admin@CLUSTER_NAME Writing RBAC policy for user: GOOGLE_ACCOUNT_EMAIL to cluster. Successfully applied the RBAC policy to cluster.
- Grants your user account the Kubernetes
When you are finished exploring, enter exit to log out of the admin workstation.
Get the
kubeconfig
entry that can access the cluster through the Connect gateway.gcloud container fleet memberships get-credentials CLUSTER_NAME
The output is similar to the following:
Starting to build Gateway kubeconfig... Current project_id: PROJECT_ID A new kubeconfig entry "connectgateway_PROJECT_ID_global_CLUSTER_NAME" has been generated and set as the current context.
You can now run
kubectl
commands through the Connect gateway:kubectl get nodes kubectl get namespaces
Log in to your cluster from Google Cloud console
To observe your workloads on Google Distributed Cloud in the Google Cloud console, you need to log in to the cluster. Before you log in to the console for the first time, you need to configure an authentication method. The easiest authentication method to configure is Google identity. This authentication method lets you log in using the email address associated with your Google Cloud account.
The gcloud container fleet memberships generate-gateway-rbac
command that
you ran in the previous section configures the cluster so that you can log in
with your Google identity.
In the Google Cloud console, go to the GKE Clusters page.
Click
Actions next to the registered cluster, then click Login.Select Use your Google identity to log in.
Click Login.
Clean up
Connect to the admin workstation to reset the cluster VMs to their state prior to installation and unregister the cluster from your Google Cloud project:
gcloud compute ssh root@abm-ws --zone ${ZONE} << EOF set -x export clusterid=CLUSTER_NAME bmctl reset -c \$clusterid EOF
List all VMs that have
abm
in their name:gcloud compute instances list | grep 'abm'
Verify that you're fine with deleting all VMs that contain
abm
in the name.After you've verified, you can delete
abm
VMs by running the following command:gcloud compute instances list --format="value(name)" | grep 'abm' | xargs gcloud \ --quiet compute instances delete