In this topic, you configure an existing AWS
Virtual Private Cloud (VPC) with
public and private subnets and run anthos-gke
to complete the installation
of a management service.
Before you begin
To complete the following steps, you need the following:
AWS IAM permissions as described in Requirements.
An existing AWS VPC with:
- At least one public subnet.
- At least one private subnet.
- An internet gateway with a route to the public subnet.
- A NAT gateway with a route to the private subnet.
- DNS hostnames enabled.
- No custom value of
domain-name
in your DHCP options sets. For more information, see Troubleshooting. GKE on AWS does not support values other than the default EC2 domain names.
For more information on configuring your AWS VPC, see VPC with public and private subnets.
Your existing VPC's public and private subnet IDs. For example,
subnet-1234567890abcdefg
.For cluster versions 1.20 or lower, an AWS security group that allows SSH (port 22) inbound from the security groups or IP ranges where you will be managing your GKE on AWS installation.
For cluster versions 1.21 or higher, an AWS security group that allows Konnectivity (port 8132) connections outbound from the VPC IP ranges housing your GKE on AWS node pools.
Creating the workspace
Create a file named
anthos-gke.yaml
in a text editor. Paste the following contents into the file.apiVersion: multicloud.cluster.gke.io/v1 kind: AWSManagementService metadata: name: management spec: version: aws-1.14.1-gke.0 region: AWS_REGION authentication: awsIAM: adminIdentityARNs: - ADMIN_AWS_IAM_ARN kmsKeyARN: KMS_KEY_ARN databaseEncryption: kmsKeyARN: DATABASE_KMS_KEY_ARN securityGroupIDs: - SECURITY_GROUP_IDS googleCloud: projectID: GCP_PROJECT_ID serviceAccountKeys: managementService: MANAGEMENT_KEY_PATH connectAgent: HUB_KEY_PATH node: NODE_KEY_PATH existingVPC: subnetID: VPC_SUBNET_ID allowedSSHSecurityGroupIDs: - SSH_SECURITY_GROUP # Optional fields rootVolume: sizeGiB: ROOT_VOLUME_SIZE volumeType: ROOT_VOLUME_TYPE iops: ROOT_VOLUME_IOPS kmsKeyARN: ROOT_VOLUME_KEY bastionHost: subnetID: BASTION_HOST_SUBNET_ID allowedSSHCIDRBlocks: - SSH_CIDR_BLOCK rootVolume: # Optional kmsKeyARN: BASTION_VOLUME_KEY proxy: PROXY_JSON_FILE
Replace the following values:
- AWS_REGION with the AWS region to run your cluster in.
- ADMIN_AWS_IAM_ARN with the Amazon Resource Name of the admin AWS IAM key.
- KMS_KEY_ARN with the Amazon Resource Name of the AWS KMS key that secures your management service's data when the management service is created.
- DATABASE_KMS_KEY_ARN with the Amazon Resource Name of the
AWS KMS key that
secures your
etcd
databases. - SECURITY_GROUP_IDS with additional security group IDs allowed access to your management service VMs.
- GCP_PROJECT_ID with the Google Cloud project that hosts your GKE Enterprise environment.
- MANAGEMENT_KEY_PATH with the location of your Google Cloud management service account key.
- HUB_KEY_PATH with the location of your Google Cloud Connect service account key.
- NODE_KEY_PATH with the location of your GKE on AWS node service account key.
- VPC_SUBNET_ID with the subnet ID where the management service runs.
- SSH_SECURITY_GROUP with the list of security group IDs allowed SSH access within the management service instances.
Optional: If you want to encrypt your management service volumes, replace the following values:
- ROOT_VOLUME_SIZE with the volume size in gigabytes for the
management service. For example,
10
. - ROOT_VOLUME_TYPE with the
EBS volume type.
For example,
gp3
. - ROOT_VOLUME_IOPS with the mount of provisioned IO operations per
second (IOPS) for the volume. Only valid when
volumeType
isGP3
. For more information, see General Purpose SSD volumes (gp3). - ROOT_VOLUME_KEY with the Amazon Resource Name of the AWS KMS key that encrypts your management service instance root volumes. For more information, see Using a customer managed CMK to encrypt volumes.
Optional: If you want to create a bastion host to access your GKE on AWS environment with an SSH tunnel, replace the following values:
- BASTION_HOST_SUBNET_ID with the subnet ID where the bastion host runs. Make sure VPC_SUBNET_ID allows inbound connections from BASTION_HOST_SUBNET_ID.
- SSH_CIDR_BLOCK with the CIDR block that your
bastion host allows inbound SSH connection from. For example,
203.0.113.0/24
. If you want to allow SSH from any IP address, use0.0.0.0/0
. - BASTION_VOLUME_KEY with the Amazon Resource Name of the AWS KMS key that secures your bastion root volume.
Optional: If you want direct traffic through an HTTP proxy, replace the following value:
- PROXY_JSON_FILE with the relative path of the proxy configuration file. If you are not using a proxy, remove this line.
Run
anthos-gke aws management init
to create configuration files:anthos-gke aws management init
Run
anthos-gke aws management apply
to create the cluster.anthos-gke aws management apply
The
anthos-gke aws management apply
command might take up to ten minutes to complete. Whenanthos-gke
finishes, your management service runs on AWS.
Tag your subnets with your cluster name
If you are using existing AWS subnets with GKE on AWS and want to
create load balancers, you need to tag your VPC and subnets with your management
service's name. If you created your VPC with anthos-gke
or have already tagged
your subnets, skip this step.
To tag your subnets, perform the following steps:
Change to the directory with your GKE on AWS configuration. You created this directory when Installing the management service.
cd anthos-aws
Export your cluster ID as the environment variable
$CLUSTER-ID
. Choose your version of Terraform, then run the following commands:Terraform 0.12, 0.13
export CLUSTER_ID=$(terraform output cluster_id)
Terraform 0.14.3+
export CLUSTER_ID=$(terraform output -raw cluster_id)
Export your AWS VPC ID as the environment variable
$VPC_ID
. Choose your version of Terraform, then run the following commands:Terraform 0.12, 0.13
export VPC_ID=$(terraform output vpc_id)
Terraform 0.14.3+
export VPC_ID=$(terraform output -raw vpc_id)
Get your private subnet IDs with the
aws
CLI tool.aws ec2 describe-subnets \ --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=*private*" \ --query "Subnets[*].SubnetId" \ --output text
Tag your subnets with your cluster ID. Run the following command for each of your subnets.
aws ec2 create-tags \ --resources SUBNET_IDS \ --tags Key=gke:multicloud:cluster-id,Value=$CLUSTER_ID
Replace SUBNET_IDS with the list of subnet IDs, separated by spaces. For example,
subnet-012345678abcdef subnet-abcdef123456789 subnet-123456789abcdef
.
Connecting to the management service
In this section, you establish a connection to your management service. You can tunnel your commands through a bastion host. If you have a direct connection to your AWS VPC, skip to Generate credentials and connect.
Connecting with a bastion host
If you use a bastion host to set up the SSH tunnel, first
configure the host in your anthos-gke.yaml
. Next, create a script
to open the tunnel by performing the following steps:
Use
terraform
to generate a script that opens an SSH tunnel to the bastion host.terraform output bastion_tunnel > bastion-tunnel.sh chmod 755 bastion-tunnel.sh
Terraform creates the
bastion-tunnel.sh
script that uses the bastion host's SSH key (~/.ssh/anthos-gke
).To open the tunnel, run the
bastion-tunnel.sh
script. The tunnel forwards fromlocalhost:8118
to the bastion host.To open a tunnel to the bastion host, run the following command:
./bastion-tunnel.sh -N
Messages from the SSH tunnel appear in this window. When you are ready to close the connection, stop the process by using Control+C or closing the window.
Change to the directory with your GKE on AWS configuration.
cd anthos-aws
Generate credentials and connect
The documentation for GKE on AWS assumes that you use an SSH tunnel
on localhost
port 8118 to access your cluster. If you use another type of
connection to your VPC, such as a direct interconnect, VPN, or other method, you
can remove the line env HTTP_PROXY=http://localhost:8118
from commands.
Generate a
kubeconfig
for authentication. Useanthos-gke
to append credentials to your configuration stored in~/.kube/config
.anthos-gke aws management get-credentials
Check that you're able to connect to the management service with
kubectl
.env HTTPS_PROXY=http://localhost:8118 \ kubectl cluster-info
kubectl
prints the status of the cluster.
What's next
- Create a user cluster.
- Use a proxy with GKE on AWS.
- Change your
kubectl
configuration to connect to GKE on AWS with fewer command-line options. - See the minimum required security groups for your GKE on AWS management service and user clusters.