A management service creates, updates, and deletes GKE on AWS clusters. This topic explains how to create a management service inside a dedicated AWS Virtual Private Cloud (VPC). If you have an existing VPC, see Integrating with existing infrastructure instead.
Before you begin
Before you start using GKE on AWS, make sure that you have performed the following tasks:
- Complete the prerequisites.
-
Authenticate with the Google Cloud CLI.
gcloud auth login && \ gcloud auth application-default login
Values you need
To complete this topic, you need the following from the prerequisites:
- KMS key ARNs or aliases
- Google Cloud service account keys
- Google Cloud project
- The
aws
,terraform
, andanthos-gke
command-line tools installed and configured. - The AWS region and availability zones where GKE on AWS creates your management cluster.
Configure your management service
You configure your GKE on AWS management service with a YAML file. The file resembles a Kubernetes custom resource configuration, but it is not a representation of a resource.
Open a terminal on the computer where you installed and configured the
aws
,terraform
, andanthos-gke
command-line tools.Create an empty directory for your GKE on AWS configuration and change to this directory. GKE on AWS documentation uses
anthos-aws
as the example configuration directory.mkdir anthos-aws cd anthos-aws
Create a file named
anthos-gke.yaml
in a text editor. Paste the following contents into the file.apiVersion: multicloud.cluster.gke.io/v1 kind: AWSManagementService metadata: name: management spec: version: aws-1.14.1-gke.0 region: AWS_REGION authentication: awsIAM: adminIdentityARNs: - ADMIN_AWS_IAM_ARN kmsKeyARN: KMS_KEY_ARN databaseEncryption: kmsKeyARN: DATABASE_KMS_KEY_ARN googleCloud: projectID: GCP_PROJECT_ID serviceAccountKeys: managementService: MANAGEMENT_KEY_PATH connectAgent: HUB_KEY_PATH node: NODE_KEY_PATH dedicatedVPC: vpcCIDRBlock: VPC_CIDR_BLOCK availabilityZones: - ZONE_1 - ZONE_2 - ZONE_3 privateSubnetCIDRBlocks: - PRIVATE_CIDR_BLOCK_1 - PRIVATE_CIDR_BLOCK_2 - PRIVATE_CIDR_BLOCK_3 publicSubnetCIDRBlocks: - PUBLIC_CIDR_BLOCK_1 - PUBLIC_CIDR_BLOCK_2 - PUBLIC_CIDR_BLOCK_3 # Optional bastionHost: allowedSSHCIDRBlocks: - SSH_CIDR_BLOCK proxy: PROXY_JSON_FILE # optional
Replace the following values:
AWS_REGION with the AWS region to run your cluster in.
ADMIN_AWS_IAM_ARN with the Amazon Resource Name of the user with AWS IAM permissions to create a management service. To get the ARN of the user authenticated to the
aws
tool, runaws sts get-caller-identity
.KMS_KEY_ARN with the Amazon Resource Name of the AWS KMS key or KMS key alias that secures your management service's data during creation. For example,
arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
. If you do not have the ARN, runaws kms list-keys
to retrieve a list of ARNs.DATABASE_KMS_KEY_ARN with the Amazon Resource Name of the AWS KMS key or key alias that secures your management service's
etcd
databases—for example,arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
.GCP_PROJECT_ID with the Google Cloud project ID that hosts your GKE Enterprise environment.
MANAGEMENT_KEY_PATH with the location of your Google Cloud management service account key.
HUB_KEY_PATH with the location of your Google Cloud Connect service account key.
NODE_KEY_PATH with the location of your GKE on AWS node service account key.
VPC_CIDR_BLOCK with the total CIDR range of IP addresses for the AWS VPC that
anthos-gke
creates. For example,10.0.0.0/16
. For more information, see VPC and subnet basics in the AWS documentation.ZONE_1, ZONE_2, and ZONE_3 with the AWS EC2 availability zones where you want to create nodes and control planes. GKE on AWS creates subnets in these zones. When you use
anthos-gke
to generate configuration for a user cluster, GKE on AWS creates control planes and node pools in these availability zones—for example,us-east-1a
.
If you would like to useanthos-gke
to create user clusters in only one zone, you can remove ZONE_2, and ZONE_3.PRIVATE_CIDR_BLOCK_1, PRIVATE_CIDR_BLOCK_2, and PRIVATE_CIDR_BLOCK_3, with the CIDR block for your private subnet. GKE on AWS components such as the management service run in the private subnet. This subnet must be within the VPC's CIDR range specified in
vpcCIDRBlock
. You need one subnet for each availability zone— for example,10.0.1.0/24
.PUBLIC_CIDR_BLOCK_1, PUBLIC_CIDR_BLOCK_2, and PUBLIC_CIDR_BLOCK_3, with the CIDR blocks for your public subnet. You need one subnet for each availability zone. The public subnet exposes cluster services such as load balancers to the security groups and address ranges specified in AWS network ACLs and security groups— for example,
10.0.100.0/24
.SSH_CIDR_BLOCK with the CIDR block that allows inbound SSH to your bastion host—for example,
203.0.113.0/24
. If you want to allow SSH from any IP address, use0.0.0.0/0
.(optional) PROXY_JSON_FILE with the relative path of the proxy configuration file. If you are not using a proxy, delete this line.
Run
anthos-gke aws management init
to generate ananthos-gke.status.yaml
file with additional configuration. Theinit
command also validates theAWSManagementService
object in youranthos-gke.yaml
file.anthos-gke aws management init
Run
anthos-gke aws management apply
to create the management service on AWS.anthos-gke aws management apply
The
anthos-gke aws management apply
command might take up to ten minutes to complete. After the command completes, your management service runs on AWS.
Optional fields
The anthos-gke.yaml
file above shows a typical set of fields that most
customers will need. Configuration in anthos-gke.yaml
also supports
a number of optional fields. These include:
- spec.bootstrapS3Bucket to specify an AWS S3 bucket for GKE on AWS configuration data
- spec.tags to tag cluster-related AWS resources
- spec.securityGroupIDs to assign additional security group IDs to the management cluster
- spec.*Volume and its sub-fields volumeType, iops, and kmsKeyARN to tune EBS volume parameters
- spec.terraform.stateGCSBucket to specify a Google Cloud Service bucket for Terraform configuration data
For more information on all fields supported in anthos-gke.yaml
, see the
AWS Management Service reference.
Connect to the management service
Next, use anthos-gke
to connect and authenticate to your
GKE on AWS management service.
When you create a management service using the default settings, the control plane has a private IP address that isn't accessible from outside the AWS VPC. You can access your management service in any one of the following three ways:
- through Amazon's AWS Direct Connect service
- through a bastion host that proxies connections between the internet and your GKE on AWS subnets
- through a VPN
When you create a management service in a dedicated VPC, GKE on AWS automatically creates a bastion host in a public subnet. If you're connecting to your management service through a VPN or AWS Direct Connect, this bastion host is not needed. Otherwise, to connect to your management service through the bastion host, perform the following steps:
Use the
terraform
tool to generate the script that opens an SSH tunnel to the bastion host. Choose your version of Terraform, then run the following commands:Terraform 0.12, 0.13
terraform output bastion_tunnel > bastion-tunnel.sh chmod 755 bastion-tunnel.sh
Terraform 0.14.3+
terraform output -raw bastion_tunnel > bastion-tunnel.sh chmod 755 bastion-tunnel.sh
Terraform creates the
bastion-tunnel.sh
script that references the bastion host's SSH key at~/.ssh/anthos-gke
.To open the tunnel, run the
bastion-tunnel.sh
script. The tunnel forwards fromlocalhost:8118
to the bastion host.To open a tunnel to the bastion host, run the following command:
./bastion-tunnel.sh -N -4
Messages from the SSH tunnel appear in this window. When you are ready to close the connection, stop the process by using Control+C or closing the window.
Open a new terminal and change directory to the directory with your GKE on AWS configuration.
Generate a
kubeconfig
for authentication. Useanthos-gke
to append credentials to your configuration stored in~/.kube/config
.anthos-gke aws management get-credentials
Check that you're able to connect to the management service with
kubectl
.env HTTPS_PROXY=http://localhost:8118 \ kubectl cluster-info
The output includes the URL for the management service API server.
What's next
- Create a user cluster.
- Use a proxy with GKE on AWS.
- Change your
kubectl
configuration to connect to GKE on AWS with fewer command-line options.