A new version of GKE on AWS was released on October 2. See the release notes for more information.

Integrating with existing infrastructure

In this topic, you configure an existing AWS Virtual Private Cloud (VPC) with public and private subnets and run anthos-gke to complete the installation of a management service.

Before you begin

To complete the following steps, you need the following:

  • AWS IAM permissions as described in Requirements.

  • An existing AWS VPC with:

    • At least one public subnet.
    • At least one private subnet.
    • An internet gateway with a route to the public subnet.
    • A NAT gateway with a route to the private subnet.
    • DNS hostnames enabled.

    For more information on configuring your AWS VPC, see VPC with public and private subnets.

  • Your existing VPC's public and private subnet IDs. For example, subnet-1234567890abcdefg.

  • Choose or create an AWS security group that allows SSH (port 22) inbound from the security groups or IP ranges where you will be managing your GKE on AWS clusters.

Creating the workspace

  1. Create a file named anthos-gke.yaml in a text editor. Paste the following contents into the file.

     apiVersion: multicloud.cluster.gke.io/v1
     kind: AWSManagementService
       name: management
       version: aws-1.5.0-gke.6
       region: AWS_REGION
           - ADMIN_AWS_IAM_ARN
       kmsKeyARN: KMS_KEY_ARN
         projectID: GCP_PROJECT_ID
           managementService: MANAGEMENT_KEY_PATH
           connectAgent: HUB_KEY_PATH
           node: NODE_KEY_PATH
         subnetID: VPC_SUBNET_ID
       # Optional
         - SSH_CIDR_BLOCK
       proxy: PROXY_JSON_FILE

    Replace the following values:

    • AWS_REGION with the AWS region to run your cluster in.
    • ADMIN_AWS_IAM_ARN with the Amazon Resource Name of the admin AWS IAM key.
    • KMS_KEY_ARN with the Amazon Resource Name of the AWS KMS key that secures your management service's data when the management service is created.
    • DATABASE_KMS_KEY_ARN with the Amazon Resource Name of the AWS KMS key that secures your etcd databases.
    • SECURITY_GROUP_IDS with additional security group IDs allowed access to your management service VMs.
    • GCP_PROJECT_ID with the Google Cloud project that hosts your Anthos environment.
    • MANAGEMENT_KEY_PATH with the location of your Google Cloud management service account key.
    • HUB_KEY_PATH with the location of your Google Cloud Connect service account key.
    • NODE_KEY_PATH with the location of your GKE on AWS node service account key.
    • VPC_SUBNET_ID with the subnet ID where the management service runs.
    • SSH_SECURITY_GROUP with the list of security group IDs allowed SSH access within the management service instances.
    • BASTION_HOST_SUBNET_ID with the subnet ID where the bastion host runs. Make sure VPC_SUBNET_ID allows inbound connections from BASTION_HOST_SUBNET_ID.
    • SSH_CIDR_BLOCK with the CIDR block that your bastion host allows inbound SSH connection from. For example, If you want to allow SSH from any IP address, use
    • PROXY_JSON_FILE with the relative path of the proxy configuration file.
  2. Run anthos-gke aws management init to create configuration files:

    anthos-gke aws management init
  3. Run anthos-gke aws management apply to create the cluster.

    anthos-gke aws management apply

    The anthos-gke aws management apply command might take up to ten minutes to complete. When anthos-gke finishes, your management service runs on AWS.

Tag your subnets with your cluster name

If you are using existing AWS subnets with GKE on AWS and want to create load balancers, you need to tag your VPC and subnets with your management service's name. If you created your VPC with anthos-gke or have already tagged your subnets, skip this step.

To tag your subnets, perform the following steps:

  1. Change to the directory with your GKE on AWS configuration. You created this directory when Installing the management service.

    cd anthos-aws

  2. Export your cluster ID as the environment variable $CLUSTER-ID:

      export CLUSTER_ID=$(terraform output cluster_id)
  3. Export your AWS VPC ID as the environment variable $VPC_ID:

      export VPC_ID=$(terraform output vpc_id)
  4. Get your private subnet IDs with the aws CLI tool.

      aws ec2 describe-subnets \
          --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=*private*" \
          --query "Subnets[*].SubnetId" \
          --output text
  5. Tag your subnets with your cluster ID. Run the following command for each of your subnets.

      aws ec2 create-tags \
          --resources SUBNET_IDS \
          --tags Key=gke:multicloud:cluster-id,Value=$CLUSTER_ID

    Replace SUBNET_IDS with the list of subnet IDs, separated by spaces. For example, subnet-012345678abcdef subnet-abcdef123456789 subnet-123456789abcdef.

Connecting to the management service

The documentation for GKE on AWS assumes that you use an SSH tunnel on localhost port 8118 to access your cluster. If you use another type of connection to your VPC, such as a direct interconnect, VPN, or other method, you can remove the line env HTTP_PROXY=http://localhost:8118 from commands.

  1. Generate a kubeconfig for authentication. Use anthos-gke to append credentials to your configuration stored in ~/.kube/config.

    anthos-gke aws management get-credentials
  2. Check that you're able to connect to the management service with kubectl.

    env HTTP_PROXY=http://localhost:8118 \
      kubectl cluster-info

If you want to use a bastion host to set up the SSH tunnel, first configure the host in your anthos-gke.yaml. Next, create a script to open the tunnel by performing the following steps:

  1. Use terraform to generate a script that opens an SSH tunnel to the bastion host.

    terraform output bastion_tunnel > bastion-tunnel.sh
    chmod 755 bastion-tunnel.sh

    Terraform creates the bastion-tunnel.sh script that references the bastion host's SSH key at ~/.ssh/anthos-gke.

  2. To open the tunnel, run the bastion-tunnel.sh script. The tunnel forwards from localhost:8118 to the bastion host.

    To open a tunnel to the bastion host, run the following command:

    ./bastion-tunnel.sh -N

    Messages from the SSH tunnel appear in this window. When you are ready to close the connection, stop the process by using Control+C or closing the window.

  3. Change to the directory with your GKE on AWS configuration.

    cd anthos-aws

What's next