A new version of Anthos clusters on AWS (GKE on AWS) was released on September 29. See the release notes for more information.

AWSManagementService

The AWSManagementServiceresource contains configuration information for your Anthos clusters on AWS (GKE on AWS) management service.

You configure a YAML file named anthos-gke.yaml before creating your management service. The anthos-gke tool uses the configuration when creating your management service.

The contents of your anthos-gke.yaml file depend on if you use anthos-gke to create a dedicated AWS Virtual Private Cloud (VPC) for you, or if you use an existing AWS VPC.

Dedicated VPC

apiVersion: multicloud.cluster.gke.io/v1
kind: AWSManagementService
metadata:
  name: management
spec:
  version: ANTHOS_CLI_VERSION
  region: AWS_REGION
  authentication:
    awsIAM:
      adminIdentityARNs:
      - ADMIN_AWS_IAM_ARN
  kmsKeyARN: KMS_KEY_ARN
  databaseEncryption:
    kmsKeyARN: KMS_KEY_ARN
  googleCloud:
    projectID: GCP_PROJECT_ID
    serviceAccountKeys:
      managementService: MANAGEMENT_KEY_PATH
      connectAgent: CONNECT_KEY_PATH
      node: NODE_KEY_PATH
  iam:
      permissionsBoundary: IAM_PERMISSIONS_BOUNDARY
      path: IAM_PATH
  dedicatedVPC:
    vpcCIDRBlock: VPC_CIDR_BLOCK
    availabilityZones:
    - ZONE_1
    - ZONE_2
    - ZONE_3
    privateSubnetCIDRBlocks:
    - PRIVATE_CIDR_BLOCK_1
    - PRIVATE_CIDR_BLOCK_2
    - PRIVATE_CIDR_BLOCK_3
    publicSubnetCIDRBlocks:
    - PUBLIC_CIDR_BLOCK_1
    - PUBLIC_CIDR_BLOCK_2
    - PUBLIC_CIDR_BLOCK_3
  # Optional section
  rootVolume:
    volumeType: ROOT_VOLUME_TYPE
    iops: ROOT_VOLUME_IOPS
    kmsKeyARN: ROOT_VOLUME_KEY
  dataVolume:
    volumeType: DATA_VOLUME_TYPE
    iops: DATA_VOLUME_IOPS
    kmsKeyARN: DATA_VOLUME_KEY
  ubuntuRepositoryMirror: 'packages.cloud.google.com'
  tags:
    tag-name: tag-value
  bastionHost:
    allowedSSHCIDRBlocks:
    - SSH_CIDR_BLOCK
    rootVolume:
      volumeType: ROOT_VOLUME_TYPE
      iops: ROOT_VOLUME_IOPS
      kmsKeyARN: ROOT_VOLUME_KEY
  bootstrapS3Bucket: BOOTSTRAP_S3_BUCKET
  proxy: PROXY_JSON_FILE
  terraform:
    invokeManually: No
    stateGCSBucket: GCS_BUCKET_NAME

Existing VPC

apiVersion: multicloud.cluster.gke.io/v1
kind: AWSManagementService
metadata:
  name: management
spec:
  version: ANTHOS_CLI_VERSION
  region: AWS_REGION
  authentication:
    awsIAM:
      adminIdentityARNs:
      - ADMIN_AWS_IAM_ARN
  kmsKeyARN: KMS_KEY_ARN
  databaseEncryption:
    kmsKeyARN: KMS_KEY_ARN
  securityGroupIDs:
    - SECURITY_GROUP_IDS
  googleCloud:
    projectID: GCP_PROJECT_ID
    serviceAccountKeys:
      managementService: MANAGEMENT_KEY_PATH
      connectAgent: CONNECT_KEY_PATH
      node: NODE_KEY_PATH
  iam:
      permissionsBoundary: IAM_PERMISSIONS_BOUNDARY
      path: IAM_PATH
  existingVPC:
    subnetID: [MANAGEMENT_SUBNET_ID]
    allowedSSHSecurityGroupIDs: [SSH_SECURITY_GROUPS]
  # Optional fields
  rootVolume:
    volumeType: ROOT_VOLUME_TYPE
    iops: ROOT_VOLUME_IOPS
    kmsKeyARN: ROOT_VOLUME_KEY
  dataVolume:
    volumeType: DATA_VOLUME_TYPE
    iops: DATA_VOLUME_IOPS
    kmsKeyARN: DATA_VOLUME_KEY
  ubuntuRepositoryMirror: 'packages.cloud.google.com'
  bastionHost:
    subnetID: [BASTION_HOST_SUBNET_ID]
    allowedSSHCIDRBlocks:
    - [SSH_CIDR_BLOCK]
    rootVolume:
      volumeType: ROOT_VOLUME_TYPE
      iops: ROOT_VOLUME_IOPS
      kmsKeyARN: ROOT_VOLUME_KEY
  bootstrapS3Bucket: BOOTSTRAP_S3_BUCKET
  tags:
    tag-name: tag-value
  proxy: PROXY_JSON_FILE
  terraform:
    invokeManually: No
    stateGCSBucket: GCS_BUCKET_NAME

The following sections describe these fields.

spec.version

The value of spec.version is the version of the management service anthos- gke creates. The version matches the output of anthos-gke version. For example, aws-1.9.0-gke.2.

spec.region

The value of spec.region is the AWS region where you want to deploy the management service. For example, us-east-1.

spec.authentication.awsIAM

The value of spec.authentication.awsIAM.adminIdentityARNs is a string with the Amazon resource name (ARN) of an AWS IAM user with permissions to create a Anthos clusters on AWS management service. For example, arn:aws:iam::123456789:user/someuser.

spec.kmsKeyARN

The value of spec.kmsKeyARN is a string with the Amazon resource name (ARN) or alias of the AWS KMS key created in Prerequisites. Anthos clusters on AWS uses the key for envelope encryption. For example, arn:aws:kms:us-west-1:123456789:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee.

spec.databaseEncryption

The value of spec.databaseEncryption.kmsKeyARN is a string with the Amazon resource name (ARN) or key alias of the AWS KMS key that Anthos clusters on AWS uses to encrypt application-layer secrets in clusters. For example, arn:aws:kms:us-west-1:123456789:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee.

spec.securityGroupIDs

The value of spec.securityGroupIDs is a list of additional AWS security group IDs assigned to the management service VM. For example [sg-00123456789abcd, sg-00123456789def1]`.

spec.dedicatedVPC

This field represents the desired VPC settings.

Name Description Type Example
vpcCIDRBlock The CIDR block of your desired AWS VPC. Must be between /16 and /28 bits. string 10.0.0.0/16
availabilityZones Availability zones to deploy control plane replicas in. Each zone will have its own subnet and needs a separate CIDR block. list(string) [us-east-1a, us-east-1b]
privateSubnetCIDRBlocks CIDR block for private subnets within the VPC in each availability zone. Needs to be a subset of vpcCIDRBlock. You need to have one subnet per availability zone. list(string) [10.0.101.0/24, 10.0.102.0/24]
publicSubnetCIDRBlocks CIDR block for public subnets within the VPC in each availability zone. Needs to be a subset of vpcCIDRBlock. You need to have one subnet per availability zone. list(string) [10.0.1.0/24, 10.0.2.0/24]
bastionAllowedSSHCIDRBlocks CIDR blocks to allow inbound SSH access from. This field is deprecated in Anthos clusters on AWS 1.5. Use spec.bastionHost in version 1.5 or higher. string [104.132.0.0/14]

spec.existingVPC

Fields under spec.existingVPC configure Anthos clusters on AWS to use an existing AWS VPC.

Name Description Type Example
subnetID Subnet ID where the management service runs. string subnet-00123456789ab
allowedSSHSecurityGroupIDs List of security group IDs allowed SSH access within the VPC. list(string) [sg-00123456789abcd, sg-00123456789def1]

spec.googleCloud

Fields under spec.googleCloud configure your Google Cloud project and IAM service account keys.

Name Description Type Example
projectID Google Cloud Cloud project. string my-gcp-project-id
serviceAccountKeys File paths to the service account keys. serviceAccountKeys

spec.googleCloud.serviceAccountKeys

Fields under spec.googleCloud.serviceAccountKeys include local file paths to the GCP service account keys. See Prerequisites for detailed descriptions of each key.

Name Description Type Example
managementService Service account key used by the management cluster string path/to/key/management-sa-key.json
connectAgent Service account key used by the GKE Hub Connect Agent string path/to/key/hub-sa-key.json
node Service account key used by the nodes string path/to/key/node-sa-key.json

spec.iam

Fields under spec.iam configure optional AWS IAM parameters for your Anthos clusters on AWS environment.

Name Description Type Example
permissionsBoundary ARN of the Permissions boundaries for IAM roles string arn:aws:iam::126285863215:policy/testPolicyBoundary
path Path to the IAM role string /org_abc/team_abc/member_abc/

spec.bastionHost

The spec.bastionHost section is optional and describes a bastion host.

A bastion host is an EC2 instance placed in a public subnet. It allows inbound SSH access from a specified CIDR block and has outbound SSH access into your Anthos clusters on AWS management service and user clusters.

You can create a bastion host for either dedicatedVPC or existingVPC installations. The allowedSSHCIDRBlocks field specifies the inbound range of addresses allowed, and must be non-empty.

If a bastion host does not exist, Anthos clusters on AWS creates a new bastion host.

  • For a dedicatedVPC installation, the Anthos clusters on AWS creates a subnet.
  • For an existingVPC installation, you need to provide a subnet for the bastion host and ensure that this subnet has connectivity into your VPC.

You can modify your bastion host by changing or deleting the spec.bastionHost field and running anthos-gke aws management init, followed by anthos-gke aws management apply.

Name Description Type Example
subnetID The ID of the subnet where the bastion host runs. For existingVPC installations, the subnet must allow ingress from where you will access it on port 22. Also, make sure your VPC allows inbound access from this subnet. For dedicatedVPC installations, do not specify this field. Anthos clusters on AWS will provision the subnet with the required access. string subnet-003bdf7b0f5e21e45
allowedSSHCIDRBlocks CIDR blocks to allow inbound SSH access from. list(string) [104.132.0.0/14]
rootVolume An object describing the bastion host's root volume. object {volumeType: gp2}

spec.rootVolume, spec.dataVolume, and spec.bastionHost.rootVolume

These following table describes common fields for the following objects:

  • spec.rootVolume, for control plane root disk volumes
  • spec.dataVolume, for control plane data disk volumes
  • spec.bastionHost.rootVolume, for bastion host root disk volumes
Name Description Type Example Required
volumeType The node's AWS EBS volume type. Can be gp2 (default) or gp3. string gp2 no
iops The amount of provisioned IO operations per second (IOPS) for volumes. Only valid when volumeType is GP3. For more information, see General Purpose SSD volumes (gp3). integer 5000 no
kmsKeyARN The ARN of the AWS KMS key used to encrypt the EBS volume. For more information, see Using a customer managed CMK to encrypt volumes. string arn:aws:kms:us-west-1:123456789:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee no

spec.ubuntuRepositoryMirror

The value of spec.ubuntuRepositoryMirror is an optional domain name for an apt package repository.

Name Description Type Example
ubuntuRepositoryMirror Optional domain name for an apt package repository. This field may only be set as an empty string or a Google-hosted mirror at packages.cloud.google.com. No other domains are allowed. To use the Google-hosted mirror, you must also enable Private Google Access. string packages.cloud.google.com

spec.tags

Name Description Type Example
tags Optional key/value metadata assigned to each AWS resource supporting the management service. map(string) {Environment: Production, Team: Analytics}

spec.proxy

The spec.proxy field configures an optional HTTP/HTTPS proxy.

Name Description Type Example
proxy Path to proxy configuration file. Relative path from anthos-gke.yaml proxy.json

spec.bootstrapS3Bucket

The spec.bootstrapS3Bucket field configures an S3 bucket name for Anthos clusters on AWS .

Name Description Type Example
bootstrapS3Bucket Optional field for the AWS S3 bucket name where Anthos clusters on AWS stores configuration information. If you want to create multiple Anthos clusters on AWS management services in the same AWS region, set this value to a unique bucket name. string example-s3-bucket

spec.terraform

The spec.terraform.invokeManually field configures the anthos-gke command to not create a management service.

Name Description Type Example
invokeManually Optional field that configures the anthos-gke command-line tool not to create a management service after building and validating your configuration. If you set invokeManually to Yes, you can edit and apply the Terraform configuration files manually. Yes, No Yes
stateGCSBucket Optional field that specifies a pre-existing Google Cloud Storage bucket for Terraform to store its configuration data. string example-GCS-bucket