The product described by this documentation, Anthos Clusters on AWS (previous generation), is now in maintenance mode. All new installs must use the current generation product, Anthos clusters on AWS.

AWSCluster

This topic describes the configuration options of the AWSCluster Custom Resource Definition.

Defining an AWSCluster

AWSCluster is a Kubernetes custom resource defined by GKE on AWS. This resource represents a GKE on AWS control plane.

To create a cluster from this template, copy the following YAML and complete the highlighted values defined in the spec.networking and spec.controlPlane field definitions. Then, apply the resource manifest to your management service.

apiVersion: multicloud.cluster.gke.io/v1
kind: AWSCluster
metadata:
  name: CLUSTER_NAME
spec:
  region: AWS_REGION
  networking:
    vpcID: VPC_ID
    podAddressCIDRBlocks: POD_ADDRESS_CIDR_BLOCKS
    serviceAddressCIDRBlocks: SERVICE_ADDRESS_CIDR_BLOCKS
    serviceLoadBalancerSubnetIDs: SERVICE_LOAD_BALANCER_SUBNETS
  controlPlane:
    version: GKE_VERSION # Latest version is 1.25.5-gke.2100
    instanceType: AWS_INSTANCE_TYPE
    keyName: SSH_KEY_NAME
    subnetIDs:
    - CONTROL_PLANE_SUBNET_IDS
    securityGroupIDs:
    - CONTROL_PLANE_SECURITY_GROUPS
    iamInstanceProfile: CONTROL_PLANE_IAM_ROLE
    databaseEncryption:
      kmsKeyARN: ARN_OF_KMS_KEY
    hub:
      membershipName: ANTHOS_CONNECT_NAME
    cloudOperations: # Optional
      projectID: YOUR_PROJECT
      location: GCP_LOCATION
      enableLogging: ENABLE_LOGGING
      enableMonitoring: ENABLE_MONITORING
    tags:
      TAG_KEY: TAG_VALUE
    proxySecretName: PROXY_SECRET_NAME
    workloadIdentity:
      oidcDiscoveryGCSBucket: WORKLOAD_IDENTITY_BUCKET
    rootVolume:  # Optional
      sizeGiB: ROOT_VOLUME_SIZE
      volumeType: ROOT_VOLUME_TYPE
      iops: ROOT_VOLUME_IOPS
      kmsKeyARN: ROOT_VOLUME_KEY
    etcd:  # Optional
      mainVolume:
        sizeGiB: ETCD_VOLUME_SIZE
        volumeType: ETCD_VOLUME_TYPE
        iops: ETCD_VOLUME_IOPS
        kmsKeyARN: ETCD_VOLUME_KEY
  authentication:
    awsIAM:
      adminIdentityARNs: ADMIN_IAM_ARN
    oidc: # Optional
    - certificateAuthorityData: CERTIFICATE_STRING
      clientID: CLIENT_ID
      clientSecret: CLIENT_SECRET
      extraParams:  EXTRA_PARAMS
      groupsClaim:  GROUPS_CLAIM
      groupPrefix:  GROUP_PREFIX
      issuerURI:  ISSUER_URL
      kubectlRedirectURI:  KUBECTL_REDIRECT_URL
      scopes:  SCOPES
      userClaim:  USER_CLAIM
      userPrefix:  USER_PREFIX

The subfields under spec are described in the following sections.

spec.networking

This object defines cluster-wide networking configuration.

Name	Description	Type	Example	Required
vpcID	The ID of the VPC where your cluster runs. A user cluster's control plane and node pools run in a single VPC.	`string`	`vpc-0814934042d983118`	yes
podAddressCIDRBlocks	Range of IPv4 addresses used by the cluster's pods. Currently only a single range is supported. The range must not overlap with any subnets reachable from your network. It is safe to use the same range across multiple different AWSCluster objects.	`list(string)`	`[10.1.0.0/16]`	yes
serviceAddressCIDRBlocks	Range of IPv4 addresses used by the cluster's services. Currently only a single range is supported. The range must not overlap with any subnets reachable from your network. It is safe to use the same range across multiple different AWSCluster objects.	`list(string)`	`[10.2.0.0/16]`	yes
serviceLoadBalancerSubnetIDs	Subnet IDs where GKE on AWS can create public or private load balancers. GKE on AWS applies tags to each of these subnets to support load balancing. ALB support requires more than one specified subnet.	`list(string)`	`[subnet-abcdefg, subnet-12345678]`

spec.controlPlane

This object includes common parameters for the cluster's control plane.

Name	Description	Type	Example	Required
version	The control plane's GKE version. See Versioning and upgrades and upgrading a user cluster for more information.	`string`	`1.25.5-gke.2100`	yes
instanceType	An AWS EC2 instance type for each control plane replica. See Supported instance types.	`string`	`m5.large`	yes
keyName	The AWS EC2 key pair assigned to each control plane replica.	`string`	`my-key-pair`	yes
subnetIDs	A list of VPC Subnets for control plane replicas.	`list(string)`	`[subnet-06a004869a1eae947]`	yes
securityGroupIDs	GKE on AWS automatically creates security groups with minimum rules needed for a functioning cluster. If you would like to add additional security groups with access to control plane replicas, add their IDs to `securityGroupIDs`.	`list(string)`	`[sg-0ec06559d997a796f]`	no
iamInstanceProfile	The name of the AWS EC2 instance profile assigned to control plane replicas.	`string`	`my-control-plane-profile`	yes
databaseEncryption.kmsKeyARN	The Amazon resource name (ARN) of the AWS KMS key that GKE on AWS uses to encrypt application-layer secrets in clusters.	string	`arn:aws:kms:us-west-1:123456789:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee`
hub.membershipName	The Connect membership name used to register your cluster.	string	projects/`YOUR_PROJECT`/locations/global/memberships/`CLUSTER_NAME`
cloudOperations	The Cloud Logging and Cloud Monitoring configuration for control plane nodes.	`object`	{projectID: `YOUR_PROJECT`, location: us-central1, enableLogging: true, enableMonitoring: true}	no
tags	Key/value metadata assigned to each AWS resource supporting the AWSCluster. For more information, see Tagging best practices	`map(string)`	{Environment: Production, Team: Analytics}	no
proxySecretName	User-chosen name of a Secret used to define cluster-specific network proxies	string	proxy-secret-0	no
workloadIdentity	Cloud Storage bucket for workload identity configuration. Contains a single field: `oidcDiscoveryGCSBucket`.	`object`	{oidcDiscoveryGCSBucket: my-bucket}	no
rootVolume	Parameters for control plane replica root volumes. Contains an object defined in the following section.	`object`	`{sizeGiB: 10}`	yes
etcd	Parameters for etcd volumes. Contains an object defined in the following section.	`object`	`{sizeGiB: 10}`	yes

spec.controlPlane.rootVolume and spec.etcd.rootVolume

These fields apply to both spec.controlPlane.rootVolume and spec.etcd.mainVolume.

Name	Description	Type	Example	Required
sizeGiB	Size of the root volume in Gigabytes.	`integer`	`10`	yes
volumeType	The control plane node's AWS EBS volume type. Can be `gp2` (default) or `gp3`.	`string`	`gp2`	no
iops	The amount of provisioned IO operations per second (IOPS) for volumes. Only valid when `volumeType` is `GP3`. For more information, see General Purpose SSD volumes (gp3).	`integer`	`5000`	no
kmsKeyARN	The ARN of the AWS KMS key used to encrypt the EBS volume. For more information, see Using a customer managed CMK to encrypt volumes.	`string`	`arn:aws:kms:us-west-1:123456789:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee`	no

spec.authentication.awsIAM

This object specifies roles granted cluster administrator access with AWS IAM.

Name	Description	Type	Example	Required
adminIdentityARNs	ARN of AWS IAM users or roles granted cluster administrator access.	string	`arn:aws:iam::123456789012:user/admin`	Only required if using AWS IAM authentication.

spec.authentication.oidc

You can specify more than one oidc object. The oidc object is defined below. For more information, see Authenticating with OIDC.

Field	Required	Description	Format
certificateAuthorityData	No	A base64-encoded PEM-encoded certificate for the OIDC provider. To create the string, encode the certificate, including headers, into base64. Include the resulting string in `certificateAuthorityData` as a single line. Example: `certificateAuthorityData: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tC...k1JSUN2RENDQWFT==`	String
clientID	Yes	ID for the client application that makes authentication requests to the OpenID provider.	String
clientSecret	No	Shared secret between OIDC client application and OIDC provider.	String
extraParams	No	Additional key-value parameters to send to the OpenID provider. If you are authorizing a group, pass in `resource=token-groups-claim`. If your authorization server prompts for consent, for authentication with Microsoft Azure and Okta, set `extraParams` to `prompt=consent`. For Google Cloud Identity, set `extraParams` to `prompt=consent,access_type=offline`.	Comma-delimited list
groupsClaim	No	JWT claim that the provider uses to return your security groups.	String
groupPrefix	No	Prefix prepended to group claims to prevent clashes with existing names. For example, given a group foobar and a prefix gid-, gid-foobar.	String
issuerURI	Yes	URL where authorization requests are sent to your OpenID, such as `https://example.com/adfs`. The Kubernetes API server uses this URL to discover public keys for verifying tokens. The URI must use HTTPS.	URL String
kubectlRedirectURI	Yes	The redirect url `kubectl` uses for authorization.	URL String
scopes	Yes	Additional scopes to send to the OpenID provider. Microsoft Azure and Okta require the `offline_access` scope.	Comma-delimited list
userClaim	No	JWT claim to use as the username. The default is `sub`, which is expected to be a unique identifier of the end user. You can choose other claims, such as email or name, depending on the OpenID provider. However, claims other than email are prefixed with the issuer URL to prevent naming clashes.	String
userPrefix	No	Prefix prepended to username claims to prevent clashes with existing names. If you do not provide this field, and the username is a value other than an email address, the prefix defaults to `issuerurl#`. When you set `userPrefix` to `-`, prefixing is disabled.	String