A new version of GKE on AWS was released on October 2. See the release notes for more information.

AWSCluster

This topic describes the configuration options of the AWSCluster Custom Resource Definition.

Defining an AWSCluster

AWSCluster is a Kubernetes custom resource defined by GKE on AWS. This resource represents a control plane in AWS.

To create a cluster, copy the following YAML and complete the highlighted values as defined in the spec.networking and spec.controlPlane field definitions. Then, apply the resource manifest to your management service.

apiVersion: multicloud.cluster.gke.io/v1
kind: AWSCluster
metadata:
  name: cluster-name
spec:
  region: aws-region
  networking:
    vpcID: vpc-id
    podAddressCIDRBlocks: pod-address-cidr-blocks
    serviceAddressCIDRBlocks: service-address-cidr-blocks
    serviceLoadBalancerSubnetIDs: service-load-balancer-subnets
  controlPlane:
    version: gke-version # Latest version is 1.17.9-gke.2800
    instanceType: aws-instance-type
    keyName: ssh-key-name
    subnetIDs:
    - control-plane-subnet-ids
    securityGroupIDs:
    - control-plane-security-groups
    iamInstanceProfile: control-plane-iam-role
    rootVolume:
      sizeGiB: root-volume-size
    etcd:
      mainVolume.sizeGIB: etcd-volume-size
    databaseEncryption:
      kmsKeyARN: arn-of-kms-key
    hub:
      membershipName: anthos-connect-name
    tags:
      tag-key: tag-value
    workloadIdentity:
      oidcDiscoveryGCSBucket: workload-identity-bucket
  authentication:
    awsIAM:
      adminIdentityARNs: [ADMIN_IAM_ARN]
    oidc:
    - certificateAuthorityData: [CERTIFICATE_STRING]
      clientID: [CLIENT_ID]
      clientSecret: [CLIENT_SECRET]
      extraParams:  [EXTRA_PARAMS]
      groupsClaim:  [GROUPS_CLAIM]
      groupPrefix:  [GROUP_PREFIX]
      issuerURI:  [ISSUER_URI]
      kubectlRedirectURI:  [KUBECTL_REDIRECT_URI]
      scopes:  [SCOPES]
      userClaim:  [USER_CLAIM]
      userPrefix:  [USER_PREFIX]

The subfields under spec are described in the following sections.

spec.networking

This object defines cluster-wide networking configuration.

Name Description Type Example Required
vpcID The ID of the VPC where your cluster runs. A user cluster's control plane and node pools run in a single VPC. string vpc-0814934042d983118 yes
podAddressCIDRBlocks Range of IPv4 addresses used by the cluster's pods. Currently only a single range is supported. list(string) [10.1.0.0/16] yes
serviceAddressCIDRBlocks Range of IPv4 addresses used by the cluster's services. Currently only a single range is supported. list(string) [10.2.0.0/16] yes
serviceLoadBalancerSubnetIDs Subnet IDs where GKE on AWS can create public or private load balancers. GKE on AWS applies tags to each of these subnets to support load balancing. ALB support requires more than one specified subnet. list(string) [subnet-abcdefg, subnet-12345678]

spec.controlPlane

This object includes common parameters for the cluster's control plane.

Name Description Type Example Required
version The control plane's GKE version. See Versioning and upgrades and upgrading a user cluster for more information. string 1.17.9-gke.2800 yes
instanceType An AWS EC2 instance type for each control plane replica. See Supported instance types. string t3.medium yes
keyName The AWS EC2 key pair assigned to each control plane replica. string my-key-pair yes
subnetIDs A list of VPC Subnets for control plane replicas. list(string) [subnet-06a004869a1eae947] yes
securityGroupIDs GKE on AWS automatically creates security groups with minimum rules needed for a functioning cluster. If you would like to add additional security groups with access to control plane replicas, add their IDs to securityGroupIDs. list(string) [sg-0ec06559d997a796f] no
iamInstanceProfile The name of the AWS EC2 instance profile assigned to control plane replicas. string my-control-plane-profile yes
rootVolume Parameters for control plane replica root volumes. Contains a single field: sizeGiB. object {sizeGiB: 10} yes
etcd Parameters for Elastic Block Store (EBS) volumes used by etcd. Contains a single field: mainVolume.sizeGIB. object {mainVolume: {sizeGiB: 10} yes
databaseEncryption.kmsKeyARN The Amazon resource name (ARN) of the AWS KMS key that GKE uses to encrypt application-layer secrets in clusters. string arn:aws:kms:us-west-1:123456789:key/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
hub.membershipName The Connect membership name used to register your cluster. string projects/your-project/locations/global/memberships/cluster-name
tags Key/value metadata assigned to each AWS resource supporting the AWSCluster. For more information, see Tagging best practices map(string) {Environment: Production, Team: Analytics} no
workloadIdentity Cloud Storage bucket for workload identity configuration. Contains a single field: oidcDiscoveryGCSBucket. object {oidcDiscoveryGCSBucket: my-bucket} no

spec.authentication.awsIAM

This object specifies roles granted cluster administrator access with AWS IAM.

Name Description Type Example Required
adminIdentityARNs ARN of AWS IAM users or roles granted cluster administrator access. string arn:aws:iam::123456789012:user/admin Only required if using OIDC authentication.

spec.authentication.oidc

You can specify more than one oidc object. The oidc object is defined below.

Field Required Description Format
certificateAuthorityData No A base64-encoded PEM-encoded certificate for the OIDC provider. To create the string, encode the certificate, including headers, into base64. Include the resulting string in certificateAuthorityData as a single line. Example: certificateAuthorityData: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tC...k1JSUN2RENDQWFT== String
clientID Yes ID for the client application that makes authentication requests to the OpenID provider. String
clientSecret No Shared secret between OIDC client application and OIDC provider. String
extraParams No Additional key-value parameters to send to the OpenID provider. If you are authorizing a group, pass in resource=token-groups-claim.

If your authorization server prompts for consent, for authentication with Microsoft Azure and Okta, set extraParams to prompt=consent. For Google Cloud Identity, set extraParams to prompt=consent,access_type=offline.

Comma-delimited list
groupsClaim No JWT claim that the provider uses to return your security groups. String
groupPrefix No Prefix prepended to group claims to prevent clashes with existing names. For example, given a group foobar and a prefix gid-, gid-foobar. String
issuerURI Yes URL where authorization requests are sent to your OpenID, such as https://example.com/adfs. The Kubernetes API server uses this URL to discover public keys for verifying tokens. The URI must use HTTPS. URL String
kubectlRedirectURI Yes The redirect url `kubectl` uses for authorization. URL String
scopes Yes Additional scopes to send to the OpenID provider. Microsoft Azure and Okta require the offline_access scope. Comma-delimited list
userClaim No JWT claim to use as the username. The default is `sub`, which is expected to be a unique identifier of the end user. You can choose other claims, such as email or name, depending on the OpenID provider. However, claims other than email are prefixed with the issuer URL to prevent naming clashes. String
userPrefix No Prefix prepended to username claims to prevent clashes with existing names. If you do not provide this field, and the username is a value other than an email address, the prefix defaults to issuerurl#. When you set userPrefix to -, prefixing is disabled. String