Creating an application load balancer (ALB)

This topic shows you how to set up an AWS Application Load Balancer (ALB) with GKE on AWS.

Before you begin

Before you start using GKE on AWS, make sure you have performed the following tasks:

  • Have permissions to create AWS IAM policies, roles, and users for the load balancer.
  • Install a management service.
  • Create a user cluster. If you are using workload identity, Create a user cluster with workload identity.
  • From your anthos-aws directory, use anthos-gke to switch context to your user cluster.
    cd anthos-aws
    env HTTPS_PROXY=http://localhost:8118 \
      anthos-gke aws clusters get-credentials CLUSTER_NAME
    Replace CLUSTER_NAME with your user cluster name.
  • Install the curl command-line tool or a similar tool.

Tagging subnets

GKE on AWS requires tags on subnets that contain ALB endpoints. GKE on AWS automatically tags all subnets specified in the spec.Networking.ServiceLoadBalancerSubnetIDs field of the AWSCluster resource.

If you installed GKE on AWS into an existing VPC, or want to use additional subnets, apply tags to subnets in two or more AWS availability zones.

Set up ALB on GKE on AWS

Before you can create an ALB, you configure GKE on AWS by setting up AWS IAM permissions and providing access keys.

Create AWS IAM permissions

To create an ALB for your user cluster, you must set up an AWS IAM user with permissions to create and operate the ALB.

  1. Download an IAM policy for the ALB Ingress Controller. You can review the policy on GitHub.

    curl -o iam-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.0/docs/install/iam_policy.json
    
  2. Use the aws command-line tool to create an IAM policy named ALBIngressControllerIAMPolicy.

    aws iam create-policy \
      --policy-name ALBIngressControllerIAMPolicy \
      --policy-document file://iam-policy.json
    

    The response includes the Amazon Resource Name (ARN) of the IAM policy. Save the ARN for later use.

Grant access to your load balancer

In this section, you associate the user policy with either an AWS IAM user, or with an IAM role that has a configured workload identity.

IAM user

  1. Use the aws tool to create an IAM user for the ALB Ingress controller.

    aws iam create-user \
    --user-name ALB_CONTROLLER_USER_NAME
    

    Replace ALB_CONTROLLER_USER_NAME with the username you want to create for your ALB Ingress controller.

  2. Attach the ALBIngressControllerIAMPolicy to the username.

    aws iam attach-user-policy \
     --user-name ALB_CONTROLLER_USER_NAME \
     --policy-arn ALB_IAM_POLICY_ARN
    

    Replace the following:

    • ALB_CONTROLLER_USER_NAME with the username you want to create for your ALB Ingress controller.
    • ALB_IAM_POLICY_ARN with the ARN of the IAM policy you created earlier.
  3. Create an AWS IAM access key for the ALB Ingress controller user.

    aws iam create-access-key --user-name ALB_CONTROLLER_USER_NAME
    

    Replace ALB_CONTROLLER_USER_NAME with the username you want to create for your ALB Ingress controller.

    The aws command-line tool prints the access key's details.

    {
      "AccessKey": {
        "UserName": ALB_CONTROLLER_USER_NAME
        "AccessKeyId": "AKIAIOSFODNN7EXAMPLE",
        "Status": "Active",
        "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
        "CreateDate": "2020-07-23T17:53:58Z"
      }
    }
    
  4. Save the access key and secret access key into environment variables. You will use these to configure your user cluster.

    ALB_ACCESS_KEY_ID=ACCESS_KEY_ID
    ALB_SECRET_ACCESS_KEY=SECRET_ACCESS_KEY
    
  5. Create a Secret in your cluster with the access key and secret access key. The ALB Ingress controller uses this Secret to authenticate to AWS and manage ALBs.

    env HTTPS_PROXY=http://localhost:8118 \
      kubectl create secret generic alb-ingress-controller-creds \
      -n kube-system \
      --from-literal=access_key_id=$ALB_ACCESS_KEY_ID \
      --from-literal=secret_access_key=$ALB_SECRET_ACCESS_KEY
    

Role with workload identity

Complete the steps in Creating a user cluster with workload identity. When you create a policy, use the ARN of ALBIngressControllerIAMPolicy for the value of EXISTING_AWS_POLICY.

Configure your cluster

To configure an ALB, you need to install the following components in your user cluster:

  1. Deploy the Jetstack cert-manager by installing the manifest from GitHub.

    env HTTPS_PROXY=http://localhost:8118 \
      kubectl apply \
      --validate=false \
      -f https://github.com/jetstack/cert-manager/releases/download/v1.5.4/cert-manager.yaml
    
  2. Download the AWS Load Balancer Controller manifest from GitHub:

    curl -Lo v2_4_0_full.yaml https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/download/v2.4.0/v2_4_0_full.yaml
    
  3. Select if you are using an IAM user or Role with workload identity.

    IAM user

    Edit the file v2_4_0_full.yaml and search for kind: Deployment. Replace the Deployment object with this modified version.

    kind: Deployment
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/name: aws-load-balancer-controller
      name: aws-load-balancer-controller
      namespace: kube-system
    spec:
      replicas: 1
      selector:
      matchLabels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/name: aws-load-balancer-controller
      template:
        metadata:
          labels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/name: aws-load-balancer-controller
        spec:
          containers:
            - args:
              - --cluster-name=CLUSTER_UID
              - --aws-region=AWS_REGION
              - --aws-vpc-id=AWS_VPC_ID
              - --ingress-class=alb
              - --disable-restricted-sg-rules=true
            image: amazon/aws-alb-ingress-controller:v2.4.0
            env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: alb-ingress-controller-creds
                  key: access_key_id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: alb-ingress-controller-creds
                  key: secret_access_key
            livenessProbe:
              failureThreshold: 2
              httpGet:
                path: /healthz
                port: 61779
                scheme: HTTP
              initialDelaySeconds: 30
              timeoutSeconds: 10
            name: controller
            ports:
            - containerPort: 9443
              name: webhook-server
              protocol: TCP
            resources:
              limits:
                cpu: 200m
                memory: 500Mi
              requests:
                cpu: 100m
                memory: 200Mi
            securityContext:
              allowPrivilegeEscalation: false
              readOnlyRootFilesystem: true
              runAsNonRoot: true
            volumeMounts:
            - mountPath: /tmp/k8s-webhook-server/serving-certs
              name: cert
              readOnly: true
          priorityClassName: system-cluster-critical
          securityContext:
            fsGroup: 1337
          serviceAccountName: aws-load-balancer-controller
          terminationGracePeriodSeconds: 10
          volumes:
          - name: cert
            secret:
              defaultMode: 420
              secretName: aws-load-balancer-webhook-tls
        ---
    

    Replace the following:

    • AWS_ACCESS_KEY_ID: the AWS access key generated when you created an AWS IAM user
    • AWS_SECRET_ACCESS_KEY: the AWS secret access key generated when you created an AWS IAM user

    Role with workload identity

    Edit the file v2_4_0_full.yaml and search for kind: Deployment. Replace the entire Deployment object with this modified version:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/name: aws-load-balancer-controller
      name: aws-load-balancer-controller
      namespace: kube-system
    spec:
      replicas: 1
      selector:
        matchLabels:
          app.kubernetes.io/component: controller
          app.kubernetes.io/name: aws-load-balancer-controller
      template:
        metadata:
          labels:
            app.kubernetes.io/component: controller
            app.kubernetes.io/name: aws-load-balancer-controller
        spec:
          containers:
          - args:
            - --cluster-name=CLUSTER_UID
            - --aws-region=AWS_REGION
            - --aws-vpc-id=AWS_VPC_ID
            - --ingress-class=alb
            - --disable-restricted-sg-rules=true
            image: amazon/aws-alb-ingress-controller:v2.4.0
            env:
            - name: AWS_ROLE_ARN
              value: LB_CONTROLLER_ROLE_ARN
            - name: AWS_WEB_IDENTITY_TOKEN_FILE
              value: /var/run/secrets/aws-load-balancer-controller/serviceaccount/token
            livenessProbe:
              failureThreshold: 2
              httpGet:
                path: /healthz
                port: 61779
                scheme: HTTP
              initialDelaySeconds: 30
              timeoutSeconds: 10
            name: controller
            ports:
            - containerPort: 9443
              name: webhook-server
              protocol: TCP
            resources:
              limits:
                cpu: 200m
                memory: 500Mi
              requests:
                cpu: 100m
                memory: 200Mi
            securityContext:
              allowPrivilegeEscalation: false
              readOnlyRootFilesystem: true
              runAsNonRoot: true
            volumeMounts:
            - mountPath: /tmp/k8s-webhook-server/serving-certs
              name: cert
              readOnly: true
            - mountPath: /var/run/secrets/aws-load-balancer-controller/serviceaccount
              name: aws-iam-token
              readOnly: true
          priorityClassName: system-cluster-critical
          securityContext:
            fsGroup: 1337
          serviceAccountName: aws-load-balancer-controller
          terminationGracePeriodSeconds: 10
          volumes:
          - name: cert
            secret:
              defaultMode: 420
              secretName: aws-load-balancer-webhook-tls
          - name: aws-iam-token
            projected:
              defaultMode: 420
              sources:
              - serviceAccountToken:
                  audience: sts.amazonaws.com
                  expirationSeconds: 86400
                  path: token
    ---
    

    Replace the following:

    • CLUSTER_UID: your cluster's UID— for example, bbc7d232-21f6-4bb1-90dd-4b064cf8ccf8
    • AWS_VPC_ID: the ID of your AWS VPC— for example, vpc-1234567890abc
    • LB_CONTROLLER_ROLE_ARN: the ARN of the AWSLBControllerRole role
    • AWS_REGION: your cluster's AWS region— for example, us-east-1
  4. Apply the controller to your cluster.

    env HTTPS_PROXY=http://localhost:8118 \
      kubectl apply -f v2_4_0_full.yaml
    

    GKE on AWS installs the ALB Ingress controller.

Creating an ALB

In this section, you create an example ALB that serves a remake of the game 2048.

  1. Copy the following YAML configuration into a file named 2048.yaml. The configuration creates a Kubernetes Namespace, Service, and Deployment. The Deployment is exposed using an Ingress.

    apiVersion: v1
    kind: Namespace
    metadata:
      name: "2048-game"
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: "service-2048"
      namespace: "2048-game"
    spec:
      ports:
        - port: 80
          targetPort: 80
          protocol: TCP
      type: NodePort
      selector:
        app: "2048"
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: "2048-deployment"
      namespace: "2048-game"
    spec:
      selector:
        matchLabels:
          app: "2048"
      replicas: 5
      template:
        metadata:
          labels:
            app: "2048"
        spec:
          containers:
          - image: alexwhen/docker-2048
            imagePullPolicy: Always
            name: "2048"
            ports:
            - containerPort: 80
    ---
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: "2048-ingress"
      namespace: "2048-game"
      annotations:
        kubernetes.io/ingress.class: alb
        alb.ingress.kubernetes.io/scheme: internet-facing
      labels:
        app: 2048-ingress
    spec:
      rules:
        - http:
            paths:
              - path: /*
                backend:
                  serviceName: "service-2048"
                  servicePort: 80
    
  2. Use kubectl to apply the configuration to your cluster.

    env HTTPS_PROXY=http://localhost:8118 \
      kubectl apply -f 2048.yaml
    
  3. Use kubectl to check the status of the Ingress resource.

    env HTTPS_PROXY=http://localhost:8118 \
      kubectl get ingress -n 2048-game 2048-ingress
    

    The status of your Ingress appears. The ADDRESS column contains the endpoint of your Ingress.

    NAME           HOSTS   ADDRESS                                                             PORTS   AGE
    2048-ingress   *       123456-2048game-2048ingr-6fa0-abcdefg.us-east-1.elb.amazonaws.com   80      2m19s
    
  4. Navigate to the ALB endpoint in a browser. For example: http://123456-2048game-2048ingr-6fa0-abcdefg.us-east-1.elb.amazonaws.com The 2048 game appears.

What's next