Configuring Cloud CDN through Ingress

This guide shows you how to use a BackendConfig custom resource definition (CRD) to configure Cloud CDN in Google Kubernetes Engine (GKE).

Overview

In a GKE cluster, HTTP(S) Load Balancing (a component of Cloud Load Balancing) handles incoming traffic. Typically, the HTTP(S) load balancer is configured by the GKE Ingress controller, which gets configuration information from a Kubernetes Ingress object. The Ingress is associated with one or more Service objects. Each Service holds routing information that is used to direct an incoming request to a particular Pod and port.

Beginning with Kubernetes version 1.10.5-gke.3, you can provide additional configuration for the external load balancer by associating a Service port with a custom resource named BackendConfig.

The GKE Ingress controller reads configuration information from the BackendConfig and sets up the load balancer accordingly. A BackendConfig holds configuration information that is specific to Cloud Load Balancing. Kubernetes Ingress and Service resources do not offer a way to configure provider-specific features like Cloud CDN; BackendConfig provides a way for you to do that configuration.

Here's the big picture of how you set up a BackendConfig in this exercise:

  1. Create a BackendConfig.
  2. Create a Service, and associate one of its ports with the BackendConfig.
  3. Create an Ingress, and associate the Ingress with the (Service, port) pair.

Before you begin

Before you start, make sure you have performed the following tasks:

Set up default gcloud settings using one of the following methods:

  • Using gcloud init, if you want to be walked through setting defaults.
  • Using gcloud config, to individually set your project ID, zone, and region.

Using gcloud init

  1. Run gcloud init and follow the directions:

    gcloud init

    If you are using SSH on a remote server, use the --console-only flag to prevent the command from launching a browser:

    gcloud init --console-only
  2. Follow the instructions to authorize gcloud to use your Google Cloud account.
  3. Create a new configuration or select an existing one.
  4. Choose a Google Cloud project.
  5. Choose a default Compute Engine zone.

Using gcloud config

  • Set your default project ID: 
    gcloud config set project project-id
  • If you are working with zonal clusters, set your default compute zone: 
    gcloud config set compute/zone compute-zone
  • If you are working with regional clusters, set your default compute region: 
    gcloud config set compute/region compute-region
  • Update gcloud to the latest version: 
    gcloud components update

Creating a namespace

Create a Kubernetes namespace for the objects used in this guide:

kubectl create namespace cdn-how-to

Creating a Deployment

  1. Create a file named my-deployment.yaml based on the following Deployment manifest. This manifest declares that you want to run two replicas of the ingress-gce-echo-amd64 web application:

    apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: cdn-how-to
  name: my-deployment
spec:
  selector:
    matchLabels:
      purpose: demonstrate-cdn
  replicas: 2
  template:
    metadata:
      labels:
        purpose: demonstrate-cdn
    spec:
      containers:
      - name: echo-amd64
        image: gcr.io/google-samples/hello-app-cdn:1.0

  2. Create the Deployment resource:

    kubectl apply -f my-deployment.yaml

Creating a BackendConfig

  1. Create a file named my-backend-config.yaml based on the following BackendConfig manifest. The manifest specifies a Cloud CDN cache policy and declares that Cloud CDN should be enabled:

    apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
  namespace: cdn-how-to
  name: my-backend-config
spec:
  cdn:
    enabled: true
    cachePolicy:
      includeHost: true
      includeProtocol: true
      includeQueryString: false

  2. Create the BackendConfig resource:

    kubectl apply -f my-backend-config.yaml

  3. View the BackendConfig:

    kubectl get backendconfig my-backend-config --output yaml --namespace cdn-how-to

    You can see the Cloud CDN cache policy in the output:

    apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
  name: my-backend-config
  namespace: cdn-how-to
  ...
spec:
  cdn:
    cachePolicy:
      includeHost: true
      includeProtocol: true
      includeQueryString: false
    enabled: true

Creating a Service

  1. Create a file named my-service.yaml based on the following Service manifest:

    apiVersion: v1
kind: Service
metadata:
  namespace: cdn-how-to
  name: my-service
  labels:
    purpose: demonstrate-cdn
  annotations:
    beta.cloud.google.com/backend-config: '{"ports": {"80":"my-backend-config"}}'
spec:
  type: NodePort
  selector:
    purpose: demonstrate-cdn
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080

  2. Create the Service resource:

    kubectl apply -f my-service.yaml

  3. View the Service:

    kubectl get service my-service --namespace cdn-how-to --output yaml

    The output is similar to this:

    apiVersion: v1
kind: Service
metadata:
  annotations:
    beta.cloud.google.com/backend-config: '{"ports": {"80":"my-backend-config"}}'
    ...
  labels:
    purpose: demonstrate-cdn
  name: my-service
  namespace: cdn-how-to
  ...
spec:
  clusterIP: 10.51.255.39
  externalTrafficPolicy: Cluster
  ports:
  - nodePort: 31484
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    purpose: demonstrate-cdn
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

    For the purposes of this exercise, these are the important things to note about your Service:

    • Port 80 of the Service is associated with a BackendConfig named my-backend-config. The beta.cloud.google.com/backend-config annotation specifies this.

    • The Service has type NodePort. This is the typical type for Services that are going to be associated with an Ingress.

    • Any Pod that has the label purpose: demonstrate-cdn is a member of the Service. The selector field specifies this.

    • Traffic directed to the service on TCP port 80 is routed to TCP port 8080 in one of the member Pods. The port and targetPort fields specify this.

Reserving a static external IP address

  1. Reserve a static external IP address:

    gcloud compute addresses create cdn-how-to-address --global

  2. View your static external IP address:

    gcloud compute addresses list --filter "name=cdn-how-to-address"

    The output shows the name and value of the address:

    NAME                ...     ADDRESS        STATUS
cdn-how-to-address          203.0.113.1    RESERVED

Creating an Ingress

  1. Create a file named my-ingress.yaml based on the following Ingress manifest:

    apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  namespace: cdn-how-to
  name: my-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "cdn-how-to-address"
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: my-service
          servicePort: 80

  2. Create the Ingress resource:

    kubectl apply -f my-ingress.yaml

  3. Wait ten minutes for the Kubernetes Ingress controller to configure a Cloud load balancer, and then view the Ingress:

    kubectl get ingress my-ingress --output yaml --namespace cdn-how-to

    The output is similar to this:

    apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  ...
  name: my-ingress
  namespace: cdn-how-to
  ...
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: my-service
          servicePort: 80
        path: /*
status:
  loadBalancer:
    ingress:
    - ip: 201.0.113.1

    For the purpose of this exercise, here are the important things to note about your Ingress:

    • The IP address for incoming traffic is listed under loadBalancer:ingress:.

    • The Ingress has one rule that applies to incoming HTTP requests from any host. This is because there is no host field in the rule. So by default, the rule applies to all hosts.

    • All incoming requests are treated the same, regardless of the URL path. This is specified by the path value /*.

    • Incoming requests are routed to a Pod that is a member of my-service. In this exercise, the member pods have the label purpose: demonstrate-cdn.

    • Requests are routed to the Pod on the target port specified in my-service. In this exercise, the Pod target port is 8080.

Viewing the web app

Enter this curl command twice:

curl -v static-address/?cache=true

where static-address is your static external IP address.

The output shows the response headers and body. In the response headers, you can see that the content was cached. The Age header tells you how many seconds the content has been cached:

...
< HTTP/1.1 200 OK
< Date: Fri, 25 Jan 2019 02:34:08 GMT
< Content-Length: 70
< Content-Type: text/plain; charset=utf-8
< Via: 1.1 google
< Cache-Control: max-age=86400,public
< Age: 2716
<
Hello, world!
Version: 1.0.0
Hostname: my-deployment-7f589cc5bc-l8kr8

Viewing load balancing logs

You can verify that content was cached by viewing the Stackdriver logs for HTTP Load Balancing. Before you check the logs, make sure you have requested a response from the app at least twice.

Console

  1. In the Cloud Console, go to the Logs page in the Logging menu.

    Go to the Logs page

  2. From the first drop-down menu, select Cloud HTTP Load Balancer.

  3. Expand the most recent log entry, and expand the httpRequest field of the entry.

  4. In the httpRequest field, you can see that cacheHit is true:

    httpRequest: {
cacheHit:  true
cacheLookup:  true
...

gcloud

Run the following command, where project-id is your project ID:

gcloud logging read \
    'logName="projects/project-id/logs/requests"' \
    --limit 2

The output shows that there was a cache hit:

httpRequest:
cacheHit: true
cacheLookup: true

Cleaning up

After completing the exercises in this guide, complete the following steps to remove the resources to prevent unwanted charges incurring on your account:

  1. Delete the Kubernetes objects that you created for this exercise:

    kubectl delete ingress my-ingress --namespace cdn-how-to
kubectl delete service my-service --namespace cdn-how-to
kubectl delete backendconfig my-backend-config --namespace cdn-how-to
kubectl delete deployment my-deployment --namespace cdn-how-to
kubectl delete namespace cdn-how-to

  2. Delete your static external IP address:

    gcloud compute addresses delete cdn-how-to-address --global

Limitations

Cloud CDN and Identity-Aware Proxy cannot be enabled for the same HTTP(S) Load Balancing backend service.

Troubleshooting

BackendConfig not found

This error occurs when a BackendConfig for a Service port is specified in the Service annotation, but the actual BackendConfig resource could not be found. This can occur if you have not created the BackendConfig resource at all, created it in the wrong namespace, or misspelled the reference in the Service annotation.

kubectl get event

KIND    ... SOURCE
Ingress ... loadbalancer-controller

MESSAGE
Error during sync: error getting BackendConfig for port 80 on service “default/my-service”:
no BackendConfig for service port exists

Cloud CDN and IAP both enabled

This error occurs when you have enabled both IAP and Cloud CDN in a BackendConfig.

kubectl get event

KIND    ... SOURCE
Ingress ... loadbalancer-controller

MESSAGE
Error during sync: BackendConfig default/config-default is not valid:
iap and cdn cannot be enabled at the same time.

Content not being cached

If you find that your content is not being cached, make sure that your application is properly configured to enable caching of content. For more information, see cacheability.

What's next