Deploying Cloud Endpoints on Kubernetes

This page shows how to configure and deploy Google Cloud Endpoints on Kubernetes. It includes information on setting up Extensible Service Proxy and using SSL.

Configuring the API

You configure an API for Cloud Endpoints on Kubernetes using the openapi.yaml file for your API. See the basic configuration you need to perform in the Getting Started with Endpoints on Kubernetes.

The openapi.yaml file provides Open API Initiative IDL that describes the API management policies such as authentication and API key restrictions for the API service application.

Each API service is configured by its openapi.yaml, which is deployed using the gcloud command-line tool to Google Service Management. This component manages the API service configurations indexed by a service name (host in the openapi.yaml) and a config ID.

Deploying to Kubernetes

You deploy an API service by deploying the Extensible Service Proxy as a Docker container in the same Kubernetes Pod as the application container.

Notice that the set of pods running the proxy and the application are grouped under a Kubernetes service using a label selector, such as app: my-api. The Kubernetes service specifies the access policy to load balance the client requests to the proxy port.

To deploy Extensible Service Proxy, start it using the command line options provided in the page Specifying Startup Options for Extensible Service Proxy.

Adding SSL Support to the proxy

See Enabling SSL for Cloud Endpoints.

Using a custom nginx.conf

If you want to use a custom nginx.conf, prepare one by extending this sample nginx.conf. Here is a snippet of the configuration required by Cloud Endpoints:

http {
  include /etc/nginx/mime.types;
  server_tokens off;
  client_max_body_size 32m;

  upstream app_server {
    server localhost:8081;
    keepalive 128;
  }

  endpoints {
    metadata_server;
  }

  server {
    # Running port
    listen 8080;

    # Running ssl port
    listen 443 ssl;
    ssl_certificate /etc/nginx/ssl/nginx.crt;
    ssl_certificate_key /etc/nginx/ssl/nginx.key;

    # Logging to stdout enables better integration with Docker and GKE/Kubernetes.
    access_log /dev/stdout;

    location / {
      # Begin Endpoints v2 Support
      endpoints {
        on;
        # After ESP 1.7.0, "server_config" field is required.
        # It has to be /etc/nginx/server_config.pb.txt exactly.
        # If not present, some new features will not work.
        server_config /etc/nginx/server_config.pb.txt;

        # After ESP 1.7.0, "api" field is not required.
        # If added, it has to be /etc/nginx/endpoints/service.json exactly.
        # api /etc/nginx/endpoints/service.json;

        # Uncomment the line below if you are not using Google Container Engine.
        # The path should be set to the “-k” path specified in the ESP container’s                 
        # args section in the Kubernetes yaml config.
        # google_authentication_secret /etc/nginx/creds/service-account-creds.json;
      }
      # End Endpoints v2 Support

      proxy_pass http://app_server;
      proxy_redirect off;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Host $server_name;
      proxy_set_header X-Google-Real-IP $remote_addr;

      # 86400 seconds (24 hours) is the maximum a server is allowed.
      proxy_send_timeout 86400s;
      proxy_read_timeout 86400s;
    }

    include /var/lib/nginx/extra/*.conf;
  }

  server {
    # expose /nginx_status but on a different port to avoid
    # external visibility / conflicts with the app.
    listen 8090;
    location /nginx_status {
      stub_status on;
      access_log off;
    }
    location / {
      root /dev/null;
    }
  }
}

Now create a Kubernetes Configmap with your custom nginx.conf using kubectl:

kubectl create configmap nginx-config --from-file=nginx.conf

Edit the Kubernetes configuration files such as esp_echo_custom_config_gke.yaml replacing SERVICE_NAME and SERVICE_CONFIG_ID shown in the snippet below with the values returned when you deployed the API.

template:
  metadata:
    labels:
      app: esp-echo
  spec:
    volumes:
    - name: nginx-config
      configMap:
        name: nginx-config
    - name: nginx-ssl
      secret:
        secretName: nginx-ssl
    containers:
    - name: esp
      image: gcr.io/endpoints-release/endpoints-runtime:1
      args: [
        "-n", "/etc/nginx/custom/nginx.conf",
        "-s", "SERVICE_NAME",
        "-v", "SERVICE_CONFIG_ID",
      ]
      ports:
        - containerPort: 8080
        - containerPort: 443
      volumeMounts:
      - mountPath: /etc/nginx/ssl
        name: nginx-ssl
        readOnly: true
      - mountPath: /etc/nginx/custom
        name: nginx-config
        readOnly: true
    - name: echo 
      image: gcr.io/endpoints-release/echo:latest
      ports:
        - containerPort: 8081

Finally, start the service with the updated Kubernetes configuration file using kubectl.

kubectl create -f esp_echo_custom_config_gke.yaml

That's it! You have deployed Cloud Endpoints on Kubernetes.

Architecture

The following diagram shows the overall architecture where Extensible Service Proxy runs as a side-car container in front of the API service application container, with my-api API hosted at my-api.com and backed by a Kubernetes Service.

Endpoints on Kubernetes overview

What’s next

Send feedback about...

Cloud Endpoints with OpenAPI