Running the Python Bookshelf on Kubernetes Engine

This tutorial shows how to run the Python Bookshelf app on Google Kubernetes Engine (GKE). Follow this tutorial to containerize and deploy an existing Python web app to GKE. It is recommended that you work through the Bookshelf app documentation as part of the tutorial for the App Engine standard environment.

Objectives

  • Create a GKE cluster.
  • Containerize a Python app.
  • Create a replicated frontend for the Bookshelf app.
  • Create a replicated backend for the Bookshelf app.
  • Create a load-balanced service to route HTTP traffic to the Bookshelf frontend.

Costs

This tutorial uses billable components of Google Cloud Platform (GCP), including:

  • GKE
  • Compute Engine
  • Cloud Storage
  • Cloud Datastore
  • Cloud Pub/Sub

Use the Pricing Calculator to generate a cost estimate based on your projected usage. New GCP users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Google Cloud Platform project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your Google Cloud Platform project.

    Learn how to enable billing

  4. Enable the Cloud Datastore, Kubernetes Engine, Cloud Storage, Cloud Pub/Sub, and Google+ APIs.

    Enable the APIs

  5. Install and initialize the Cloud SDK.
  6. Install Docker. Docker is used to build container images locally.
  7. Install kubectl.
    gcloud components install kubectl

Creating a GKE cluster

A GKE cluster is a managed set of Compute Engine virtual machines that operate as a single GKE cluster. This tutorial needs a cluster with a minimum of two nodes, and these nodes need access to all Google APIs.

  1. Create the cluster. Replace [YOUR_GCP_ZONE] with the GCP zone where you want to host your cluster.

    gcloud container clusters create bookshelf \
        --scopes "cloud-platform" \
        --num-nodes 2 \
        --enable-basic-auth \
        --issue-client-certificate \
        --enable-ip-alias \
        --zone [YOUR_GCP_ZONE]
    
  2. Verify that you have access to the cluster. The following command lists the nodes in your container cluster and indicates that your container cluster is up and running and that you have access to it.

    kubectl get nodes
    

You use the kubectl command to create resources in a GKE cluster. To learn more about kubectl, see GKE cluster operations. In general, you use gcloud to manage resources in your GCP project and you use kubectl to manage resources within your GKE cluster. A single project can have multiple clusters, which makes it easy to have clusters that are made up of different machine types to satisfy different needs.

When you create a cluster with gcloud, it automatically sets up authentication for kubectl. Clusters created by using the Google Cloud Platform Console can set up authentication using the gcloud container clusters get-credentials command.

Cloning the sample app

The sample app is available on GitHub at GoogleCloudPlatform/getting-started-python.

  1. Clone the repository.

    git clone https://github.com/GoogleCloudPlatform/getting-started-python.git
    
  2. Navigate to the sample directory.

    cd getting-started-python/optional-kubernetes-engine
    

Initializing Cloud Datastore

The Bookshelf app uses Cloud Datastore to store the books. To initialize Cloud Datastore in your project for the first time, complete the following steps:

  1. In the GCP Console, open Cloud Datastore.

    Go to Cloud Datastore

  2. Select a region for your datastore and click Continue until you reach the Create an Entity page. Close the window as the Bookshelf app is ready to create entities in Cloud Datastore.

Creating a Cloud Storage bucket

The Bookshelf app uses Cloud Storage to store image files.

The following instructions show how to create a Cloud Storage bucket. Buckets are the basic containers that hold your data in Cloud Storage.

  1. In a terminal window, enter the following command:

    gsutil mb gs://[YOUR-BUCKET-NAME]

    Where:

    • [YOUR-BUCKET-NAME] represents the name of your Cloud Storage bucket.
  2. To view uploaded images in the bookshelf app, set the bucket's default access control list (ACL) to public-read.

    gsutil defacl set public-read gs://[YOUR-BUCKET-NAME]

    Configuring the app

    1. Open config.py for editing.
    2. Update [PROJECT_ID] with your project ID.
    3. Update [CLOUD_STORAGE_BUCKET] with the name of the bucket you created in the previous step.

    Leave the remaining configuration values unchanged.

    Containerizing the app

    The sample app includes a Dockerfile, which is used to create the app's Docker image. This Docker image runs the app on GKE.

    # The Google App Engine python runtime is Debian Jessie with Python installed
    # and various os-level packages to allow installation of popular Python
    # libraries. The source is on github at:
    #   https://github.com/GoogleCloudPlatform/python-docker
    FROM gcr.io/google-appengine/python
    
    # Create a virtualenv for the application dependencies.
    # If you want to use Python 2, add the -p python2.7 flag.
    RUN virtualenv -p python3.4 /env
    
    # Set virtualenv environment variables. This is equivalent to running
    # source /env/bin/activate. This ensures the application is executed within
    # the context of the virtualenv and will have access to its dependencies.
    ENV VIRTUAL_ENV /env
    ENV PATH /env/bin:$PATH
    
    # Install dependencies.
    ADD requirements.txt /app/requirements.txt
    RUN pip install -r /app/requirements.txt
    
    # Add application code.
    ADD . /app
    
    # Instead of using gunicorn directly, we'll use Honcho. Honcho is a python port
    # of the Foreman process manager. $PROCESSES is set in the pod manifest
    # to control which processes Honcho will start.
    CMD honcho start -f /app/procfile $PROCESSES
    
    The sample app also includes a .dockerignore file that lists filepaths that aren't included in the resulting Docker container. Typically, this file includes build artifacts and local dependency installations.

    __pycache__
    *.pyc
    *.pyo
    *.pyd
    .Python
    env
    pip-log.txt
    pip-delete-this-directory.txt
    .tox
    .coverage
    .coverage.*
    .cache
    nosetests.xml
    coverage.xml
    *,cover
    *.log
    .git
    
  3. Build the app's Docker image.

    docker build -t gcr.io/[YOUR_PROJECT_ID]/bookshelf
    
  4. Push the image to Google Container Registry so that your cluster can access the image.

    gcloud docker -- push gcr.io/[YOUR_PROJECT_ID]/bookshelf
    

Deploying the Bookshelf frontend

The Bookshelf app has a frontend server that handles the web requests and a backend worker that processes books and adds additional information.

The cluster resources needed to run the frontend are defined in bookshelf-frontend.yaml. These resources are described as a Kubernetes Deployment. Deployments make it easy to create and update a replica set and its associated pods.

# Copyright 2015 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

# This file configures the bookshelf application frontend. The frontend serves
# public web traffic.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: bookshelf-frontend
  labels:
    app: bookshelf
# The bookshelf frontend replica set ensures that at least 3
# instances of the bookshelf app are running on the cluster.
# For more info about Pods see:
#   https://cloud.google.com/kubernetes-engine/docs/pods/
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: bookshelf
        tier: frontend
    spec:
      containers:
      - name: bookshelf-app
        # Replace [GCLOUD_PROJECT] with your project ID or use `make template`.
        image: gcr.io/[GCLOUD_PROJECT]/bookshelf
        # This setting makes nodes pull the docker image every time before
        # starting the pod. This is useful when debugging, but should be turned
        # off in production.
        imagePullPolicy: Always
        # The PROCESSES environment variable is used by Honcho in the
        # Dockerfile's CMD to control which processes are started. In this
        # case, only the bookshelf process is needed.
        env:
        - name: PROCESSES
          value: bookshelf
        # The bookshelf process listens on port 8080 for web traffic by default.
        ports:
        - name: http-server
          containerPort: 8080
  1. In bookshelf-frontend.yaml, replace [GCLOUD_PROJECT] with your project ID.

  2. Deploy the resources to the cluster.

    kubectl create -f bookshelf-frontend.yaml
    
  3. Track the status of the deployment.

    kubectl get deployments
    

    Once the deployment has the same number of available pods as desired pods, the deployment is complete. If you run into issues with the deployment, you can delete it and start over.

    kubectl delete deployments bookshelf-frontend
    
  4. Once the deployment is complete you can see the pods that the deployment created.

    kubectl get pods
    

Deploying the Bookshelf backend

The Bookshelf backend is deployed the same way as the frontend.

# Copyright 2015 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

# This file configures the bookshelf task worker. The worker is responsible
# for processing book requests and updating book information.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: bookshelf-worker
  labels:
    app: bookshelf
# The bookshelf worker replica set ensures that at least 2 instances of the
# bookshelf worker pod are running on the cluster.
# For more info about Pods see:
#   https://cloud.google.com/kubernetes-engine/docs/pods/
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: bookshelf
        tier: worker
    spec:
      containers:
      - name: bookshelf-app
        # Replace [GCLOUD_PROJECT] with your project ID or use `make template`.
        image: gcr.io/[GCLOUD_PROJECT]/bookshelf
        # This setting makes nodes pull the docker image every time before
        # starting the pod. This is useful when debugging, but should be turned
        # off in production.
        imagePullPolicy: Always
        # The PROCESSES environment variable is used by Honcho in the
        # Dockerfile's CMD to control which processes are started. In this
        # case, only the worker process is needed.
        env:
        - name: PROCESSES
          value: worker
  1. In bookshelf-worker.yaml, replace [GCLOUD_PROJECT] with your project ID.

  2. Deploy the resources to the cluster.

    kubectl create -f bookshelf-worker.yaml
    
  3. Verify that the pods are running.

    kubectl get pods
    

Creating the Bookshelf service

Kubernetes Services provide a single point of access to a set of pods. While it's possible to access a single pod, pods are ephemeral and it's usually more convenient to address a set of pods with a single endpoint. In the Bookshelf app, the Bookshelf service lets you access the Bookshelf frontend pods from a single IP address. This service is defined in bookshelf-service.yaml.

# Copyright 2016 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

# The bookshelf service provides a load-balancing proxy over the bookshelf
# frontend pods. By specifying the type as a 'LoadBalancer', Kubernetes Engine
# will create an external HTTP load balancer.
# For more information about Services see:
#   https://cloud.google.com/kubernetes-engine/docs/services/
# For more information about external HTTP load balancing see:
#   https://cloud.google.com/kubernetes-engine/docs/load-balancer
apiVersion: v1
kind: Service
metadata:
  name: bookshelf-frontend
  labels:
    app: bookshelf
    tier: frontend
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: http-server
  selector:
    app: bookshelf
    tier: frontend

Notice that the pods and the service that uses the pods are separate. Kubernetes uses labels to select the pods that a service addresses. With labels, you can have a service that addresses pods from different replica sets and have multiple services that point to an individual pod.

  1. Create the Bookshelf service.

    kubectl create -f bookshelf-service.yaml
    
  2. Get the service's external IP address.

    kubectl describe service bookshelf
    

    Note that it may take up to 60 seconds to allocate the IP address. The external IP address is listed under LoadBalancer Ingress.

Accessing the Bookshelf app

You have now deployed all the resources needed to run the Bookshelf app on GKE. Use the external IP address from the previous step to load the app in your web browser and create books. If you deployed the worker, the books are automatically updated with information from the Google Books API.

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Delete the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

To delete the project:

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete the cluster

Deleting the cluster removes all GKE and Compute Engine resources, but you need to manually remove any resources in Cloud Storage, Cloud Datastore, and Cloud Pub/Sub.

Delete the cluster using the following command. Replace [YOUR_GCP_ZONE] with the zone you used when creating the cluster.

gcloud container clusters delete bookshelf --zone [YOUR_GCP_ZONE]

What's next

  • Try out other Google Cloud Platform features for yourself. Have a look at our tutorials.
  • Explore other GCP services.
Was this page helpful? Let us know how we did:

Send feedback about...