Running Django on Google Kubernetes Engine

Django apps that run on Google Kubernetes Engine (GKE) scale well because they run on the same infrastructure that powers all of Google's products.

This tutorial assumes you are familiar with Django web development. If you are new to Django development, it's a good idea to work through writing your first Django app before continuing. In that tutorial, the app's models represent polls that contain questions, and you can interact with the models by using the Django admin console.

This tutorial requires Python 2.7 or 3.4 or later. You also need to have Docker installed.

Before you begin

Sign in to your Google Account.
If you don't already have one, sign up for a new account.
In the Cloud Console, on the project selector page, select or create a Cloud project.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to the project selector page
Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.
Install and initialize the Cloud SDK.
Enable the Datastore, Pub/Sub, Cloud Storage JSON, Cloud Logging, and Google+ APIs.
Enable the APIs

Downloading and running the app

After you've completed the prerequisites, download and run the Django sample app. The following sections guide you through configuring, running, and deploying the app.

Cloning the Django app

The code for the Django sample app is in the GoogleCloudPlatform/python-docs-samples repository on GitHub.

You can either download the sample as a zip file and extract it or clone the repository to your local machine by using the following command:
```
git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
```

Change to the directory that contains the sample code:

cd python-docs-samples/kubernetes_engine/django_tutorial

Setting up your local environment

When deployed, your app uses the Cloud SQL Proxy that is built in to the App Engine environment to communicate with your Cloud SQL instance. However, to test your app locally, you must install and use a local copy of the proxy in your development environment.

Learn more about the Cloud SQL Proxy.

To perform basic admin tasks on your Cloud SQL instance, you can use the PostgreSQL client.

Enable the Cloud SQL Admin API

Before using Cloud SQL, you must enable the Cloud SQL Admin API:

gcloud services enable sqladmin

Installing the Cloud SQL Proxy

Download and install the Cloud SQL Proxy. The Cloud SQL Proxy connects to your Cloud SQL instance when running locally.

Linux 64-bit

Download the proxy:

wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy

Make the proxy executable:
```
chmod +x cloud_sql_proxy
```

Linux 32-bit

Download the proxy:

wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.386 -O cloud_sql_proxy

Make the proxy executable:
```
chmod +x cloud_sql_proxy
```

macOS 64-bit

Download the proxy:

curl -o cloud_sql_proxy https://dl.google.com/cloudsql/cloud_sql_proxy.darwin.amd64

Make the proxy executable:
```
chmod +x cloud_sql_proxy
```

macOS 32-bit

Download the proxy:

curl -o cloud_sql_proxy https://dl.google.com/cloudsql/cloud_sql_proxy.darwin.386

Make the proxy executable:
```
chmod +x cloud_sql_proxy
```

Windows 64-bit

Right-click https://dl.google.com/cloudsql/cloud_sql_proxy_x64.exe and select Save Link As to download the proxy. Rename the file to cloud_sql_proxy.exe.

Windows 32-bit

Right-click https://dl.google.com/cloudsql/cloud_sql_proxy_x86.exe and select Save Link As to download the proxy. Rename the file to cloud_sql_proxy.exe.

If your operating system isn't included here, you can also compile the proxy from source.

Creating a Cloud SQL instance

Create a Cloud SQL for PostgreSQL instance.
Name the instance polls-instance or similar. It can take a few minutes for the instance to be ready. When the instance is ready, it's visible in the instances list.
Use the Cloud SDK to run the following command where [YOUR_INSTANCE_NAME] represents the name of your Cloud SQL instance:
```
gcloud sql instances describe [YOUR_INSTANCE_NAME]
```
In the output, note the value shown for [CONNECTION_NAME].

The [CONNECTION_NAME] value is in the format [PROJECT_NAME]:[REGION_NAME]:[INSTANCE_NAME].

Initializing your Cloud SQL instance

Start the Cloud SQL Proxy by using the [CONNECTION_NAME] value from the previous step:
Linux/macOS
```
./cloud_sql_proxy -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:5432
```
Windows
```
cloud_sql_proxy.exe -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:5432
```
Replace [YOUR_INSTANCE_CONNECTION_NAME] with the [CONNECTION_NAME] value that you recorded in the previous step.

This step establishes a connection from your local computer to your Cloud SQL instance for local testing purposes. Keep the Cloud SQL Proxy running the entire time you test your app locally.
Create a Cloud SQL user and database:
Cloud Console
1. Create a new database by using the Cloud Console for your Cloud SQL instance polls-instance. For example, you can use the name polls.
2. Create a new user by using the Cloud Console for your Cloud SQL instance polls-instance.
Postgres client
1. In a separate command-line tab, install the Postgres client.
```
sudo apt-get install postgresql
```
2. Use the Postgres client or similar program to connect to your instance. When prompted, use the root password you configured.
```
psql --host 127.0.0.1 --user postgres --password
```
3. Create the required databases, users, and access permissions in your Cloud SQL database by using the following commands. Replace [POSTGRES_USER] and [POSTGRES_PASSWORD] with the username and password you want to use.
```
CREATE DATABASE polls;
CREATE USER [POSTGRES_USER] WITH PASSWORD '[POSTGRES_PASSWORD]';
GRANT ALL PRIVILEGES ON DATABASE polls TO [POSTGRES_USER];
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO [POSTGRES_USER];
```

Creating a service account

The proxy requires a service account with Editor privileges for your Cloud SQL instance. For more information about service accounts, see the Google Cloud authentication overview.

Note: To create a service account with the required permissions, you must have resourcemanager.projects.setIamPolicy permission. This permission is included in the Project Owner, Project IAM Admin, and Organization Administrator roles.
You must also have enabled the Cloud SQL Admin API.

When you use a service account to provide the credentials for the proxy, you must create it with sufficient permissions. If you are using the finer-grained Identity Access and Management (IAM) roles to manage your Cloud SQL permissions, you must give the service account a role that includes the cloudsql.instances.connect permission. The predefined Cloud SQL roles that include this permission are:

Cloud SQL Client
Cloud SQL Editor
Cloud SQL Admin

If you are using the legacy project roles (Viewer, Editor, Owner), the service account must have at least the Editor role.

Go to the Service accounts page of the Google Cloud Console.
Go to the Service accounts page
Select the project that contains your Cloud SQL instance.
Click Create service account.
In the Create service account dialog, provide a descriptive name for the service account.
For Role, select one of the following roles:
- Cloud SQL > Cloud SQL Client
- Cloud SQL > Cloud SQL Editor
- Cloud SQL > Cloud SQL Admin
Change the Service account ID to a unique, easily recognizable value.
Click Furnish a new private key and confirm that the key type is JSON.
Click Create.
The private key file is downloaded to your machine. You can move it to another location. Keep the key file secure.

Configuring the database settings

Use the following commands to set environment variables for database access. These environment variables are used for local testing.

Linux/macOS

export DATABASE_USER=<your-database-user>
export DATABASE_PASSWORD=<your-database-password>

Windows

set DATABASE_USER=<your-database-user>
set DATABASE_PASSWORD=<your-database-password>

Setting up your GKE configuration

This application is represented in a single Kubernetes configuration, called polls. In polls.yaml replace <your-project-id> with your Google Cloud project ID.
Run the following command and note the value of connectionName:
```
gcloud beta sql instances describe [YOUR_INSTANCE_NAME]
```
In the polls.yaml file, replace <your-cloudsql-connection-string> with the connectionName value.

Running the app on your local computer

To run the Django app on your local computer, set up a Python development environment, including Python, pip, and virtualenv.
Create an isolated Python environment and install dependencies. If your Python 3 installation has a different name, use that in the first command:
```
virtualenv env
source env/bin/activate
pip install -r requirements.txt
```

Run the Django migrations to set up your models:

python manage.py makemigrations
python manage.py makemigrations polls
python manage.py migrate

Start a local web server:
```
python manage.py runserver
```
In your browser, go to http://localhost:8000.

You see a page with the following text: "Hello, world. You're at the polls index." The Django web server running on your computer delivers the sample app pages.
Press Control+C to stop the local web server.

Using the Django admin console

Create a superuser. You need to specify a username and password.
```
python manage.py createsuperuser
```
Run the main program:
```
python manage.py runserver
```
In your browser, go to http://localhost:8000/admin.
Log in to the admin site using the username and password you used when you ran createsuperuser.

Deploying the app to GKE

When the app is deployed to Google Cloud, it uses the Gunicorn server. Gunicorn doesn't serve static content, so the app uses Cloud Storage to serve static content.

Collect and upload static resources

Create a Cloud Storage bucket and make it publicly readable. Replace [YOUR_GCS_BUCKET] with a bucket name of your choice. For example, you could use your project ID as a bucket name.
```
gsutil mb gs://[YOUR_GCS_BUCKET]
gsutil defacl set public-read gs://[YOUR_GCS_BUCKET]
```
Gather all the static content locally into one folder:
```
python manage.py collectstatic
```

Upload the static content to Cloud Storage:

gsutil rsync -R static/ gs://[YOUR_GCS_BUCKET]/static

In mysite/settings.py, set the value of STATIC_URL to the following URL, replacing [YOUR_GCS_BUCKET] with your bucket name:
```
http://storage.googleapis.com/[YOUR_GCS_BUCKET]/static/
```

Set up GKE

To initialize GKE, go to the Clusters page.

Go to the Clusters page

When you use GKE for the first time in a project, you need to wait for the "Kubernetes Engine is getting ready. This may take a minute or more" message to disappear.
Create a GKE cluster:
```
gcloud container clusters create polls \
  --scopes "https://www.googleapis.com/auth/userinfo.email","cloud-platform" \
  --num-nodes 4 --zone "us-central1-a"
```
Did you get the error: "Project [PROJECT_ID] is not fully initialized with the default service accounts."?

Initialize GKE

If you received an error, go to the Google Cloud Console to initialize GKE in your project.

Go to the Clusters page

Wait for the "Kubernetes Engine is getting ready. This can take a minute or more" message to disappear.
After the cluster is created, use the kubectl command-line tool, which is integrated with the gcloud tool, to interact with your GKE cluster. Because gcloud and kubectl are separate tools, make sure kubectl is configured to interact with the right cluster.
```
gcloud container clusters get-credentials polls --zone "us-central1-a"
```

Set up Cloud SQL

You need several secrets to enable your GKE app to connect with your Cloud SQL instance. One is required for instance-level access (connection), while the other two are required for database access. For more information about the two levels of access control, see Instance access control.
1. To create the secret for instance-level access, provide the location ([PATH_TO_CREDENTIAL_FILE]) of the JSON service account key you downloaded when you created your service account (see Creating a service account):
```
kubectl create secret generic cloudsql-oauth-credentials --from-file=credentials.json=[PATH_TO_CREDENTIAL_FILE]
```
2. To create the secrets for database access, use the SQL [PROXY_USERNAME] and [PASSWORD] defined in step 2 of Initializing your Cloud SQL instance:
```
kubectl create secret generic cloudsql --from-literal=username=[PROXY_USERNAME] --from-literal=password=[PASSWORD]
```
Retrieve the public Docker image for the Cloud SQL proxy.
```
docker pull b.gcr.io/cloudsql-docker/gce-proxy
```
Build a Docker image, replacing <your-project-id> with your project ID.
```
docker build -t gcr.io/<your-project-id>/polls .
```
Configure Docker to use gcloud as a credential helper, so that you can push the image to Container Registry:
```
gcloud auth configure-docker
```
Push the Docker image. Replace <your-project-id> with your project ID.
```
docker push gcr.io/<your-project-id>/polls
```
Note: This command requires write access to Cloud Storage. If you run this tutorial on a Compute Engine instance, your access to Cloud Storage might be read-only. To get write access, create a service account and use the service account to authenticate on your instance.
Create the GKE resource:
```
kubectl create -f polls.yaml
```

Deploy the app to GKE

After the resources are created, there are three polls pods on the cluster. Check the status of your pods:

    kubectl get pods

Wait a few minutes for the pod statuses to display as Running. If the pods aren't ready or if you see restarts, you can get the logs for a particular pod to figure out the issue. [YOUR-POD-ID] is a part of the output returned by the previous kubectl get pods command.

    kubectl logs [YOUR_POD_ID]

Seeing the app run in Google Cloud

After the pods are ready, you can get the public IP address of the load balancer:

kubectl get services polls

Go to the EXTERNAL-IP address in your browser to see the Django basic landing page and access the admin console.

Understanding the code

The Django sample app was created using the standard Django tooling. These commands create the project and the polls app:

django-admin startproject mysite
python manage.py startapp polls

The settings.py contains the configuration for your SQL database:

kubernetes_engine/django_tutorial/mysite/settings.py

View on GitHub

DATABASES = {
    'default': {
        # If you are using Cloud SQL for MySQL rather than PostgreSQL, set
        # 'ENGINE': 'django.db.backends.mysql' instead of the following.
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'polls',
        'USER': os.getenv('DATABASE_USER'),
        'PASSWORD': os.getenv('DATABASE_PASSWORD'),
        'HOST': '127.0.0.1',
        'PORT': '5432',
    }
}

The polls.yaml file specifies two Kubernetes resources. The first is the Service, which defines a consistent name and private IP address for the Django web app. The second is an HTTP load balancer with a public-facing external IP address.

kubernetes_engine/django_tutorial/polls.yaml

View on GitHub

# The polls service provides a load-balancing proxy over the polls app
# pods. By specifying the type as a 'LoadBalancer', Container Engine will
# create an external HTTP load balancer.
# For more information about Services see:
#   https://cloud.google.com/container-engine/docs/services/
# For more information about external HTTP load balancing see:
#   https://cloud.google.com/container-engine/docs/load-balancer
apiVersion: v1
kind: Service
metadata:
  name: polls
  labels:
    app: polls
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: polls

The service provides a network name and IP address, and GKE pods run the app's code behind the service. The polls.yaml file specifies a deployment that provides declarative updates for GKE pods. The service directs traffic to the deployment by matching the service's selector to the deployment's label. In this case, the selector polls is matched to the label polls.

kubernetes_engine/django_tutorial/polls.yaml

View on GitHub

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: polls
  labels:
    app: polls
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: polls
    spec:
      containers:
      - name: polls-app
        # Replace  with your project ID or use `make template`
        image: gcr.io/<your-project-id>/polls
        # This setting makes nodes pull the docker image every time before
        # starting the pod. This is useful when debugging, but should be turned
        # off in production.
        imagePullPolicy: Always
        env:
            - name: DATABASE_USER
              valueFrom:
                secretKeyRef:
                  name: cloudsql
                  key: username
            - name: DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: cloudsql
                  key: password
        ports:
        - containerPort: 8080

      - image: gcr.io/cloudsql-docker/gce-proxy:1.16
        name: cloudsql-proxy
        command: ["/cloud_sql_proxy", "--dir=/cloudsql",
                  "-instances=<your-cloudsql-connection-string>=tcp:5432",
                  "-credential_file=/secrets/cloudsql/credentials.json"]
        volumeMounts:
          - name: cloudsql-oauth-credentials
            mountPath: /secrets/cloudsql
            readOnly: true
          - name: ssl-certs
            mountPath: /etc/ssl/certs
          - name: cloudsql
            mountPath: /cloudsql
      volumes:
        - name: cloudsql-oauth-credentials
          secret:
            secretName: cloudsql-oauth-credentials
        - name: ssl-certs
          hostPath:
            path: /etc/ssl/certs
        - name: cloudsql
          emptyDir:

Running Django on Google Kubernetes Engine

Before you begin

Downloading and running the app

Cloning the Django app

Setting up your local environment

Enable the Cloud SQL Admin API

Installing the Cloud SQL Proxy

Linux 64-bit

Linux 32-bit

macOS 64-bit

macOS 32-bit

Windows 64-bit

Windows 32-bit

Creating a Cloud SQL instance

Initializing your Cloud SQL instance

Linux/macOS

Windows

Cloud Console

Postgres client

Creating a service account

Configuring the database settings

Linux/macOS

Windows

Setting up your GKE configuration

Running the app on your local computer

Using the Django admin console

Deploying the app to GKE

Collect and upload static resources

Set up GKE

Initialize GKE

Set up Cloud SQL

Deploy the app to GKE

Seeing the app run in Google Cloud

Understanding the code