Running Django on Container Engine

It's easy to get started developing Django apps running on Google Container Engine, and because the apps you create will run on the same infrastructure that powers all of Google's products, you can be confident that they will scale to serve all of your users, whether there are a few or millions of them.

This tutorial assumes you are familiar with Django web development. It walks you through deploying the official Django tutorial app, Writing your first Django App. It's a good idea work through that tutorial before this one, especially if you are new to Django development. The app's models represent polls that contain questions, and you can interact with the models using the Django admin console.

This tutorial requires Python 2.7 or 3.4. You also need to have Docker installed.

Before you begin

Check off each step as you complete it.

  1. check_box_outline_blank check_box Create a project in the Google Cloud Platform Console.
    If you haven't already created a project, create one now. Projects enable you to manage all Google Cloud Platform resources for your app, including deployment, access control, billing, and services.
    1. Open the Cloud Platform Console.
    2. In the drop-down menu at the top, select Create a project.
    3. Click Show advanced options. Under App Engine location, select a United States location.
    4. Give your project a name.
    5. Make a note of the project ID, which might be different from the project name. The project ID is used in commands and in configurations.
  2. check_box_outline_blank check_box Enable billing for your project, and sign up for a free trial.

    If you haven't already enabled billing for your project, enable billing now, and sign up for a free trial. Enabling billing allows the application to consume billable resources such as running instances and storing data. During your free trial period, you won't be billed for any services.

  3. check_box_outline_blank check_box Install the Google Cloud SDK.

    If you haven't already installed the Google Cloud SDK, install and initialize the Google Cloud SDK now. The SDK contains tools and libraries that enable you to create and manage resources on Google Cloud Platform.

  4. check_box_outline_blank check_box Enable APIs for your project.

    This takes you to the Cloud Platform Console and automatically enables the APIs used by this tutorial. The APIs used are: Google Cloud SQL API, Compute Engine API.

Download and run the app

After you've completed the prerequisites, you can download and deploy the Django sample app. The following sections guide you through configuring, running, and deploying the sample.

Clone the Django app

The code for the Django sample app is in the Google Cloud Platform Python Samples repository on GitHub.

Clone the repository to your local machine:

git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

Go to the directory that contains the sample code:

cd python-docs-samples/container_engine/django_tutorial

Alternatively, you can download the sample as a zip and extract it.

Setting up your local environment

When deployed, your application uses the Cloud SQL Proxy that is built in to the App Engine environment to communicate with your Cloud SQL instance. However, to test your application locally, you must install and use a local copy of the Cloud SQL Proxy in your development environment.

Learn more about the Cloud SQL Proxy.

To perform basic administrative tasks on your Cloud SQL instance, you can use the PostgreSQL Client.

Install the SQL proxy

Download and install the Cloud SQL Proxy. The Cloud SQL Proxy is used to connect to your Cloud SQL instance when running locally.

Linux 64-bit

  1. Download the proxy:
    wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64
    
  2. Rename the proxy to use the standard filename:
    mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy
    
  3. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

Linux 32-bit

  1. Download the proxy:
    wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.386
    
  2. Rename the proxy to use the standard filename:
    mv cloud_sql_proxy.linux.386 cloud_sql_proxy
    
  3. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

OS X 64-bit

  1. Download the proxy:
    curl -o cloud_sql_proxy https://dl.google.com/cloudsql/cloud_sql_proxy.darwin.amd64
    
  2. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

OS X 32-bit

  1. Download the proxy:
    curl -o cloud_sql_proxy https://dl.google.com/cloudsql/cloud_sql_proxy.darwin.386
    
  2. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

Windows 64-bit

Right-click https://dl.google.com/cloudsql/cloud_sql_proxy_x64.exe and select "Save link as..." to download the proxy, renaming it to cloud_sql_proxy.exe.

Windows 32-bit

Right-click https://dl.google.com/cloudsql/cloud_sql_proxy_x86.exe and select "Save link as..." to download the proxy, renaming it to cloud_sql_proxy.exe.
If your operating system is not included here, you can also compile the proxy from source.

Create a Cloud SQL instance

  1. Create a Cloud SQL for PostgreSQL instance.

    Name the instance polls-instance or similar. It can take a few minutes for the instance to be ready. After the instance is ready, it should be visible in the instances list.

  2. Now use the Cloud SDK from command line to run the following command. Copy the connectionName value for the next step.
  3. gcloud beta sql instances describe [YOUR_INSTANCE_NAME]

Initialize your Cloud SQL instance

  1. Start the Cloud SQL Proxy using the connectionName from the previous step.

    Linux/Mac OS X

    ./cloud_sql_proxy -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:5432

    Windows

    cloud_sql_proxy.exe -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:5432

    Replace [YOUR_INSTANCE_CONNECTION_NAME] with the Instance Connection Name of your Cloud SQL instance. This step establishes a connection from your local computer to your Cloud SQL instance for local testing purposes. Keep the Cloud SQL Proxy running the entire time you test your application locally.

  2. Next you create a new Cloud SQL user and database.

    CONSOLE

    1. Create a new database using the Cloud Platform Console for your Cloud SQL instance polls-instance. For example you can use the name polls.
    2. Create a new user using the Cloud Platform Console for your Cloud SQL instance polls-instance.

    POSTGRES CLIENT

    1. In a separate command-line tab, install the Postgres client.
      sudo apt-get install postgres
      
    2. Use the Postgres client or similar program to connect to your instance. When prompted, use the root password you configured.
      psql --host 127.0.0.1 --user postgres --password
      
    3. Create the required databases, users, and access permissions in your Cloud SQL database using the commands below. Replace [POSTGRES_USER] and [POSTGRES_PASSWORD] with your desired username and password.
      CREATE DATABASE polls;
      CREATE USER [POSTGRES_USER] WITH PASSWORD '[POSTGRES_PASSWORD]';
      GRANT ALL PRIVILEGES ON DATABASE polls TO [POSTGRES_USER];
      GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO [POSTGRES_USER];
      

Create a service account

The proxy requires a service account with Editor privileges for your Cloud SQL instance. For more information about service accounts, see the Google Cloud Platform Auth Guide.

  1. Go to the Cloud SQL Service accounts page of the Google Cloud Platform Console.

    Go to the Service accounts page

  2. If needed, select the project that contains your Cloud SQL instance.
  3. Click Create service account.
  4. In the Create service account dialog, provide a descriptive name for the service account.
  5. For Role, select Cloud SQL > Cloud SQL Client.
  6. Change the Service account ID to a shorter value if needed.
  7. Click Furnish a new private key.
  8. The default key type is JSON, the correct value.
  9. Click Create.

    The private key file is downloaded to your machine. You can move it to another location. Keep the key file secure.

Configure the database settings

  1. Set environment variables for database access for local testing.

    export DATABASE_USER=<your-database-user>
    
    export DATABASE_PASSWORD=<your-database-password>
    

Setting up your Kubernetes configuration

  1. This application is represented in a single Kubernetes configuration, called polls. In polls.yaml replace <your-project-id> with your project ID.

  2. In polls.yaml replace <your-cloudsql-connection-string> with the value of connectionName outputted from the following command:

    gcloud beta sql instances describe [YOUR_INSTANCE_NAME]
    

Run the app on your local computer

  1. To run the Hello World app on your local computer, you'll need a Python development environment set up, including Python, pip, and virtualenv. Follow these instructions to install on Linux, OS X, or Windows.

  2. Create an isolated Python environment, and install dependencies:

    virtualenv env
    source env/bin/activate
    pip install -r requirements.txt
    
  3. Run the Django migrations to set up your models:

    python manage.py makemigrations
    python manage.py makemigrations polls
    python manage.py migrate
    
  4. Start a local web server:

    python manage.py runserver
    
  5. In your web browser, browse to http://localhost:8000/admin

You should see a simple webpage with the following text: "Hello, world. You're at the polls index." The sample app pages are delivered by the Django web server running on your computer. When you're ready to move forward, press Ctrl+C to stop the local web server.

Use the Django admin console

  1. Create a superuser:

    python manage.py createsuperuser
    
  2. Run the main program:

    python manage.py runserver
    
  3. In your web browser, browse to http://localhost:8000/admin

  4. Log on to the admin site using the username and password you created when you ran createsuperuser.

Deploy the app to Container Engine

  1. When the app is deployed to Cloud Platform, it uses the Gunicorn server. Gunicorn doesn't serve static content, so the app uses Google Cloud Storage to serve static content.

    Create a Cloud Storage bucket and make it publicly readable. Replace <your-gcs-bucket> with a bucket name of your choice. For example, you could use your project ID as a bucket name:

    gsutil mb gs://<your-gcs-bucket>
    gsutil defacl set public-read gs://<your-gcs-bucket>
    
  2. Gather all the static content locally into one folder:

    python manage.py collectstatic
    
  3. Upload the static content to CloudStorage:

    gsutil rsync -R static/ gs://<your-gcs-bucket>/static
    
  4. In mysite/settings.py, set the value of STATIC_URL to this URL, replacing <your-gcs-bucket> with the your bucket name.

    http://storage.googleapis.com/<your-gcs-bucket>/static/
    
  5. Visit the Container Engine Console to initialize Container Engine. Wait for the "Container Engine is getting ready. This may take a minute or more" message to disappear.

  6. Create a Container Engine Cluster:

    gcloud container clusters create polls \
      --scopes "https://www.googleapis.com/auth/userinfo.email","cloud-platform" \
      --num-nodes 4 --zone "us-central1-a"
    

    Did you get the error: "Project [PROJECT_ID] is not fully initialized with the default service accounts."?

    Initialize Container Engine

    If you received an error, visit the console to initialize Container Engine in your project:

    Go to the Container Engine page

    Wait for the "Container Engine is getting ready. This can take a minute or more" message to disappear.

  7. After the cluster is created, you use the kubectl command line tool, which is integrated with the gcloud tool, to interact with your Kubernetes cluster. Because gcloud and kubectl are separate tools, you need to make sure kubectl is configured to interact with the right cluster:

    gcloud container clusters get-credentials polls --zone "us-central1-a"
    
  8. You need several secrets to enable your Google Container Engine application to connect with your Cloud SQL instance. One is required for instance-level access (connection), while the other two are required for database access. For more information about the two levels of access control, see Instance Access Control.

    1. Create the secret for instance-level access, providing the location of the key you downloaded when you created your service account:

      kubectl create secret generic cloudsql-oauth-credentials --from-file=credentials.json=[PATH_TO_CREDENTIAL_FILE]
      
    2. Create the secrets needed for database access:

      kubectl create secret generic cloudsql --from-literal=username=[PROXY_USERNAME] --from-literal=password=[PASSWORD]
      
  9. Run this command to get the public Docker image for the Cloud SQL Proxy.

    docker pull b.gcr.io/cloudsql-docker/gce-proxy:1.05
    
  10. Build a Docker image, replacing <your-project-id> with your project ID.

    docker build -t gcr.io/<your-project-id>/polls .
    
  11. Push the Docker image to the Google Container Registry, replacing <your-project-id> with your project ID.

    gcloud docker -- push gcr.io/<your-project-id>/polls
    
  12. Create the Kubernetes resource:

    kubectl create -f polls.yaml
    
  13. After the resources are created, there should be three polls pods on the cluster. Check the status of your pods:

    kubectl get pods
    

    Wait a few minutes for the pod statuses to turn to Running. If the pods are not ready or if you see restarts, you can get the logs for a particular pod to figure out the issue:

    kubectl logs <your-pod-id>
    

See the app run in the cloud

After the pods are ready, you can get the public IP address of the load balancer:

kubectl get services polls

Navigate to the EXTERNAL-IP address in your browser to see the Django basic landing page and access the admin console.

Understanding the code

The Django sample app was created using the standard Django tooling. These commands create the project and the polls app:

django-admin startproject mysite
python manage.py startapp polls

The settings.py contains the configuration for your SQL database:

DATABASES = {
    'default': {
        # If you are using Cloud SQL for MySQL rather than PostgreSQL, set
        # 'ENGINE': 'django.db.backends.mysql' instead of the following.
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'polls',
        'USER': os.getenv('DATABASE_USER'),
        'PASSWORD': os.getenv('DATABASE_PASSWORD'),
        'HOST': '127.0.0.1',
        'PORT': '5432',
    }
}

The polls.yaml files specifies two Kubernetes resources. The first is the Service, which defines a consistent name and private IP address for the Django web app. The second is a Http Load Balancer with a public-facing extrnal IP address.

# The polls service provides a load-balancing proxy over the polls app
# pods. By specifying the type as a 'LoadBalancer', Container Engine will
# create an external HTTP load balancer.
# For more information about Services see:
#   https://cloud.google.com/container-engine/docs/services/
# For more information about external HTTP load balancing see:
#   https://cloud.google.com/container-engine/docs/load-balancer
apiVersion: v1
kind: Service
metadata:
  name: polls
  labels:
    app: polls
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: polls

The service provides a network name and IP address, and Kubernetes pods run the application code behind the service. The polls.yaml file specifies a deployment that provides declarative updates for Kubernetes pods. The service directs traffic to the deployment by matching the service's selector to the deployment's label. In this case, the selector polls is matched to the label polls.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: polls
  labels:
    app: polls
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: polls
    spec:
      containers:
      - name: polls-app
        # Replace  with your project ID or use `make template`
        image: gcr.io/<your-project-id>/polls
        # This setting makes nodes pull the docker image every time before
        # starting the pod. This is useful when debugging, but should be turned
        # off in production.
        imagePullPolicy: Always
        env:
            - name: DATABASE_USER
              valueFrom:
                secretKeyRef:
                  name: cloudsql
                  key: username
            - name: DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: cloudsql
                  key: password
        ports:
        - containerPort: 8080

      - image: b.gcr.io/cloudsql-docker/gce-proxy:1.05
        name: cloudsql-proxy
        command: ["/cloud_sql_proxy", "--dir=/cloudsql",
                  "-instances=<your-cloudsql-connection-string>=tcp:5432",
                  "-credential_file=/secrets/cloudsql/credentials.json"]
        volumeMounts:
          - name: cloudsql-oauth-credentials
            mountPath: /secrets/cloudsql
            readOnly: true
          - name: ssl-certs
            mountPath: /etc/ssl/certs
          - name: cloudsql
            mountPath: /cloudsql
      volumes:
        - name: cloudsql-oauth-credentials
          secret:
            secretName: cloudsql-oauth-credentials
        - name: ssl-certs
          hostPath:
            path: /etc/ssl/certs
        - name: cloudsql
          emptyDir:

Send feedback about...