Running Django on Cloud Run (fully managed)

Deploying stateful applications like Django involves integrating multiple services together to interact together to form a cohesive project.

This tutorial assumes that you are familiar with Django web development. The sample code implements base views, models, and route configurations. Combined with the interactive Django admin, this code sample shows how to integrate managed databases, object storage, encrypted secrets, and build pipelines with serverless compute.

While this tutorial demonstrates Django specifically, you can use this deployment process with other Django-based frameworks, such as Wagtail and Django CMS.

The Django site is served from Cloud Run, which uses multiple backing services to store different data types (relational database information, media assets, configuration secrets, and container images). The backend services are updated by Cloud Build as part of a build and migrate task.

Costs

Important: This tutorial uses the following billable components of Google Cloud:

• Cloud SQL
• Cloud Storage
• Cloud Run (fully managed)
• Cloud Build
• Container Registry
• Secret Manager

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Account.

   If you don't already have one, sign up for a new account.

2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to the project selector page

3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

4. Enable the Cloud Run, Cloud SQL, Cloud Build, Secret Manager, and Compute Engine APIs.

   Enable the APIs

5. Install and initialize the Cloud SDK.
   Note: You can run the gcloud tool in the Cloud Console without installing the Cloud SDK. To run the gcloud tool in the Cloud Console, use Cloud Shell .
6. Ensure sufficient permissions are available to the account used for this tutorial.
   Note: In cases where the Owner role cannot be used, the following minimum roles are required to complete thie tutorial:

   • Cloud SQL Admin
   • Storage Admin
   • Cloud Run Admin
   • Secret Manager Admin

Clone the Django app

After you've completed the prerequisites, download and deploy the Django sample app. The following sections guide you through configuring, running, and deploying the app.

Cloning the Django app

The code for the Django sample app is in the GoogleCloudPlatform/python-docs-samples repository on GitHub.

1. You can either download the sample as a ZIP file and extract it or clone the repository to your local machine:

   git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

2. Go to the directory that contains the sample code:

Linux/macOS

  cd python-docs-samples/run/django

Windows

  cd python-docs-samples\run\django

Preparing the backing services

This tutorial uses a number of Google Cloud services to provide the database, media storage, and secret storage that support the deployed Django project. These services are deployed in a specific region. For efficency between services, it is best that all services are deployed in the same region. For more information about the closest region to you, see Products available by region.

Set up a Cloud SQL for PostgreSQL instance

Django supports multiple relational databases, but offers the most support for PostgreSQL, which is one of the offered database types in Cloud SQL.

The following sections describes the creation of a PostgreSQL instance, database, and database user for the polls app.

Create a PostgreSQL instance

Console

1. In the Cloud Console, go to the Cloud SQL Instances page.

   Go to the Cloud SQL Instances page

2. Click Create Instance.

3. Click PostgreSQL.

4. In the Instance ID field, enter a name for the instance (INSTANCE_NAME).

5. Enter a password for the postgres user.

6. Use the default values for the other fields.

7. Click Create.

gcloud

1. Create the PostgreSQL instance:

   gcloud sql instances create INSTANCE_NAME \
       --project PROJECT_ID \
       --database-version POSTGRES_12 \
       --tier db-f1-micro \
       --region REGION
   

   Replace the following:

   • INSTANCE_NAME: the Cloud SQL instance name
   • PROJECT_ID: the Google Cloud project ID
   • REGION: the Google Cloud region

   It takes a few minutes to create the instance and for it to be ready for use.

2. Set a password for the postgres user:

   gcloud sql users set-password postgres \
       --instance INSTANCE_NAME --prompt-for-password
   

Create a database

Console

1. In the Cloud Console, go to the Cloud SQL Instances page.

   Go to the Cloud SQL Instances page

2. Select the INSTANCE_NAME instance.

3. Go to the Databases tab.

4. Click Create database.

5. In the Database name dialog, enter DATABASE_NAME.

6. Click Create.

gcloud

• Create the database within the recently created instance:

  gcloud sql databases create DATABASE_NAME \
      --instance INSTANCE_NAME
  

  Replace DATABASE_NAME with a name for the database inside the instance.

Create a user

Console

Note: Any users that are created by using Cloud SQL have the privileges associated with the cloudsqlsuperuser role: CREATEROLE, CREATEDB, and LOGIN. This tutorial opts to create the user directly in PostgreSQL.

1. In the Cloud Console, activate Cloud Shell.

   Activate Cloud Shell

2. In Cloud Shell, use the built-in gcloud client to connect to your INSTANCE_NAME instance:

   gcloud sql connect INSTANCE_NAME --user postgres
   

3. Enter the postgres user password.

   You are now using psql. You should see the postgres=> prompt.

4. Create a user:

   CREATE USER 'django' WITH PASSWORD 'PASSWORD';
   

   Replace PASSWORD with a sufficiently secret password.

5. Grant full rights on the new database to the new user:

   GRANT ALL PRIVILEGES ON DATABASE 'DATABASE_NAME' TO 'django';
   

6. Exit psql:

   \q
   

gcloud

Note: Any users that are created by using Cloud SQL have the privileges associated with the cloudsqlsuperuser role: CREATEROLE, CREATEDB, and LOGIN. This tutorial opts to create the user directly in PostgreSQL.

1. Start a connection to the SQL instance:

   gcloud sql connect INSTANCE_NAME --user postgres
   

   Replace INSTANCE_NAME with the created Cloud SQL instance.

2. Enter the postgres user password.

   You are now using psql. You should see the postgres=> prompt.

3. Create a user:

   CREATE USER 'django' WITH PASSWORD 'PASSWORD';
   

4. Grant full rights on the new database to the new user:

   GRANT ALL PRIVILEGES ON DATABASE 'DATABASE_NAME' TO 'django';
   

5. Exit psql:

   \q
   

Set up a Cloud Storage bucket

You can store Django's included static assets, as well as user-uploaded media, in highly avaiable object storage using Cloud Storage. The django-storages[google] package handles Django's interaction with this storage backend.

Console

1. In the Cloud Console, go to the Cloud Storage Browser page.

   Go to the Cloud Storage Browser page

2. Click Create bucket.
3. In the Create bucket dialog, specify the following attributes:
   • A unique bucket name, subject to the bucket name requirements.
   • Default storage class: Standard
   • Location: MEDIA_BUCKET
4. Click Create.

gcloud

The gsutil command-line tool was installed as part of installing the Cloud SDK.

• Create a Cloud Storage bucket:

  gsutil mb -l REGION gs://PROJECT_ID_MEDIA_BUCKET
  

  Replace MEDIA_BUCKET with a suffix for the media bucket. Combined with the project ID, this creates a unique bucket name.

Store secret values in Secret Manager

Now that the backing services are configured, Django needs information about these services. Instead of putting these values directly into the Django source code, this tutorial uses Secret Manager to store this information securely.

Cloud Run and Cloud Build interact with secrets by using their respective service accounts, identified by an email address that contains the project number.

Create Django environment file as a Secret Manager secret

You store the settings required to start Django in a secured .env file. The sample code uses the Secret Manager API to retrieve the secret value, and the django-environ package to load the values into the Django environment. The secret is configured to be accessible by both Cloud Build and Cloud Run, because both those services will be running Django.

1. Create a file called .env, defining the database connection string, the media bucket name, and a new SECRET_KEY value:

   DATABASE_URL=postgres://django:PASSWORD@//cloudsql/PROJECT_ID:REGION:INSTANCE_NAME/DATABASE_NAME
   GS_BUCKET_NAME=PROJECT_ID_MEDIA_BUCKET
   SECRET_KEY=(a random string, length 50)
   

2. Store the secret in Secret Manager:

Console

1. In the Cloud Console, go to the Secret Manager page.

   Go to the Secret Manager page

2. Click Create secret

3. In the Name field, enter django_settings.

4. In the Upload file dialog, select the locally stored .env file.

5. Click Create secret.

6. In Details for django_settings, note the project number:

   projects/PROJECTNUM/secrets/django_settings
   

7. Click Add Member

8. In the New Members field, enter [e-mail], and then press Enter.

9. In the New Members field, enter [e-mail], and then press Enter.

10. In the Role drop-down menu, select Secret Manager Secret Accessor.

11. Click Save.

gcloud

1. Create a new secret, django_settings:

   gcloud secrets create django_settings --replication-policy automatic
   

2. Add the .env file as a version of the secret.

   gcloud secrets versions add django_settings --data-file .env
   

3. To confirm the creation of the secret, check it:

   gcloud secrets describe django_settings
   

   Make a note of the project number:

   projects/PROJECTNUM/secrets/django_settings
   

4. Grant access to the secret to the Cloud Run service account:

   gcloud secrets add-iam-policy-binding django_settings \
       --member serviceAccount:[e-mail] \
       --role roles/secretmanager.secretAccessor
   

5. Grant access to the secret to the Cloud Build service account:

   gcloud secrets add-iam-policy-binding django_settings \
       --member serviceAccount:[e-mail] \
       --role roles/secretmanager.secretAccessor
   

   In the output, confirm that bindings lists the two service accounts as members.

Create secret for Django's admin password

The Django admin user is normally created by running the interactive management command createsuperuser.

This tutorial uses a data migration to create the admin user, retrieving said password from Secret Manager.

Console

1. In the Cloud Console, go to the Secret Manager page.

2. Click Create secret.

3. In the Name field, enter superuser_password.

4. In the Secret value field, enter a random password.

5. Click Create secret.

6. In Details for superuser_password, make a note of the project number ( projects/PROJECTNUM/secrets/superuser_password).

7. Click Add Member.

8. In the New Members field, enter [e-mail], and then press Enter.

   Note: This secret is only accessible by Cloud Build.

9. In the Role drop-down menu, select Secret Manager Secret Accessor.

10. Click Save

gcloud

1. Create a new secret, superuser_password:

   gcloud secrets create superuser_password --replication-policy automatic
   

2. Add a randomly generated password as a version of that secret:

   cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c30 > superuser_password
   
   gcloud secrets versions add superuser_password --data-file superuser_password
   

3. Grant access to the secret to Cloud Build:

   gcloud secrets add-iam-policy-binding superuser_password \
       --member serviceAccount:[e-mail] \
       --role roles/secretmanager.secretAccessor
   

   In the output, confirm that bindings lists only the Cloud Build as a member.

Grant Cloud Build access to Cloud SQL

In order for Cloud Build to apply the database migrations, you need to grant permissions for Cloud Build to access Cloud SQL.

Console

1. In the Cloud Console, go to the Identity and Access Management page.

   Go to the Identity and Access Management page

2. To edit the entry for [e-mail], click Edit.

3. Click Add another role

4. In the Select a role dialog, select Cloud SQL Client.

5. Click Save

gcloud

1. Grant permission for Cloud Build to access Cloud SQL:

   gcloud projects add-iam-policy-binding PROJECT_ID \
       --member serviceAccount:[e-mail] \
       --role roles/cloudsql.client
   

First deployment of the application

With the backing services setup, you can now deploy the Cloud Run service.

1. Using the supplied cloudmigrate.yaml, use Cloud Build to build the image, run the database migrations, and populate the static assets:

   gcloud builds submit --config cloudmigrate.yaml
       --substitutions _INSTANCE_NAME=INSTANCE_NAME,_REGION=REGION
   

   This first build takes a few minutes to complete.

2. When the build is successful, deploy the Cloud Run service for the first time, setting the service region, base image, and connected Cloud SQL instance:

   gcloud run deploy polls-service \
       --platform managed \
       --region REGION \
       --image gcr.io/PROJECT_ID/polls \
       --add-cloudsql-instances PROJECT_ID:REGION:INSTANCE_NAME \
       --allow-unauthenticated
   

   You should see output that shows the deployment succeeded, with a service URL:

   Service [polls-service] revision [polls-service-00001-tug] has been deployed
   and is serving 100 percent of traffic at https://polls-service--uc.a.run.app
   

3. To see the deployed service, go to the service URL.

   

   If the service URL shows "You're at the polls index", the Django application has been successfully deployed.

4. To log into the Django admin, append /admin to the URL, and login with the username admin, and the password set earlier.

   

   Once authenticated, the Django admin will display, showing the Questions model in the Polls app.

Setting minimum permissions (optional)

By default, this service is deployed with the default compute service account. However, in some cases, using the default service account can provide too many permissions. If you want to be more restrictive, you need to create your own service account and assign only the permissions that are required by your service. The permissions required can vary from service to service, depending on the resources used by a particular service.

The minimum project roles required by this service are the following:

• Cloud Run Invoker
• Cloud SQL Client
• Storage Admin, on the media bucket
• Secret Manager Accessor, on the Django settings secret. (Access to the Django admin secret is not required by the service itself.)

To create a service account with the required permissions, do the following:

1. In Cloud Shell, create a service account with the required roles:

   gcloud iam service-accounts create polls-service-account
   SERVICE_ACCOUNT=polls-service-account@PROJECT_ID.iam.gserviceaccount.com
   
   # Cloud Run Invoker
   gcloud projects add-iam-policy-binding PROJECT_ID \
       --member serviceAccount:${SERVICE_ACCOUNT} \
       --role roles/run.Invoker
   
   # Cloud SQL Client
   gcloud projects add-iam-policy-binding PROJECT_ID \
       --member serviceAccount:${SERVICE_ACCOUNT} \
       --role roles/cloudsql.client
   
   # Storage Admin, on the media bucket
   gsutil iam ch \
       serviceAccount:${SERVICE_ACCOUNT}:roles/storage.objectAdmin \
       gs://MEDIA_BUCKET
   
   # Secret Accessor, on the Django settings secret.
   gcloud secrets add-iam-policy-binding django_settings \
       --member serviceAccount:${SERVICE_ACCOUNT} \
       --role roles/secretmanager.secretAccessor
   

2. Deploy the service, associating it with the new service account:

   gcloud run deploy polls-service \
       --platform managed \
       --region REGION \
       --service-account ${SERVICE_ACCOUNT}
   

Redeploying the application

While the initial provisoning and deployment steps were complex, redeployment is relatively straightfoward.

1. Run the Cloud Build build and migration script:

   gcloud builds submit --config cloudmigrate.yaml \
       --substitutions _INSTANCE_NAME=INSTANCE_NAME,_REGION=REGION
   

2. Deploy the service, specifying only the region and image:

   gcloud run deploy polls-service \
       --platform managed \
       --region REGION \
       --image gcr.io/PROJECT_ID/polls
   

Understanding the code

The Django sample app was created using standard Django tooling. These commands create the project and the polls app:

django-admin startproject mysite .
python manage.py startapp polls

The base views, models, and route configurations were copied from Writing your first Django app (Part 1 and Part 2).

Secrets from Secret Manager

The settings.py file contains code that uses the Secret Manager Python API to retrieve the latest version of the named secret, and pull it into the environment (using django-environ):

run/django/mysite/settings.py

View on GitHub

import google.auth
from google.cloud import secretmanager

_, project = google.auth.default()

if project:
    client = secretmanager.SecretManagerServiceClient()

    SETTINGS_NAME = os.environ.get("SETTINGS_NAME", "django_settings")
    name = f"projects/{project}/secrets/{SETTINGS_NAME}/versions/latest"
    payload = client.access_secret_version(name=name).payload.data.decode(
        "UTF-8"
    )
env.read_env(io.StringIO(payload))

The django_settings was used to store multiple secret values to reduce the number of different secrets that needed to be configured.

While the superuser_password could have been created directly from the command-line , the file-based method was used instead. If generated from the command line, care was taken using head -c to determine the length of the randomly generated string used, while ensuring there was no new-line character at the end of the file, which would have caused issues when the password was entered into the Django admin.

Database connection

Because the django_settings secret contained a specially-formed database connection string, the database helper from django-environ can be invoked to assign the DATABASES value:

run/django/mysite/settings.py

View on GitHub

# Use django-environ to parse the connection string
DATABASES = {"default": env.db()}

This method was used as opposed to defining separate values for the components of the database connection information.

Cloud-stored static

The settings.py file also uses django-storages to integrate the Cloud Storage media bucket directly into the project:

run/django/mysite/settings.py

View on GitHub

# Define static storage via django-storages[google]
GS_BUCKET_NAME = env("GS_BUCKET_NAME")

DEFAULT_FILE_STORAGE = "storages.backends.gcloud.GoogleCloudStorage"
STATICFILES_STORAGE = "storages.backends.gcloud.GoogleCloudStorage"
GS_DEFAULT_ACL = "publicRead"

Automation with Cloud Build

The cloudmigrate.yaml file performs not only the typical image build steps (creating the container image and pushing that to the container registry), but also the Django migrate and collectstatic commands. These require access to the database, which is performed by using the app-engine-exec-wrapper, a helper for Cloud SQL Proxy:

run/django/cloudmigrate.yaml

View on GitHub

steps:
  - id: "build image"
    name: "gcr.io/cloud-builders/docker"
    args: ["build", "-t", "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}", "."]

  - id: "push image"
    name: "gcr.io/cloud-builders/docker"
    args: ["push", "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}"]

  - id: "apply migrations"
    name: "gcr.io/google-appengine/exec-wrapper"
    args:
      [
        "-i",
        "gcr.io/$PROJECT_ID/${_SERVICE_NAME}",
        "-s",
        "${PROJECT_ID}:${_REGION}:${_INSTANCE_NAME}",
        "-e",
        "SETTINGS_NAME=${_SECRET_SETTINGS_NAME}",
        "--",
        "python",
        "manage.py",
        "migrate",
      ]

  - id: "collect static"
    name: "gcr.io/google-appengine/exec-wrapper"
    args:
      [
        "-i",
        "gcr.io/$PROJECT_ID/${_SERVICE_NAME}",
        "-s",
        "${PROJECT_ID}:${_REGION}:${_INSTANCE_NAME}",
        "-e",
        "SETTINGS_NAME=${_SECRET_SETTINGS_NAME}",
        "--",
        "python",
        "manage.py",
        "collectstatic",
        "--verbosity",
        "2",
        "--no-input"
      ]

substitutions:
  _INSTANCE_NAME: django-instance
  _REGION: us-central1
  _SERVICE_NAME: polls-service
  _SECRET_SETTINGS_NAME: django_settings

images:
  - "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}"

Substitution variables are used in this configuration. Changing the values in the file directly mean the --substitutions flag can be dropped at migration time.

In this configuration, only existing migrations are applied. In order to extend this configuration to generate migrations, add a makemigrations step.

To extend the Cloud Build configuration to include the deployment in the one configuration without having to run two commands, see Continuous deployment from git using Cloud Build. This requires IAM changes, as described.

Superuser creation with data migrations

The Django management command createsuperuser can only be run interactively -- that is, when the user can enter in information in response to prompts. While you can use this command with Cloud SQL Proxy and executing commands within a local Docker setup, another way is to create the superuser as a data migration:

run/django/mysite/migrations/0001_createsuperuser.py

View on GitHub

import os

from django.contrib.auth.models import User
from django.db import migrations
from django.db.backends.postgresql.schema import DatabaseSchemaEditor
from django.db.migrations.state import StateApps

import google.auth
from google.cloud import secretmanager


def createsuperuser(apps: StateApps, schema_editor: DatabaseSchemaEditor) -> None:
    """
    Dynamically create an admin user as part of a migration
    Password is pulled from Secret Manger (previously created as part of tutorial)
    """
    if os.getenv("TRAMPOLINE_CI", None):
        # We are in CI, so just create a placeholder user for unit testing.
        admin_password = "test"
    else:
        client = secretmanager.SecretManagerServiceClient()

        # Get project value for identifying current context
        _, project = google.auth.default()

        # Retrieve the previously stored admin password
        PASSWORD_NAME = os.environ.get("PASSWORD_NAME", "superuser_password")
        name = f"projects/{project}/secrets/{PASSWORD_NAME}/versions/latest"
        admin_password = client.access_secret_version(name=name).payload.data.decode(
            "UTF-8"
        )

    # Create a new user using acquired password, stripping any accidentally stored newline characters
    User.objects.create_superuser("admin", password=admin_password.strip())


class Migration(migrations.Migration):

    initial = True
    dependencies = []
    operations = [migrations.RunPython(createsuperuser)]

Cleanup

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Cloud Console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.