Using Cloud SQL with Python

This part of the Python Bookshelf tutorial shows how the sample app stores its persistent data in Google Cloud SQL.

This page is part of a multi-page tutorial. To start from the beginning and see instructions for setting up, go to Python Bookshelf App.

Creating a Cloud SQL instance and database

When deployed, your application uses the Cloud SQL Proxy that is built in to the App Engine environment to communicate with your Cloud SQL instance. However, to test your application locally, you must install and use a local copy of the Cloud SQL Proxy in your development environment.

Learn more about the Cloud SQL Proxy.

To perform basic administrative tasks on your Cloud SQL instance, you can use the MySQL Client.

Install the SQL proxy

Download and install the Cloud SQL Proxy. The Cloud SQL Proxy is used to connect to your Cloud SQL instance when running locally.

Linux 64-bit

  1. Download the proxy:
    wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy
    
  2. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

Linux 32-bit

  1. Download the proxy:
    wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.386 -O cloud_sql_proxy
    
  2. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

macOS 64-bit

  1. Download the proxy:
    curl -o cloud_sql_proxy https://dl.google.com/cloudsql/cloud_sql_proxy.darwin.amd64
    
  2. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

macOS 32-bit

  1. Download the proxy:
    curl -o cloud_sql_proxy https://dl.google.com/cloudsql/cloud_sql_proxy.darwin.386
    
  2. Make the proxy executable:
    chmod +x cloud_sql_proxy
    

Windows 64-bit

Right-click https://dl.google.com/cloudsql/cloud_sql_proxy_x64.exe and select "Save link as..." to download the proxy, renaming it to cloud_sql_proxy.exe.

Windows 32-bit

Right-click https://dl.google.com/cloudsql/cloud_sql_proxy_x86.exe and select "Save link as..." to download the proxy, renaming it to cloud_sql_proxy.exe.
If your operating system is not included here, you can also compile the proxy from source.

Create a Cloud SQL instance

  1. Create a Cloud SQL for MySQL Second Generation instance. Name the instance library or similar. It can take a few minutes for the instance to be ready. After the instance is ready, it should be visible in the instances list.
  2. Now use the Cloud SDK from command line to run the following command. Copy the value shown for connectionName for the next step.
    gcloud sql instances describe [YOUR_INSTANCE_NAME]

    The connectionName value is in the format [PROJECT_NAME]:[REGION_NAME]:[INSTANCE_NAME].

Initialize your Cloud SQL instance

  1. Start the Cloud SQL Proxy using the connectionName from the previous step.

    Linux/Mac OS X

    ./cloud_sql_proxy -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:3306

    Windows

    cloud_sql_proxy.exe -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:3306

    Replace [YOUR_INSTANCE_CONNECTION_NAME] with the value of connectionName that you recorded in the previous step.

    This step establishes a connection from your local computer to your Cloud SQL instance for local testing purposes. Keep the Cloud SQL Proxy running the entire time you test your application locally.

  2. Next you create a new Cloud SQL user and database.

    CONSOLE

    1. Create a new database using the GCP Console for your Cloud SQL instance library. For example, you can use the name bookshelf.
    2. Create a new user using the GCP Console for your Cloud SQL instance library.

    MYSQL CLIENT

    1. In a separate command-line tab, use the MySQL client or similar program to connect to your instance. When prompted, use the root password you configured.
      mysql --host 127.0.0.1 --user root --password
      
    2. Create the required databases, users, and access permissions in your Cloud SQL database using the commands below. Replace [MYSQL_USER] and [MYSQL_PASSWORD] with your desired username and password.
      CREATE DATABASE bookshelf;
      CREATE USER '[MYSQL_USER]' IDENTIFIED BY '[MYSQL_PASSWORD]';
      GRANT ALL ON *.* TO '[MYSQL_USER]';
      

Configuring settings

This section uses code in the 2-structured-data directory. Edit the files and run commands in this directory.

  1. Open config.py for editing.
  2. Set the value of PROJECT_ID to your project ID, which is visible in the GCP Console.
  3. Set the value of DATA_BACKEND to cloudsql.
  4. Set the value of CLOUDSQL_USER, CLOUDSQL_PASSWORD, and CLOUDSQL_DATABASE to the values you used when configuring the Cloud SQL instance. For simplicity, you can use the root user and root password configured while creating your instance.
  5. Set the value of CLOUDSQL_CONNECTION_NAME to the connection name for your Cloud SQL instance. This should be visible in the Google Cloud Platform Console in the details for your Cloud SQL instance. It should be in the format project:region:cloudsql-instance.
  6. Save and close config.py.

You need to additionally configure a setting in app.yaml before deploying:

  1. Open app.yaml for editing.
  2. Set the value of cloud_sql_instances to the same value used for CLOUDSQL_CONNECTION_NAME in config.py . It should be in the format project:region:cloudsql-instance. Uncomment this entire line.
  3. Save and close app.yaml.

Installing dependencies

Enter these commands to create a virtual environment and install dependencies:

Linux/Mac OS X

virtualenv -p python3 env
source env/bin/activate
pip install -r requirements.txt

Windows

virtualenv -p python3 env
env\scripts\activate
pip install -r requirements.txt

Creating the database tables

The application needs to create the database tables used to store the bookshelf data. This command will connect to the Cloud SQL instance and create all of the needed tables:

Linux/Mac OS X

python bookshelf/model_cloudsql.py

Windows

python bookshelf\model_cloudsql.py

Running the app on your local machine:

  1. Start a local web server:

    python main.py
    
  2. In your web browser, enter this address:

    http://localhost:8080

Now you can browse the app's web pages and add, edit, and delete books.

Press Control+C to exit the local web server.

Deploying the app to the App Engine flexible environment

  1. Deploy the sample app:

    gcloud app deploy
    
  2. In your web browser, enter this address. Replace [YOUR_PROJECT_ID] with your project ID:

    https://[YOUR_PROJECT_ID].appspot.com
    

If you update your app, you can deploy the updated version by entering the same command you used to deploy the app the first time. The new deployment creates a new version of your app and promotes it to the default version. The older versions of your app remain, as do their associated VM instances. Be aware that all of these app versions and VM instances are billable resources.

You can reduce costs by deleting the non-default versions of your app.

To delete an app version:

  1. In the GCP Console, go to the App Engine Versions page.

    Go to the Versions page

  2. Click the checkbox next to the non-default app version you want to delete.
  3. Click the Delete button at the top of the page to delete the app version.

For complete information about cleaning up billable resources, see the Cleaning up section in the final step of this tutorial.

Application structure

Bookshelf app deployment process and structure

The application stores all persistent data in Cloud SQL.

Understanding the code

This section walks you through the application code and explains how it works.

Handling user submissions with forms

The add/edit HTML form allows users to add and edit book submissions within the app.

Image of add/edit Form

The HTML form is created using Jinja2, which is a Python template engine. The following Jinja2 template specifies that the form include text input fields for Title, Author, Date Published, and Description:

{% extends "base.html" %}

{% block content %}
<h3>{{action}} book</h3>

<form method="POST" enctype="multipart/form-data">

  <div class="form-group">
    <label for="title">Title</label>
    <input type="text" name="title" id="title" value="{{book.title}}" class="form-control"/>
  </div>

  <div class="form-group">
    <label for="author">Author</label>
    <input type="text" name="author" id="author" value="{{book.author}}" class="form-control"/>
  </div>

  <div class="form-group">
    <label for="publishedDate">Date Published</label>
    <input type="text" name="publishedDate" id="publishedDate" value="{{book.publishedDate}}" class="form-control"/>
  </div>

  <div class="form-group">
    <label for="description">Description</label>
    <textarea name="description" id="description" class="form-control">{{book.description}}</textarea>
  </div>

  <button type="submit" class="btn btn-success">Save</button>
</form>

{% endblock %}

Handling form submissions

When a user clicks Add Book, the crud.add view displays the form. After the form is filled in and the user clicks Save, the same add function handles the form's HTTP POST action and starts the process of sending the submitted data to the Cloud SQL database by passing it to the model.create function.

@crud.route('/add', methods=['GET', 'POST'])
def add():
    if request.method == 'POST':
        data = request.form.to_dict(flat=True)

        book = get_model().create(data)

        return redirect(url_for('.view', id=book['id']))

    return render_template("form.html", action="Add", book={})

The bookshelf/model_cloudsql.py file contains the code that performs CRUD functions for data stored in the Cloud SQL database. The SQL is created using an Object Relational Mapper (ORM) called SQLAlchemy. Object relational mappers make it possible to interact with your data models as Python objects and have the equivalent SQL generated for you. Another popular ORM you might have worked with is the Django ORM.

To simplify integration with Flask, use the Flask extension Flask-SQLAlchemy.

For example, the create statement in the preceding code calls the create function in model_cloudsql.py, which then updates the data model by converting the attributes passed in as a Python dict into keyword arguments for the Book constructor. It then adds the model to the database session and commits the session. SQLAlchemy converts this to a SQL INSERT operation and creates a new book entry in the database.

def create(data):
    book = Book(**data)
    db.session.add(book)
    db.session.commit()
    return from_sql(book)

When a user edits a book's information after it has been submitted, the update function in model_cloudsql.py is called, which queries for the existing entry by its id field. Then it iterates through the updated fields and edits them in the existing model. Finally, the function commits the session, and in this case, SQLAlchemy converts the edit to a model as a SQL UPDATE operation.

def update(data, id):
    book = Book.query.get(id)
    for k, v in data.items():
        setattr(book, k, v)
    db.session.commit()
    return from_sql(book)

After users have added books, clicking the Books link navigates to the /books page, which lists all the books currently stored in the Cloud SQL database. The list function does the work of listing all the books by using data retrieved from the Cloud SQL database.

def list(limit=10, cursor=None):
    cursor = int(cursor) if cursor else 0
    query = (Book.query
             .order_by(Book.title)
             .limit(limit)
             .offset(cursor))
    books = builtin_list(map(from_sql, query.all()))
    next_page = cursor + limit if len(books) == limit else None
    return (books, next_page)

Here you can see how Python object methods can be converted to the appropriate SQL. In the preceding code, the limit method specifies the number of records to return from a database, which in turn adds a LIMIT clause to the SELECT SQL statement generated by the query. The offset method adds an OFFSET clause to the SELECT statement, which allows you to specify a starting point for handling pagination and allows additional pages to be requested based on the value of cursor.

Send feedback about...