Logging Application Events with Python

This part of the tutorial shows how an app can incorporate detailed logging to help with detecting, debugging, and monitoring potential issues. Logging application events can help you identify issues and solve problems, both during development and after your app is in production.

This page uses the Bookshelf application as its example app. To start from the beginning of the Bookshelf app, go to Python Bookshelf App. You can use the Python standard logging handlers introduced in Understanding the Code on App Engine Flexible, Google Compute Engine, or Google Kubernetes Engine.

Configuring settings

This section uses code in the 5-logging directory. Edit the files and run commands in this directory.

  1. Open config.py for editing.
  2. Set the value of PROJECT_ID to your project ID, which is visible in the GCP Console.
  3. Set DATA_BACKEND to the same value you used during the Using Structured Data tutorial.
  4. If you are using Cloud SQL or MongoDB, set the values under the Cloud SQL or Mongo section to the same values you used during the Using Structured Data step.
  5. Set the value CLOUD_STORAGE_BUCKET to your Cloud Storage bucket name.

  6. Under the OAuth2 configuration section, set the values of GOOGLE_OAUTH2_CLIENT_ID and GOOGLE_OAUTH2_CLIENT_SECRET to the application client ID and secret that you created previously.

  7. Save and close config.py.

If you are using Cloud SQL:

  1. Open app.yaml for editing.
  2. Set the value of cloud_sql_instances to the same value used for CLOUDSQL_CONNECTION_NAME in config.py . It should be in the format project:region:cloudsql-instance. Uncomment this entire line.
  3. Save and close app.yaml.

Installing dependencies

Enter these commands to create a virtual environment and install dependencies:

Linux/macOS

virtualenv -p python3 env
source env/bin/activate
pip install -r requirements.txt

Windows

virtualenv -p python3 env
env\scripts\activate
pip install -r requirements.txt

Running the app on your local machine:

  1. Start a local web server:

    python main.py
    
  2. In your web browser, enter this address:

    http://localhost:8080

Press Control+C to exit the worker and then the local web server.

Deploying the app to the App Engine flexible environment

  1. Deploy the sample app:

    gcloud app deploy
    
  2. In your web browser, enter this address. Replace [YOUR_PROJECT_ID] with your project ID:

    https://[YOUR_PROJECT_ID].appspot.com
    

If you update your app, you can deploy the updated version by entering the same command you used to deploy the app the first time. The new deployment creates a new version of your app and promotes it to the default version. The older versions of your app remain, as do their associated VM instances. Be aware that all of these app versions and VM instances are billable resources.

You can reduce costs by deleting the non-default versions of your app.

To delete an app version:

  1. In the GCP Console, go to the App Engine Versions page.

    Go to the Versions page

  2. Click the checkbox next to the non-default app version you want to delete.
  3. Click Delete delete at the top of the page to delete the app version.

For complete information about cleaning up billable resources, see the Cleaning up section in the final step of this tutorial.

Understanding the code

After you deploy an app, it's important to understand how well the app is working. Google Cloud Platform provides logging and monitoring tools built right into the Google Cloud Platform Console, so you can quickly get a powerful view into the detailed activity within your app, which can help you identify critical issues or trends.

The Bookshelf sample app uses Python's standard logging library to manage request and error logging.

The app can also log important events from anywhere by using the following code.

logging.info("Something happened");
logging.error("Something bad happened");

When the sample app is running in the flexible environment, anything logged to stderr and stdout is automatically collected by Stackdriver Logging and available for viewing, searching, and exporting in logs viewer in the GCP Console.

Viewing logs

After you deploy the sample app, logging data is collected. You can analyze this data with the log monitoring tools built right into the GCP Console. The following image shows logged events in the GCP Console.

Logging in the console

If you want more a detailed analysis, you can stream or import the app's logs into BigQuery or export them to a Google Cloud Storage bucket. You can do both right from within the GCP Console.

Using Python logging handlers

By default, the log messages captured in stdout and stderr are not captured with metadata such as severity. You can query for log statements that contain WARN, but the results will include any statements with the word WARN, regardless of the severity of the logging call. However, if the metatdata is correctly reported, you can filter for log messages of severity WARN with a metadata.severity >= WARN logging filter to return only log statements logged at WARN or above.

The Google Cloud Python SDK includes Python standard logging handlers, which ensure that all logging statements are reported to Stackdriver Logging with the correct metadata. The following code attaches the default Stackdriver Logging handlers, which vary depending on your deployment environment, to the root logger. In App Engine Flexible, Logging reports logs via the fluentd agent that accompanies the runtime. See the Cloud Python SDK docs for more information.

if not app.testing:
    client = google.cloud.logging.Client(app.config['PROJECT_ID'])
    # Attaches a Google Stackdriver logging handler to the root logger
    client.setup_logging(logging.INFO)
Was this page helpful? Let us know how we did:

Send feedback about...