Migrating to Cloud Logging

With just a few lines of code, you can update your app to use Cloud Logging and achieve nearly the same filtering and log correlation features that were available with the App Engine logging service.

To migrate to Cloud Logging, attach the Cloud Logging handler to the Python root logger, then continue using the Python logging module to write log entries.

Key differences between App Engine logging and Cloud Logging

App Engine logging requires you use the Python logging module to write app log entries. Cloud Logging supports all of the following approaches:
- Use the Cloud Logging client library directly.
- Write entries to stdout or stderr. These entries appear in the Logs Explorer, but to enable filtering and correlation with request logs you need to format the entries as a JSON object and provide specific metadata. For more information about this approach, see Structured logging.
With App Engine logging, the Logs Explorer assigns a severity level to request logs, and the severity level reflects the highest severity of any app log entry that is correlated with the request. For example, if a request results in your app emitting a warning log entry, the Logs Explorer will display a warning icon next to the request log entry. When you expand the request entry, you see the warning log entry nested within the request entry.

With Cloud Logging, the Log Viewer does not assign a severity level to request logs. When you expand a request entry, you'll see app entries nested within the request entry, and those app entries will show a severity icon. But the request entry itself does not show a severity icon and cannot be filtered by severity level.
Quotas and data retention are the same for both App Engine and Cloud Logging, but if you exceed the free quota of logs data, Cloud Logging pricing may be different from App Engine logs pricing.

Before you start migrating

If you have not done so already, set up your Python development environment to use a Python version that is compatible with Google Cloud, and install testing tools for creating isolated Python environments.
Enable the Cloud Logging API in the project that contains your app.

Enable the API
Make sure your app has permission to write logs.

By default, your app's default service account has permission to write logs.

If your app uses a different service account or user account, or if you have changed permissions for the default service account, make sure the account you use has permission to write logs.

Overview of the migration process

To migrate your Python app to use Cloud Logging instead of App Engine logging:

Install the Cloud Client Library for Cloud Logging.
Write logs with the standard Python logging module.
Deploy your app to App Engine and confirm logging works as expected.

Installing the Cloud Client Library for Cloud Logging

Install and update your configuration files:

Update the app.yaml file. Follow the instructions for your version of Python:
Python 2
For Python 2 apps, add the latest versions of grpcio, setuptools, and ssl libraries.

The following is an example app.yaml file:
```
runtime: python27
threadsafe: yes
api_version: 1

libraries:
- name: grpcio
  version: latest
- name: setuptools
  version: latest
- name: ssl
   version: latest
```
Some client libraries don't need the SSL library. If you don't include the SSL library for a client library that needs it, you'll see an SSL error in the Logs Explorer when your app receives a request.
Python 3
For Python 3 apps, delete all lines except for the runtime element. For example:
```
runtime: python310 # or another support version
```
The Python 3 runtime installs libraries automatically, so you do not need to specify built-in libraries from the previous Python 2 runtime. If your Python 3 app is using other legacy bundled services when migrating, leave the app.yaml file as is.
Update the requirements.txt file. Follow the instructions for your version of Python:
Python 2
Add the Cloud Client Libraries for Cloud Logging to your list of dependencies in the requirements.txt file.
```
google-cloud-logging==1.14.0
```
We recommend you use the 1.14.0 version of the Cloud Logging client library since it supports Python 2.7 apps.

Then run pip install -t lib -r requirements.txt to update the list of available libraries for your app.
Python 3
Add the Cloud Client Libraries for Cloud Logging to your list of dependencies in the requirements.txt file.
```
google-cloud-logging
```
App Engine automatically installs these dependencies during app deployment in the Python 3 runtime, so delete the lib folder if one exists.
For Python 2 apps, if your app is using built-in or copied libraries specified in the lib directory, you must specify those paths in the appengine_config.py file:
```
import pkg_resources
from google.appengine.ext import vendor

# Set PATH to your libraries folder.
PATH = 'lib'
# Add libraries installed in the PATH folder.
vendor.add(PATH)
# Add libraries to pkg_resources working set to find the distribution.
pkg_resources.working_set.add_entry(PATH)
```
The appengine_config.py file in the preceding example assumes that the the lib folder is located in the current working directory. If you can't guarantee that lib will always be in the current working directory, specify the full path to the lib folder. For example:
```
import os
path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')
```

Write logs with the standard Python logging module

In each file that writes log entries:

Import the Cloud Logging client library.
Instantiate the Cloud Logging client.
Attach the Cloud Logging handler to the Python root logger by calling the setup_logging method.
Run the Cloud Logging client's setup_logging() method, which attaches its default listener as the logging handler for the Python root logger.

For example:

samples/snippets/handler.py

View on GitHub

# Imports the Cloud Logging client library
import google.cloud.logging

# Instantiates a client
client = google.cloud.logging.Client()

# Retrieves a Cloud Logging handler based on the environment
# you're running in and integrates the handler with the
# Python logging module. By default this captures all logs
# at INFO level and higher
client.setup_logging()

Once the handler is attached, any logs at, by default, INFO level or higher which are emitted in your application will be sent to Logging: