With just a few lines of code, you can update your app to use Cloud Logging and achieve nearly the same filtering and log correlation features that were available with the App Engine logging service.
To migrate to Cloud Logging, attach the
log handler to the Python logging module, then continue using the Python logging
module to write log entries. You can make these updates in your Python 2 app and
see the same logging behavior when you app runs in either the Python 2 or
Python 3 runtimes.
Key differences between App Engine logging and Cloud Logging
App Engine logging requires you use the Python logging module to write app log entries. Cloud Logging supports all of the following approaches:
(Recommended) Use the Python logging module with
AppEngineHandleras the logging handler. This idiomatic approach only requires a few additional lines of code to register the Cloud Logging handler, and therefore makes your app more portable to other Python environments.
Write entries to
stderr. These entries appear in the Logs Explorer, but to enable filtering and correlation with request logs you need to format the entries as a JSON object and provide specific metadata. For more information about this approach, see Structured logging.
Use the Cloud Logging client library directly. If you use the
AppEngineHandler, this approach achieves the same result as using the Python logging module, but changing from this library to a different logging framework could require significant updates to your code.
With App Engine logging, the Logs Explorer assigns a severity level to request logs, and the severity level reflects the highest severity of any app log entry that is correlated with the request. For example, if a request results in your app emitting a warning log entry, the Logs Explorer will display a warning icon next to the request log entry. When you expand the request entry, you see the warning log entry nested within the request entry.
With Cloud Logging, the Log Viewer does not assign a severity level to request logs. When you expand a request entry, you'll see app entries nested within the request entry, and those app entries will show a severity icon. But the request entry itself does not show a severity icon and cannot be filtered by severity level.
Before you start migrating
Enable the Cloud Logging API in the project that contains your app.
Make sure your app has permission to write logs.
By default, your app's default service account has permission to write logs.
Overview of the migration process
To migrate your Python app to use Cloud Logging instead of App Engine logging:
Install the Cloud Client Library for Cloud Logging.
AppEngineHandlerto the Python logging module.
Deploy your app to App Engine and confirm logging works as expected.
Installing the Cloud Client Library for Cloud Logging
AppEngineHandler available to your app when it runs in App Engine:
requirements.txtfile in the same folder as your
app.yamlfile and add the following lines:
We recommend you use the 1.14.0 version of the Cloud Logging client library since it supports Python 2.7 apps.
In your app's
app.yamlfile, specify the required Google RPC and
setuptoolslibraries, and the optional SSL library in the
libraries: - name: grpcio version: 1.0.0 - name: setuptools version: 36.6.0 - name: ssl version: latest
Some client libraries don't need the SSL library. If you don't include the SSL library for a client library that needs it, you'll see an SSL error in the Logs Explorer when your app receives a request.
Create a directory to store your third-party libraries, such as
lib/. Then use
pip installto install the libraries into the directory. For example:
pip install -t lib -r requirements.txt
appengine_config.pyfile in the same folder as your
app.yamlfile. Add the following to your
# appengine_config.py import pkg_resources from google.appengine.ext import vendor # Set path to your libraries folder. path = 'lib' # Add libraries installed in the path folder. vendor.add(path) # Add libraries to pkg_resources working set to find the distribution. pkg_resources.working_set.add_entry(path)
appengine_config.pyfile in the preceding example assumes that the the
libfolder is located in the current working directory. If you can't guarantee that
libwill always be in the current working directory, specify the full path to the
libfolder. For example:
import os path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')
For local development, we recommend that you install dependencies in a virtual environment such as virtualenv for Python 2.
AppEngineHandler to the Python logging module
In each file that writes log entries:
- Import the Cloud Logging client library.
- Instantiate the Cloud Logging client.
Retrieve the Cloud Logging handler that is appropriate for the current environment. When your app runs in App Engine, the Cloud Logging client retrieves
AppEngineHandlerwill format the entries as a JSON object and provide the metadata needed to support filtering by severity and correlating your app's entries with request log entries.
Run the Cloud Logging client's
setup_logging()method, which attaches its default listener as the logging handler for the Python root logger.
# Imports the Cloud Logging client library import google.cloud.logging # Instantiates a client client = google.cloud.logging.Client() # Retrieves a Cloud Logging handler based on the environment # you're running in and integrates the handler with the # Python logging module. By default this captures all logs # at INFO level and higher client.setup_logging()
Once the handler is attached, any logs at, by default,
INFO level or higher
which are emitted in your application will be sent to Logging:
# Imports Python standard library logging import logging # The data to log text = "Hello, world!" # Emits the data using the standard logging module logging.warning(text)
Deploying your app
Once you are ready to deploy your app, you should:
The only way to test your updates and confirm that severity levels are assigned and app entries are correlated with request entries is to deploy your app to the Python 3 runtime.
If you deploy your updates to the Python 2 runtime, the App Engine logging service will listen for entries from the Python root logger and format them appropriately for the Logs Explorer.
If you run your app locally, Cloud Logging does not use
AppEngineHandler, and App Engine cannot write request entries.
View the Logs Explorer and confirm that severity levels are assigned to your app's entries. Expand a request entry and confirm that entries written by your app while it was processing the request appear nested below the request entry.
If the app runs without errors, use traffic splitting to slowly ramp up traffic for your updated app. Monitor the app closely for any issues before routing more traffic to the updated app.