Using Stackdriver logging in App Engine apps

Deepak Tiwari, Doug Anderson
November 2014

Google Stackdriver Logging provides facilities for importing, viewing, storing, and retrieving logs from applications running on the Google Cloud Platform.

What is available

Stackdriver Logging includes the following features for App Engine:

  • Request logs and application logs for App Engine applications.
  • A logs API for accessing logs of applications in the standard environment (Python, Java, PHP, Go).

Stackdriver Logging and the flexible environment

When a request is sent to your application running in App Engine, request and response details are logged automatically, and can be viewed in the Google Cloud Platform Console Logs Viewer.

When your application handles a request, it can also write its own logging messages by using the appropriate Google Cloud Client library or by writing to stdout and stderr. These files are automatically collected and can be viewed in the Logs Viewer. Only the most recent entries to stdout and stderr are retained, in order to limit their size.

The request and application logs for your app are collected by a Cloud Logging agent and are kept for a maximum of 90 days, up to a maximum size of 1GB. If you want to store your logs for a longer period or store a larger size than 1GB, you can export your logs to Cloud Storage. You can also export your logs to BigQuery and Pub/Sub for further processing.

See below for other logs that you can use.

Runtime logs in the flexible environment

In addition to App Engine-generated request logs, apps running in the flexible environment can write additional logs. The following are some of the logs that are configured by default:

Log name Purpose
activity Shows audit log information.
nginx.health_check Shows nginx logs specifically for health checks. Reduces noise in the request logs.
nginx.request Shows logs from the nginx proxy as they are passed through to the app.
stdout Standard output from your app.
stderr Standard error from your container.
vm.crash Information logged when setup fails. If your application fails to run, check this log.
vm.shutdown Information logged on shutdown.
vm.syslog The VM syslog, outside of the Docker container.

Using the Logs Viewer

The Logs Viewer provides a web-based UI to navigate, display, and search your logs. With the Logs Viewer you can view and search logs from all your instances and apply filters to narrow in on a specific event, regardless of where it was generated.

To view logs:

  1. In the Google Cloud Platform Console, go to the Logging page.

  2. Ensure that App Engine is selected in the dropdown menu.

  3. If your app runs in the flexible environment and writes custom logs, select the desired log name.

Filtering and finding logs

You can choose to view only those logged events that are above a certain severity. Debug events are considered the lowest severity, Critical events are considered the highest. For instance, if you select a minimum severity of 'Warning', all warning, error, and critical log events will be displayed.

When viewing the Request log, you can combine field filters (for example, status:, protocol:, and so forth) and regular expressions in the same query, using the filter text box in the Logs Viewer. In addition, you can add and remove filters to help drill down and then zoom out again until you find what you’re looking for. Simply modify the query and press Enter to refresh the logs.

When you click the search bar, the Logs Viewer shows possible completions for filter fields as you type.

Filters of the same type are ORed together, while different filter types are ANDed together. So for example, status:400 status:500 regex:quota would produce all requests that returned HTTP status of either 400 OR 500, AND have the word quota in the log.

Field filters aren't yet supported for apps writing custom logs in the flexible environment, but you can use regular expressions to search custom logs.

Search or scroll through all of your logs

When you scroll through your logs in the new Logs Viewer, results are fetched until the console window is full. To retrieve additional logs that match the query, scroll down for newer results or up for older ones.

This provides you with a continuous view of your events to enable you to move forward and backward in time without requiring you to click Next or refresh the console. While results are being fetched you will see a Loading… indicator at the top or bottom of the viewer.

You can also click the refresh icon to load the latest logs, or click the play icon to live stream the logs for all of your logs from all of your instances.

Downloading logs from App Engine

For download information, see the log downloading instructions for Python, Java, or Go, and PHP.

How to read the request log

The request log fields are documented in the Logs API pages for Python, Java, Go, and PHP.

Writing application logs from your app

Writing application logs is described in the Logs pages for Python, Java, Go, and PHP.

Send feedback about...

App Engine Documentation