Using Cloud logging in App Engine apps

Deepak Tiwari, Doug Anderson
November 2014

Google Cloud Logging provides facilities for importing, viewing, storing, and retrieving logs from applications running on the Google Cloud Platform.

What is available

Google Cloud Logging includes the following features for App Engine:

  • Request logs and application logs for App Engine applications.
  • Custom log files for applications using the flexible environment.
  • A Logs API (Python, Java, PHP, Go) for accessing logs.

Cloud Logging and the flexible environment

Applications using the flexible environment should write custom log files to the VM's log directory at /var/log/app_engine/custom_logs. These files are automatically collected and made available in the Logs Viewer.

Custom log files must have the suffix .log or .log.json. If the suffix is .log.json, the logs must be in JSON format with one JSON object per line. If the suffix is .log, log entries are treated as plain text.

Runtime logs in the flexible environment

Apps running in the flexible environment can write additional logs from the VM runtimes. Following are some of the logs that are configured by default:

Log name Payload type Purpose
crash.log text Information logged when setup fails.
monitoring.* text Information from the Google Docker container publishing data to Cloud Monitoring.
shutdown.log text Information logged on shutdown.
stdout text Standard output from your app.
stderr text Standard error from your container.
syslog text The VM syslog, outside of the Docker container.

Using the Logs Viewer

The Logs Viewer provides a web-based UI to navigate, display, and search your logs. With the Logs Viewer you can view and search logs from all your instances and apply filters to narrow in on a specific event, regardless of where it was generated.

To view logs:

  1. In the Google Cloud Platform Console, go to the Logging page.

  2. Ensure that App Engine is selected in the dropdown menu.

  3. If your app runs in the flexible environment and writes custom logs, select the desired log name.

Filtering and finding logs

You can choose to view only those logged events that are above a certain severity. Debug events are considered the lowest severity, Critical events are considered the highest. For instance, if you select a minimum severity of 'Warning', all warning, error, and critical log events will be displayed.

When viewing the Request log, you can combine field filters (for example, status:, protocol:, and so forth) and regular expressions in the same query, using the filter text box in the Logs Viewer. In addition, you can add and remove filters to help drill down and then zoom out again until you find what you’re looking for. Simply modify the query and press Enter to refresh the logs.

When you click the search bar, the Logs Viewer shows possible completions for filter fields as you type.

Filters of the same type are ORed together, while different filter types are ANDed together. So for example, status:400 status:500 regex:quota would produce all requests that returned HTTP status of either 400 OR 500, AND have the word quota in the log.

Field filters aren't yet supported for apps writing custom logs in the flexible environment, but you can use regular expressions to search custom logs.

Search or scroll through all of your logs

When you scroll through your logs in the new Logs Viewer, results are fetched until the console window is full. To retrieve additional logs that match the query, scroll down for newer results or up for older ones.

This provides you with a continuous view of your events to enable you to move forward and backward in time without requiring you to click Next or refresh the console. While results are being fetched you will see a Loading… indicator at the top or bottom of the viewer.

You can also click the refresh icon to load the latest logs, or click the play icon to live stream the logs for all of your logs from all of your instances.

Downloading logs from App Engine

For download information, see the log downloading instructions for Python, Java, or Go.

How to read the request log

The request log fields are documented in the Logs API pages for Python, Java, Go, and PHP.

Writing application logs from your app

Writing application logs is described in the Logs pages for Python, Java, PHP.

Send feedback about...