Deepak Tiwari, Doug Anderson
Google Stackdriver Logging provides facilities for importing, viewing, storing, and retrieving logs from applications running on the Google Cloud Platform.
What is available
Stackdriver Logging includes the following features for App Engine:
- Request logs and application logs for App Engine applications.
- A logs API for accessing logs of applications in the standard environment (Python, Java, PHP, Go).
Stackdriver Logging and the flexible environment
When a request is sent to your application running in App Engine, request and response details are logged automatically, and can be viewed in the Google Cloud Platform Console Logs Viewer.
When your application handles a request, it can also write its own logging
messages by using the appropriate Google Cloud Client library
or by writing to
stderr. These files are automatically collected
and can be viewed in the Logs Viewer. Only the most recent entries to
stderr are retained, in order to limit their size.
The request and application logs for your app are collected by a Cloud Logging agent and are kept for a maximum of 90 days, up to a maximum size of 1GB. If you want to store your logs for a longer period or store a larger size than 1GB, you can export your logs to Cloud Storage. You can also export your logs to BigQuery and Pub/Sub for further processing.
See below for other logs that you can use.
Runtime logs in the flexible environment
In addition to App Engine-generated request logs, apps running in the flexible environment can write additional logs. The following are some of the logs that are configured by default:
||Shows audit log information.|
||Shows nginx logs specifically for health checks. Reduces noise in the request logs.|
||Shows logs from the nginx proxy as they are passed through to the app.|
||Standard output from your app.|
||Standard error from your container.|
||Information logged when setup fails. If your application fails to run, check this log.|
||Information logged on shutdown.|
||The VM syslog, outside of the Docker container.|
Using the Logs Viewer
The Logs Viewer provides a web-based UI to navigate, display, and search your logs. With the Logs Viewer you can view and search logs from all your instances and apply filters to narrow in on a specific event, regardless of where it was generated.
To view logs:
In the Google Cloud Platform Console, go to the Logging page.
Ensure that App Engine is selected in the dropdown menu.
If your app runs in the flexible environment and writes custom logs, select the desired log name.
Filtering and finding logs
You can choose to view only those logged events that are above a certain severity. Debug events are considered the lowest severity, Critical events are considered the highest. For instance, if you select a minimum severity of 'Warning', all warning, error, and critical log events will be displayed.
When viewing the Request log,
you can combine field filters (for example,
protocol:, and so forth) and
regular expressions in the same query, using the filter text box in the
Logs Viewer. In addition, you can add and
remove filters to help drill down and then zoom out again until you find what
you’re looking for. Simply modify the query and press Enter to refresh the logs.
When you click the search bar, the Logs Viewer shows possible completions for filter fields as you type.
Filters of the same type are ORed together, while different
filter types are ANDed together. So for example,
status:400 status:500 regex:quota would produce all requests that returned
HTTP status of either
500, AND have the word
quota in the log.
Field filters aren't yet supported for apps writing custom logs in the flexible environment, but you can use regular expressions to search custom logs.
Search or scroll through all of your logs
When you scroll through your logs in the new Logs Viewer, results are fetched until the console window is full. To retrieve additional logs that match the query, scroll down for newer results or up for older ones.
This provides you with a continuous view of your events to enable you to move forward and backward in time without requiring you to click Next or refresh the console. While results are being fetched you will see a Loading… indicator at the top or bottom of the viewer.
You can also click the refresh icon to load the latest logs, or click the play icon to live stream the logs for all of your logs from all of your instances.