Priya Agrawal, Deepak Tiwari, Doug Anderson
Stackdriver Logging provides facilities for importing, viewing, storing, and retrieving logs from apps running on the Google Cloud Platform.
This page summarizes the logs available in Stackdriver Logging from App Engine.
What is available
Stackdriver Logging includes the following features for App Engine:
- Request logs, app logs, shutdown logs, runtime logs, and audit logs for App Engine apps.
- A logs API for accessing logs of apps in the standard environment (Python, Java, PHP, Go).
App Engine writes Admin Activity audit logs,
activity in the Logs Viewer.
Stackdriver Logging and the standard environment
The App Engine standard environment produces the following logs:
request_login the Logs Viewer. This log records requests sent to all App Engine apps. The request log is provided by default and you cannot opt out of receiving it. For more details, see the
App logs consist of additional app information included in the request logs. The information is provided by default and you cannot opt out of receiving it. For more details, see the
Third-party package logs. Some third-party package logs are also produced, including
nginx.requestin the Logs Viewer.
Log entries from App Engine use the resource type
Fields in request logs
For the App Engine standard environment request log, the following predefined
names can be used as Logs Viewer field labels; see
Field path identifiers for
All fields contain string values except for
||The number of bytes in the response to the client. Example:
||The HTTP method. Example:
||The path appended to your host's address. Example:
||The protocol used. Example:
||The query string on the request, if any. Example:
||The host making the request. Example:
||The RFC 1413 client ID.|
||The numeric status code returned to the client: Example:
||An authorized user, if any.|
||The browser or other agent making the requests. Example:
Stackdriver Logging and the flexible environment
When a request is sent to your app running in App Engine, request and response details are logged automatically, and can be viewed in the Google Cloud Platform Console Logs Viewer.
When your app handles a request, it can also write its own logging
messages by using the appropriate Google Cloud Client library
or by writing to
stderr. These files are automatically collected
and can be viewed in the Logs Viewer. Only the most recent entries to
stderr are retained, in order to limit their size.
The App Engine flexible environment produces the following logs:
Request logs record requests sent to all App Engine apps. The request log is provided by default and you cannot opt out of receiving it.
App logs record activity by software within the App Engine app. The log is provided by default and you cannot opt out of receiving it.
Runtime logs are provided from the flexible environment using a preinstalled Logging agent.
Logs from the flexible environment include resources of type
App logs and request logs
Request and app logs are stored separately in the flexible environment.
|Log type||Payload type||Purpose|
||text||Requests made to your apps in the flexible environment.|
||text or structured||App logs entries from your apps in the flexible environment.|
Linking app logs and requests
In the App Engine flexible environment, app logs can be correlated with a particular request. Following are the requirements:
The "parent" log entry must include a value for
The "child" log entry must have a different log name than the parent. Typically, the child log entry is an app log entry in the
applog, whereas the parent log entry is in the
requestlog. The values for
tracemust be the same in the parent and its child(ren).
The child log entry's
timestampmust be within the time interval covered by the parent request. Note that the parent log entry is written at the end of the request which means that the interval is [
parent.timestamp - parent.httpRequest.latency,
parent.timestamp]. Also, some buffer is added to allow for clock skew between the request and app logs.
For more information on log entries, see the LogEntry data type in Stackdriver Logging.
The App Engine flexible environment can write additional logs from the runtimes.
Following are some of the logs that are configured by default:
||Shows audit log information.|
||Information from the container where your app is running.|
||Shows errors surfaced from within the nginx process.|
||Shows nginx logs specifically for health checks. Reduces noise in the request logs.|
||Shows logs from the nginx proxy as they are passed through to the app container.|
||Information logged on shutdown.|
||Standard output from your app container.|
||Standard error from your app container.|
||Operational events related to instances.|
||Information logged on shutdown.|
||The VM syslog, outside of the app container.|
VM events logging
You can gain insight into events related to your App Engineflexible
instances using the
vm.events stream in Logging. The following events are
logged in this stream:
- Instance creation
- Instance termination (and why, reported on a best-effort basis)
- Instance health check aggregate statistics reported approximately every minute
- Instance unlocked for debugging or instance was accessed via SSH
Note that the
vm.events stream may not report events in chronological order.
You can get diagnostic information when your app container is stopped. For example, this might happen if an app is unresponsive due to a deadlock or high load and has stopped returning requests, including health checks.
To help diagnose issues, you can get debug information, such as thread traces
and heap statistics, from the app container through
vm.shutdown, there are logs from Docker containers including the app
container and other sidecar containers, such as nginx, for both healthy and
If your app is unhealthy, you can get other logs in
vm.shutdown such as:
- Tail of app logs
- Additional debug information: machine type, memory and disk usage, etc.
In the Java runtime, shutdown logging for the app container is built into the JVM. For more information, see the documentation.
Using the Logs Viewer
The Logs Viewer provides a web-based UI to navigate, display, and search your logs. With the Logs Viewer you can view and search logs from all your instances and apply filters to narrow in on a specific event, regardless of where it was generated.
To view logs:
In the Google Cloud Platform Console, go to the Logging page.
Ensure that App Engine is selected in the dropdown menu.
Filtering and finding logs
You can choose to view only those logged events that are above a certain severity. Debug events are considered the lowest severity, Critical events are considered the highest. For instance, if you select a minimum severity of 'Warning', all warning, error, and critical log events will be displayed.
When viewing the Request log,
you can combine field filters (for example,
protocol:, and so forth) and
regular expressions in the same query, using the filter text box in the
Logs Viewer. In addition, you can add and
remove filters to help drill down and then zoom out again until you find what
you’re looking for. Simply modify the query and press Enter to refresh the logs.
When you click the search bar, the Logs Viewer shows possible completions for filter fields as you type.
Filters of the same type are ORed together, while different
filter types are ANDed together. So for example,
status:400 status:500 regex:quota would produce all requests that returned
HTTP status of either
500, AND have the word
quota in the log.
Search or scroll through all of your logs
When you scroll through your logs in the new Logs Viewer, results are fetched until the console window is full. To retrieve additional logs that match the query, scroll down for newer results or up for older ones.
This provides you with a continuous view of your events to enable you to move forward and backward in time without requiring you to click Next or refresh the console. While results are being fetched you will see a Loading… indicator at the top or bottom of the viewer.
You can also click the refresh icon to load the latest logs, or click the play icon to live stream the logs for all of your logs from all of your instances.