Writing and viewing logs

Go Java Node.js PHP Python Ruby .NET Custom

This page describes the logs that are available for App Engine apps, and how to write and view log entries.

App Engine collects two types of logs:

• Request log: A log of the requests that are sent to your app. App Engine automatically creates entries in the request log.

• App log: log entries that you write to a supported framework or file as described on this page.

App Engine automatically sends these logs to the Cloud Logging agent, and you can view them in the Logs Viewer, on the command line, or programmatically.

Writing app logs

To write log entries, we recommend that you do the following:

Important: An app log entry must not exceed 16k bytes. Nothing exceeding this limit will be shown in the Logs Explorer.

Alternative: Writing structured logs to stdout and stderr

If data accuracy isn't as important, you can send simple text strings to stdout and stderr. The strings will appear as messages in the Logs Explorer, the command line, and the Cloud Logging API, and will be associated with the App Engine service and version that emitted them.

If you want to filter these strings in the Logs Explorer by severity level, you need to format them as structured data. For more information, see Structured logging.

If you want to correlate entries in the app log with the request log, your structured app log entries need to contain the request's trace identifier. You can extract the trace identifier from the X-Cloud-Trace-Context request header. In your structured log entry, write the ID to a field named logging.googleapis.com/trace. For more information about the X-Cloud-Trace-Context header, see Forcing a request to be traced. To view and search for correlated request and app logs, see View correlated log entries in the Logs Explorer.

See an example of writing structured log entries with a trace ID in the Cloud Run documentation. You can use the same technique in your App Engine apps.

Note that stdout and stderr logs from the flexible environment are sent to Cloud Logging using the legacy logging agent running in the virtual machine instances. The logging agent runs with limited resources and the logs that are written are rotated after exceeding a 10 MB limit. If the rate at which logs are written to stdout and stderr is greater than the rate at which logs are read by the logging agent, the logs can get rotated before the logs are sent to Cloud Logging. If you do not want to lose logs, use Cloud Logging.

Viewing logs

You can view your application and request logs using the Logs Explorer:

1. Go to the Logs Explorer in the Google Cloud console:

   Go to the Logs Explorer

2. Select an existing Google Cloud project at the top of the page.

3. From the drop-down menus, select GAE Application as your resource type.

4. To see which app log entries correlate to request log entries:

   1. Select the httpRequest.requestUrl log from the filter menus.

   2. Expand a request log entry. The app logs related to that request show up nested under the request log entry.

   If you send simple text entries to standard output, you cannot use the Logs Viewer to filter app entries by severity, nor can you see which app logs correspond to specific requests. You can still use other types of filtering in the Logs Explorer, such as text and timestamp.

You can filter the Logs Explorer by App Engine service and version and other criteria. You can also search the logs for specific entries. See Using the Logs Explorer for details.

Note: In older App Engine runtimes, a request log entry would inherit the highest severity level from its child entries. For example, if a request entry was correlated with a warning from your app log, the request entry would appear as a warning in the Logs Explorer. This inheritance feature is not available in newer runtimes. Instead, all entries maintain the severity level that was assigned when the entry was generated.

Viewing logs using the command line

To view your App Engine logs from the command line, use the following command:

gcloud app logs tail

For more information, see gcloud app logs tail.

Reading logs programmatically

If you want to read the logs programmatically, you can use one of these methods:

• Use a log sink to Pub/Sub and a script to pull from Pub/Sub.
• Call the Cloud Logging API through the client library for your programming language.
• Call the Cloud Logging API REST endpoints directly.

Pricing, quotas, and logs retention policy

For information about pricing that applies to both request and app logs, see pricing for Cloud Logging.

For the logs retention policy and the maximum size of log entries, see Quotas and limits. If you want to store your logs for a longer period, you can export your logs to Cloud Storage. You can also export your logs to BigQuery and Pub/Sub for further processing.

Managing log resource usage

You can control the amount of logging activity from your app logs by writing more or fewer entries from your app's code. Request logs are created automatically, so to manage the number of request log entries associated with your app, Use the logs exclusion feature from Cloud Logging.

Known issues

Sometimes app log entries are not correlated with the request log. This happens the first time your app receives a request and any other time App Engine writes status messages to your app's log. For more information, see https://issuetracker.google.com/issues/138365527.